Advertisement
Advanced Time: 3–4 weeks IT & Networking

Self-Hosted Email Server

Set up a production-ready self-hosted email server with Postfix, Dovecot, anti-spam, DKIM/SPF/DMARC, and webmail.

EmailPostfixDovecotDKIMSPFDMARCPrivacy
DifficultyAdvanced
Duration3–4 weeks
Components10 items
Steps3 steps

Introduction

Set up a production-ready self-hosted email server with Postfix, Dovecot, anti-spam, DKIM/SPF/DMARC, and webmail. This comprehensive guide covers everything from design through implementation, testing, and deployment.

Theory & Background

Before sending any mail: set all DNS records. MX record: @ IN MX 10 mail.catb.in (points to your server). A record: mail.catb.in → your_server_ip. PTR record (reverse DNS): ask VPS provider to set IP → mail.catb.in. SPF TXT record: v=spf1 mx a ~all. DKIM: generate keys with OpenDKIM, add TXT record at selector._domainkey.catb.in. DMARC TXT record: _dmarc.catb.in v=DMARC1; p=reject; rua=mailto:dmarc@catb.in. Without these records, your email will be marked as spam or rejected.

Advertisement

Components & Requirements

10 components required for this project.

#ComponentPurposeQty
1VPS (Ubuntu 22.04, static IP, reverse DNS)Email server hostx1
2Domain name (catb.in)Email domainx1
3Postfix MTAMail Transfer Agent (sending/receiving)x1
4Dovecot IMAP serverMail delivery and IMAP/POP3x1
5SpamAssassinSpam filteringx1
6ClamAVEmail virus scanningx1
7OpenDKIMDKIM email signingx1
8Roundcube WebmailBrowser-based email clientx1
9Rspamd (alternative spam filter)Modern, faster spam filterx1
10MailHog (testing)Email testing without sending real mailx1

Step-by-Step Implementation

Follow these 3 steps carefully.

1
DNS Records for Email (Critical)

Before sending any mail: set all DNS records. MX record: @ IN MX 10 mail.catb.in (points to your server). A record: mail.catb.in → your_server_ip. PTR record (reverse DNS): ask VPS provider to set IP → mail.catb.in. SPF TXT record: v=spf1 mx a ~all. DKIM: generate keys with OpenDKIM, add TXT record at selector._domainkey.catb.in. DMARC TXT record: _dmarc.catb.in v=DMARC1; p=reject; rua=mailto:dmarc@catb.in. Without these records, your email will be marked as spam or rejected.

2
Postfix and Dovecot Configuration

Postfix (MTA): handles sending and receiving SMTP. Configure main.cf: myhostname=mail.catb.in, mydomain=catb.in, mynetworks=127.0.0.1 (only allow local relay), smtpd_tls_cert_file=Let's Encrypt cert (TLS required), smtpd_sasl_auth_enable=yes (SMTP authentication with Dovecot SASL). Dovecot (MDA): handles IMAP/POP3, delivers mail to mailboxes. Configure mail_location, ssl_cert/key, auth mechanisms (PLAIN over TLS only), userdb/passdb (system users or MySQL).

3
Anti-Spam and Email Security

Rspamd: modern spam filter with machine learning. Actions: reject (very high spam score), add spam header (medium score), accept (low score). DKIM signing: all outbound mail signed with private key — recipients can verify authenticity. SPF: receiving servers check if your IP is authorized to send for your domain. DMARC: tells receivers what to do if SPF/DKIM fail (reject/quarantine/none) and where to send aggregate reports. Check score on mail-tester.com (aim for 10/10).

Code & Implementation

Core code for email_server_test.sh:

email_server_test.sh Shell
#!/bin/bash # Email server configuration verification  DOMAIN="catb.in" MAIL_HOST="mail.catb.in"  echo "=== DNS Records Check ===" echo "MX record:"    ; dig $DOMAIN MX +short echo "SPF record:"   ; dig $DOMAIN TXT +short | grep spf echo "DMARC record:" ; dig _dmarc.$DOMAIN TXT +short echo "PTR record:"   ; dig -x $(dig $MAIL_HOST A +short | head -1) +short  echo "\\n=== Port Checks ===" nc -zv $MAIL_HOST 25  2>&1 | grep -c "succeeded" && echo "SMTP (25): OPEN" || echo "SMTP (25): CLOSED" nc -zv $MAIL_HOST 587 2>&1 | grep -c "succeeded" && echo "SUBMISSION (587): OPEN" nc -zv $MAIL_HOST 993 2>&1 | grep -c "succeeded" && echo "IMAPS (993): OPEN"  echo "\\n=== TLS Certificate ===" echo | openssl s_client -connect $MAIL_HOST:993 2>/dev/null | openssl x509 -noout -dates  echo "\\n=== Test SMTP Connection ===" telnet $MAIL_HOST 25

Testing & Troubleshooting

Test Self-Hosted Email Server by verifying each subsystem individually before full integration.

!
Troubleshooting Tips

Verify power voltages, check ground connections, use serial monitor for debug.

Real-World Applications

*Private family email domain
*Business email independence from Google/Microsoft
*Privacy-focused email communication
*Email list management for newsletter
*Development email testing server
*Corporate email compliance and archive
*Domain-specific email aliases
*Transactional email sending for web apps

Extensions & Next Steps

  • Add email encryption with S/MIME or PGP auto-key exchange
  • Implement email list manager (Mailman) for newsletters
  • Build an AI-powered spam classifier using message history
  • Add email archiving with search (Elasticsearch)
  • Implement email delivery analytics dashboard

Interactive Playground

Coming Soon

An interactive simulator will be available here — simulate circuits and run code in-browser without hardware.

Frequently Asked Questions

Why do major providers (Gmail, Outlook) sometimes block email from self-hosted servers?
Email reputation: major providers maintain IP reputation databases and apply machine learning to detect spam sources. New IP addresses have no reputation — treated with suspicion. Getting blocklisted: common for new servers even with perfect configuration. Getting delisted: submit deblocking request to each provider (Google Postmaster Tools, Microsoft JMRP/SNDS). Factors affecting reputation: IP age (new IPs start with neutral reputation), volume (don't send bulk mail initially), content (avoid spam-like subject lines), bounce rate (high bounces indicate purchased lists).
Advertisement