MTA-STS, TLS-RPT & DANE: Complete Guide to Email Transport Security

SPF, DKIM, and DMARC authenticate who sent your email and prevent domain spoofing. But they do not protect the email itself while it travels between mail servers. Every time your message passes from one SMTP server to another, that connection relies on STARTTLS — an optional encryption mechanism that can be stripped by a man-in-the-middle attacker, causing your email to travel in plaintext across the internet.

MTA-STS, TLS-RPT, and DANE are the three protocols that close this gap by making TLS encryption mandatory rather than optional. Together they form the transport security layer of a complete email security stack — the layer that ensures your authentication-verified message also travels encrypted end to end.

The Problem: STARTTLS, Opportunistic Encryption, and Downgrade Attacks

SMTP was designed in 1982 without encryption. When STARTTLS was added in 1999, it was implemented as opportunistic encryption: the sending server offers to upgrade the connection to TLS, and if the receiving server advertises 250 STARTTLS in its EHLO response, the upgrade happens. If the receiving server does not advertise STARTTLS — or if an attacker intercepts the EHLO response and removes the STARTTLS advertisement — the connection falls back to plaintext.

This creates a STARTTLS downgrade attack: an attacker positioned between two mail servers intercepts the EHLO exchange, strips the 250 STARTTLS capability from the response, and the sending server delivers the email in plaintext to the attacker’s controlled server. The sending server has no way to know the EHLO response was tampered with. The attack is invisible, leaves no delivery error, and the email reaches the recipient normally — but its contents have been exposed in transit.

STARTTLS also does not validate the TLS certificate presented by the receiving server. The sending server will accept any certificate — expired, self-signed, belonging to a different hostname — as long as a TLS connection can be established. This allows a man-in-the-middle to present their own certificate and intercept the encrypted connection.

MTA-STS addresses both vulnerabilities by making TLS mandatory and specifying which mail servers and certificates are valid for a domain.

MTA-STS: How It Works and How to Deploy It

MTA-STS (Mail Transfer Agent Strict Transport Security, defined in RFC 8461) enables receiving domains to publish a policy declaring: “Only deliver email to my mail servers over a validated TLS connection. If you cannot establish a valid TLS connection, do not deliver the message at all.”

The protocol works through two components: a DNS TXT record that signals MTA-STS support, and an HTTPS-hosted policy file that specifies the exact requirements.

Step 1: Create the MTA-STS Policy File

The policy file must be hosted at https://mta-sts.yourdomain.com/.well-known/mta-sts.txt — that exact path and subdomain. The file content:

version: STSv1
mode: testing
mx: mail.yourdomain.com
mx: mail2.yourdomain.com
max_age: 604800

The three policy modes:

• none: MTA-STS exists but provides no enforcement. Use this only during initial DNS setup verification.
• testing: Sending servers attempt TLS validation and generate TLS-RPT reports on failures, but still deliver even if TLS validation fails. This is the correct phase for initial deployment — it gives you visibility without risking mail loss.
• enforce: Sending servers must establish validated TLS to all listed MX servers. If they cannot, the message is not delivered. Only move to this mode after reviewing TLS-RPT reports and confirming clean operation.

The mx field must list every MX hostname your domain uses. Any MX server not listed will be unreachable when enforce mode is active.

Step 2: Publish the DNS TXT Record

Create a TXT record at _mta-sts.yourdomain.com:

v=STSv1; id=20260320001

The id value must change whenever you update the policy file. Sending servers cache the MTA-STS policy; incrementing the ID signals them to fetch the updated policy. A date-based ID (YYYYMMDDNNN) works well in practice.

Step 3: TLS-RPT Configuration (Required Before Enforcing)

TLS-RPT (SMTP TLS Reporting, RFC 8460) sends you daily JSON reports on TLS connection failures and successes. Deploying TLS-RPT before switching to enforce mode is essential — it reveals misconfigurations before they cause mail loss.

Add a DNS TXT record at _smtp._tls.yourdomain.com:

v=TLSRPTv1; rua=mailto:tls-reports@yourdomain.com

TLS-RPT reports include: which sending MTAs are connecting, which MX servers they’re connecting to, whether TLS is succeeding or failing, and failure reasons (certificate errors, protocol mismatches, STARTTLS not offered). This data lets you identify any MX server with TLS configuration problems before enforcement makes failures permanent.

Step 4: Move to Enforce Mode

After 4 – 6 weeks in testing mode with clean TLS-RPT reports, update the policy file to mode: enforce and increment the DNS record ID. The NCSC (UK’s National Cyber Security Centre) guidance recommends monitoring TLS-RPT reports for at least one month in testing mode before enforcing.

When switching to enforce mode:

1. Update the policy file from mode: testing to mode: enforce
2. Increment the DNS record ID (e.g., id=20260415001)
3. Verify the HTTPS endpoint is serving the updated file correctly
4. Monitor TLS-RPT reports for any enforcement failures in the first 48 hours

TLS-RPT: Reading and Acting on Reports

TLS-RPT delivers daily JSON reports that, in raw form, are not human-readable. The reports contain structured data about connection attempts, successes, and failures. Commercial DMARC platforms (PowerDMARC, Red Sift OnDMARC, EasyDMARC) typically parse and display TLS-RPT alongside DMARC aggregate reports. If you’re processing reports manually, basic JSON parsing to extract failure reasons and counts is the minimum viable approach.

The key fields to monitor in TLS-RPT:

• result-type and failure-reason-code — what specifically failed
• failed-session-count — how many messages affected
• receiving-mx-hostname — which of your MX servers is experiencing failures

Common failure reasons and their remediation:

• starttls-not-offered: One of your MX servers does not advertise STARTTLS. Either the service isn’t configured for TLS, or a firewall is stripping the STARTTLS advertisement. Fix the MX server’s TLS configuration or exclude it from the MTA-STS policy.
• certificate-expired: TLS certificate on an MX server has expired. Renew immediately — or switch to Let’s Encrypt with auto-renewal to prevent recurrence.
• certificate-not-for-domain: Certificate CN/SAN does not match the MX hostname. Reissue the certificate with the correct hostname.
• validation-failure: Certificate is invalid or self-signed. Replace with a certificate from a trusted CA.

DANE: The Cryptographic Alternative to MTA-STS

DANE (DNS-Based Authentication of Named Entities, RFC 7671) provides the same protection as MTA-STS but through DNSSEC rather than HTTPS. Instead of hosting a policy file at an HTTPS endpoint, DANE publishes TLSA records in DNS that specify the exact TLS certificate (or certificate fingerprint) expected for each mail server. Sending servers validate the certificate by checking it against the TLSA record in DNSSEC-signed DNS data.

DANE’s advantages over MTA-STS:

• No HTTPS hosting required — everything is DNS
• Certificate pinning is more specific — you specify the exact certificate or CA
• Theoretically harder to attack since DNSSEC provides cryptographic DNS integrity

DANE’s disadvantages vs MTA-STS:

• Requires DNSSEC on the receiving domain — DNSSEC adoption is still limited (some domain registrars and DNS providers do not support it)
• Certificate rotation requires updating TLSA records in advance — requires careful key management
• Less broadly supported by sending MTAs than MTA-STS

Most organizations find MTA-STS easier to deploy than DANE because it does not require DNSSEC. The two protocols can coexist — for domains that have both DANE TLSA records and MTA-STS deployed, sending servers typically validate using DANE when available (DNSSEC-signed TLSA records provide stronger guarantees) and fall back to MTA-STS.

MTA-STS and Microsoft 365

Microsoft added MTA-STS outbound enforcement to Exchange Online, meaning Exchange Online will validate MTA-STS policies when delivering to recipient domains that have MTA-STS deployed. This is significant — it means if you have MTA-STS in enforce mode, Exchange Online (which handles a substantial portion of commercial email) will not deliver to your domain except over validated TLS.

For inbound protection on Microsoft 365 tenants, Microsoft updated their MX record infrastructure to new mx.microsoft endpoints that support inbound MTA-STS enforcement. Tenants that update their MX records to the new format gain inbound MTA-STS protection for their Microsoft-hosted mailboxes.

Where MTA-STS Fits in the Complete Email Security Stack

MTA-STS, TLS-RPT, and DANE address transport-layer security — distinct from the sender authentication addressed by SPF, DKIM, and DMARC. The complete stack:

Deploying MTA-STS without SPF/DKIM/DMARC is like encrypting a letter but not sealing the envelope — you’ve addressed one threat while leaving others open. Deploying SPF/DKIM/DMARC without MTA-STS means you’ve proved who you are but cannot guarantee the message travels to the recipient securely. Both layers are needed for complete email security coverage.