Invalid DKIM Signature Due to Missing 8BITMIME

Why DKIM signatures can become invalid when 8BITMIME is missing, and which workarounds are available

The Theory

Anyone who runs a mail server these days has to implement DKIM, SPF, and DMARC. These techniques help identify fake email senders more easily and therefore also to reduce spam and phishing.

The relevant information is stored in the domain’s DNS record. Simply put, an SPF record (Sender Policy Framework) defines which mail servers are authorized to send emails on behalf of the domain. This is intended, for example, to prevent a third party from impersonating a bank or a parcel service and stealing customer data.

Hint: SPF records are not automatically inherited by all subdomains. Each subdomain requires its own record.

With DKIM (DomainKeys Identified Mail), to put it simply, a cryptographic key is stored that must be used to invisibly digitally sign the emails. This makes it possible to determine whether the messages were tampered with during transmission. At the same time, this also makes it more difficult for a customer to impersonate another customer in large email systems with thousands of domains, since the IP address would still be correct in such case.

There are many reasons why one of these mechanisms might fail: forwarded emails are usually sent via a different server, so SPF does not validate, and some mailing lists modify the subject line of each message, which breaks the DKIM signature.

The two mechanisms are therefore combined via DMARC (Domain-based Message Authentication, Reporting, and Conformance). Postmasters use it to configure, among other things, how to handle emails that fail the DMARC check.

Tip: DMARC records are automatically inherited by all subdomains unless they have their own record.

The DMARC check, in turn, passes if one of the two mechanisms – SPF or DKIM – validates. Forwarded emails are not rejected as long as the DKIM signature is still good, and messages with a missing or broken signature are delivered as long as the sending mail server is listed in the SPF record.

The Practice

Apart from from intentional modifications of the email, there are other reasons why the DKIM signature can become invalid, as I learned a while back. My emails to a specific recipient bounced back with the following message:

550 DKIM Sender Invalid - envelope rejected

At first, I assumed the problem was a missing DKIM signature, a missing DNS record, issues with the DNS server itself, or a DKIM key that was too long. However, various test messages sent to other eail providers were successfully delivered, and the DKIM signature was shown as valid. Several test sites, such as DKIMValidator.com or mail-tester.com, also showed no problems.

After extensive research and thanks to several very helpful tips from the Mailop mailing list, to which the operator of AboutMy.email is also subscribed, the problem slowly became clearer. An email sent to AboutMy.email did indeed fail and resulted in an invalid DKIM signature – provided the message contained special characters and was sent using certain mail clients. Emails from other mail clients worked without any problems, even with special characters.

8-bit is the Issue

The problem was the use of 8-bit characters. Simply put, there are two ways to represent special characters in an email: natively as 8-bit, or as Quoted-Printable, which essentially is a conversion to 7-bit characters. How the message will be encoded depends on the mail client being used. If 8-bit is chosen, then all involved mail servers must also support this kind of transmission, which in turn is ensured by the SMTP standard 8BITMIME (don’t confuse with SMTPUTF8).

This ensures that mail servers can exchange data in 8-bit format. If only one of the mail servers in the transmission chain does not support 8BITMIME, the message is automatically converted to Quoted-Printable, i.e., to 7-bit format.

While this allows the message to be transmitted, it also renders the DKIM signature invalid – since it was created for the original message and is specifically designed to detect modifications.

According to my information, the operator of AboutMy.email intentionally disabled 8BITMIME on their system to identify exactly such kind of issues – my sincere thanks again for that; otherwise, I probably wouldn’t have noticed the root cause so quickly!

Workarounds

There are several workarounds for this issue. amavisd-new includes built-in conversion to Quoted-Printable, i.e. 7-bit, for outgoing messages before they are signed with DKIM. Rspamd, which I use nowadays, does not offer such feature.

However, starting with version 3.9, Postfix has the force_mime_input_conversion option, which was developed specifically for issues like this. It ensures that emails are always converted to Quoted-Printable format, i.e., 7-bit.

The setup depends on the specific configuration of the mail server. I use a combination of the excellent howtos by Christoph Haas and Thomas Leister, plus many additions of my own, which I plan to publish here on the blog at a later time.

It is important not to enable the Postfix option globally, as otherwise incoming emails can no longer be validated correctly because the DKIM signature would then become invalid in the other direction. Instead, only outgoing emails should be converted.

In my master.cf, I have set up a service for both SMTPS and Submission; here is a short excerpt:

smtps     inet  n       -       y       -       -       smtpd
 -o syslog_name=postfix/smtps
 -o smtpd_tls_wrappermode=yes
 -o smtpd_tls_security_level=encrypt
 -o smtpd_sasl_auth_enable=yes
 [..]

submission inet n       -       y       -       -       smtpd
 -o syslog_name=postfix/submission
 -o smtpd_tls_security_level=encrypt
 -o smtpd_sasl_auth_enable=yes
 [..]

To enable the Postfix option, a special cleanup service has to be be configured at the end of each block, which is done with the following line:

 -o cleanup_service_name=cleanup_out

The service itself is also configured in master.cf and adds only this one option to the existing cleanup service:

cleanup_out     unix    n       -       y       -       0       cleanup
 -o force_mime_input_conversion=yes

After a restart of Postfix, the DKIM validation now also works with remote servers that do not support 8BITMIME.

Further Thoughts

I have only been using the above configuration for a short time, so I cannot yet comment on potential issues or side effects. For decades, I had no problems with outgoing messages, as the number of 7-bit-only servers is rather small.

Furthermore, in my opinion, the bounce (550 DKIM Sender Invalid - envelope rejected) is due to very strict DMARC enforcement. SPF validated correctly, only DKIM did not – so the email would have passed the DMARC check and, at the very least, should not have been rejected.

Many thanks to the readers of Mailop for their fantastic support, and to Christian, who tested this implementation together with me