Return to sender – Florian’s beginner guide to improve e-mail deliverability (II)

How you can improve e-mail deliverability by working on your IP and domain reputation

If you run your own mail server, you’ll sooner or later likely run into e-mail deliverability problems due to what is called IP or domain reputation. Especially larger providers have various and sometimes not easy to understand filtering systems in place that might refuse messages from your machine.

While there is no magic wand to solve all deliverability problems, this article will give you an overview of some essentials that help messages from your mail server hopefully reach their destination.

Thanks a lot to Bastiaan for reviewing this blogpost and providing very valuable feedback to the list of services!

This article is split in two parts. The already published first part explains the general issue and gives some advice and best practice, while in this second part of this article I will introduce you to the concept of DNS-based blackhole list (DNSBL), feedback loops (FBLs), DNS-based whitelists (DNSWL), and share a set of antispam services to check your IP address against, to help remedy delivery issues.

So, how can you now improve your mail server’s IP address and your sending domains’ reputation? Each provider has their very own way of dealing with that, but there are some general concepts adapted by many. You should start from these, and then resolve issues you have with individual recipient domains later on - if the general housekeeping is not done, it’s unlikely other postmasters are willing to remedy a block of your server.

Unblocking outgoing SMTP traffic

Nowadays, several providers by default block outgoing traffic on TCP port 25, i.e. SMTP. They do this to avoid getting a bad IP reputation for their network block, be it due to unexperienced customers, people abusing the service, or hacked machines. Usually, for customers whose identiy is verified, and who have paid one or more invoices, they unblock outgoing SMTP traffic. In case you can’t reach any other e-mail server, check if such filtering is in place at your provider and reach out to their support to get assistance.

Negative listing (DNSBL)

Probably one of the most widely used concepts is called DNS-based blackhole list (DNSBL). It provides a mechanism for a mail server to determine the reputation of the sending server’s IP address. The majority of available DNSBLs supports only the querying of IPv4 addresses, but the amount of IPv6-aware lists is growing.

When your server connects to another machine, this machine queries your IP address via DNS, and as result receives some sort of classification, determining the reputation your IP address has on that respective list. Some servers immediately block incoming e-mail from certain DNSBLs, others use it just as one of many factors when determining a message’s spam level.

Additionally, there are more specific DNSBLs, which only list IP addresses classified as part of “dial-up” provider’s dynamic end-user address ranges - which is also why you should not run your mail server at home, unless you have a dedicated business line with a static IP address. The first step in evaluating your IP address reputation therefore is to check various DNSBLs if you are listed, and if so, ask for a delisting. The delisting process differs amongst the various DNSBLs: While some ask for an e-mail, explaining the reason, others require you to fill in a form, or setup an account on their website. There are also DNSBLs which only delist after payment - I personally didn’t pay for delisting yet.

The cumbersome process of querying dozens of list one by one can be shortened thanks to several website doing the query for you. Here are some references, in alphabetical order:

  • multirbl.valli.org (IPv4 and IPv6 queries) - lists a lot of available DNSBLs, including some much less relevant
  • MXToolBox (IPv4 and IPv6 queries) - provides a curated list of relevant DNSBLs
  • SpamHaus Project (IPv4 and IPv4 queries) - one important list, that however is already queried by both of the above

Be aware that delisting can take from a couple of minutes to days and even weeks, and that some DNSBLs don’t provide a delisting option at all. You might also need to provide additional proof e.g. that you are the new owner of an IP address, so it’s advisable to start that process as soon as you have your IP address. At this stage, if you followed the advice from the previous article and used a failover/floating IP address, you could still exchange the IP for a better one. Usually, I try to reserve a pool of IPs, then check them against DNSBLs, and only keep those that have the lowest number of listings.

Be advised that not all DNSBLs inform you when you get listed, or only inform your provider as owner of the netblock, who then needs to have a mechanism in place to forward the messages to you. Known to inform about listings seem to be SpamCops, SORBS, Spamhaus SBL and 0SPam.

To fill that gap, there are various external services or local scripts to run on your server that help you to monitor your reputation, so you can react in time. That, in combination with the pflogsumm tool mentioned in the first article, should do the job quite well.

Further IP reputation lists

Apart from DNSBLs that can be queried via DNS, there are several lists used, mostly in commercial filter systems, that you can’t query that way. They usually provide a web form to inquiry about the status of your IP, where you normally can also ask for delisting. Amongst these are, in alphabetical order:

Some DNSBL providers also offer to add your IP address to a “positive list” or “approved list”, which can further help to avoid false listings in the future. The requirements to be added can differ, and might require a certain amount of e-mail volume each day to be considered eligible.

Positive listing (DNSWL)

Apart from general-purpose DNSBLs, there are also “positive lists”, which actually increase a sender’s reputation. dnswl.org is one of these, and allows self-registration of both IPv4 and IPv6 addresses, as well as domain names. Every one in a while you should login and confirm your data is still accurate.

When registering for dnswl.org, and you have an IP out of a bigger provider’s netblock, double-check your listing is on “hi”, and if not, ask the dnswl.org admins for assistance.

I’d like to highlight that dnswl.org is volunteer-driven - thanks a lot for all your work, you do a great job!

Feedback loops (FBL)

Feedback loops don’t automatically influence your reputation, but they can help in identifiying issues early on. They provide a mechanism to report spam messages to the mail server’s postmaster. If you’re registered with a feedback loop, providers inform you about messages marked as spam by their users, so you can react. Some providers might raise your reputation if you’re also participating in a FBL, but I’m not sure about that - in any case, it’s a very helpful tool to identify problems from the beginning. That, in combination with DMARC reports, gives you quite some insight on what’s happening on your machine.

There are various reporting formats available, you often can configure the amount of messages you want to receive per day, but in any case I strongly recommend using a dedicated address like postmaster@ to receive these reports, so you can filter them into the proper subfolder straight away.

Some FBLs require the DKIM selection, or allow “*” for any selector. In case you update your selector, don’t forget to also update the FBL registrations. Some register the sending domain, others the sending IP address.

Not all e-mail providers offer FBLs, some only provide them when you hit a certain mail volume threshold, but below are some examples of FBLs you can subscribe to, in alphabetical order:

Provider specific: Deutsche Telekom, T-Online

Deutsche Telekom, respectively T-Online, by default blocks IP addresses that haven’t been used for sending e-mails to their servers for a certain amount of time. You can test if you are blocked by connecting to their mail server on port 25 - if the blocking is active, the connection will get immediately dropped with an 5xx error message, that lists a contact address to request unblocking from. To test, run the following command from your mail server:

telnet [-b floating IPv4] mx00.t-online.de 25

When I ran into the problem, they were quite fast in reacting and removed the blocking in about an hour. However, as per their use policy, they require the mail server’s main domain to have a proper imprint. In other words, if your mail server’s hostname is mail.mydomain.tld, you must place a proper imprint at mydomain.tld.

Provider specific: GMX, WEB.DE

For GMX and WEB.DE, I don’t have much information at hand. They seem to run internal blocklists, and provide contact forms to remove a blocked IP address from their system:

However, I don’t know if there’s a way to test if you’re listed (apart from sending an e-mail to an address hosted there), and I don’t know if they share a blocklist amongst both services or not.

Provider specific: Google

Google provides its own tool for postmasters to monitor the IP and domain reputation, called Postmaster Tools. After you’ve registered and verified the sending domain name, and depending on the amount of messages received from your server, you will get detailed statistics e.g. on spam detection or IP and domain reputation. From what I know, Google seems to use a combination of both IP as well as domain reputation for their spam filter, with measures ranging from marking e-mails as spam, throttling the delivery rate or blocking e-mails.

With their Postmaster Tools, you need to register every domain you want to send messages from, not only your mail server’s domain name. I recommend to use the DNS method for verification, as this is the one you likely have most control of. If you also run a website, keep in mind that domains verified for the Postmaster Tools are automatically verified also for the Search Console and vice versa.

If your domain or IP address got blocked by Google, there’s a Sender Contact Form to ask for delisting.

Provider specific: Microsoft (Office 365, Outlook.com, Hotmail.com, Live.com)

For Microsoft e-mail servers, you need to distinguish between two kinds of services: e-mail hosted at Outlook.com, Live.com and Hotmail.com on one hand, and e-mail hosted at Office 365 on the other hand. Each of them uses a different set of blocklists and filters, and being blocked in one system doesn’t necessarily mean being blocked in the other. Hetzner has summarized this in a very good article.

Outlook.com, Hotmail.com, Live.com

For e-mails to users at Hotmail.com, Outlook.com and Live.com, you should register your IP address at the Smart Network Data Services (SNDS). After registration, you’ll see if your IP address is blocked, and there might be more detailed statistics available after a certain threshold of messages has been reached. Checking the status of your IP address is advisable especially if you’ve just recently acquired it, as you might have inherited bad reputation from the previous owner. Currently, only IPv4 addresses are supported.

Should your IP address be flagged, you can ask for delisting via contact form, which is also available an an alternate link. This is a semi-automatic process. In the first step, the system will determine if the IP address can be delisted automatically (“mitigated”). Should your IP address not be eleigible for automatic mitigation, you can reply to the e-mail, provide further information and ideally some contact data, which will escalate the ticket to the support. This has a good chance for delisting then. Keep in mind that propagation of the change amongst the various systems can take a bit of time.

Microsoft also provides a Junk Mail Reporting Program (JMRP) to which you should subscribe. If your e-mail is marked as spam by the recipient, you will get a report.

There’s one thing to keep in mind: Should you want to remove your IP address from SNDS and JMRP, e.g. because you switch providers, you should always remove the entry at JMRP first - otherwise you might lose access to the IP and can’t delete it anymore from the system.

Office 365

Microsoft also provides hosted Exchange mailboxes as part of Office 365, and this uses a different filtering system. Unfortunately, there seems no way to query the reputation of your IP address, so you’ll only discover when an e-mail bounces back to you. The easiest way to test is to ask someone with an Office 365 account if you can send them a test message.

There seem to be at least three different ways an e-mail can get filtered. Some mentioned that their e-mails are accepted by the system, but end up in the Junk folder. In such case, it might help to flag the message as “not spam” from the recipient’s account, but I have no experience with this.

Messages might also directly get blocked at delivery. In this situation, there are at least two different kind of blocklists. Sometimes, in the rejection message, you get referred to the Office 365 Anti-Spam IP Delist Portal at https://sender.office.com. Fill in your data and ask for a delisting. Based on my experience, this happens quite fast, but keep in mind that propagation of the change amongst the various systems can take a bit of time.

In other cases, the rejection message mentions a delist@ address. If you receive such a bounce, send an e-mail to the address mentioned, provide details and contact data, and you will receive a so called SRX ticket. This seems to get manually processed and might take a bit longer than the automatic process.

Ask for an exemption at the recipient’s mail server

There are also situations where you end up in a chicken and egg situation, especially when the recipient’s mail server has a wrong configuration. Some time ago, I experienced this problem myself when sending e-mail to a larger German city. They used a commercial blocklist that builds up IP reputation based on the messages received. If there’s not enough e-mails yet from your IP address, the reputation is unknown.

Unfortunately, that specific city seems to have misconfigured their antispam filter. I didn’t send them mails before, so my reputation was unknown. This should have resulted in a temporary delay of the e-mail, also known as greylisting. However, they directly rejected the message. I therefore was not able to build up any reputation - my mails were rejected straight away, but in order to get a good reputation, some mails need to be processed.

Convincing them that their mail setup was wrong probably would have taken too long, so I directly e-mailed the postmaster and asked for an exception of my mail server’s IP address, which they’ve granted. While this only cures the symptoms, not the underlying problem, I can now send messages to their mail server without issues.

Further bits and pieces

Not directly related to antispam filtering, but probably still helpful is to check the geolocation of your IP address, especially if you host e-mails on a business dialup with a static IP address. Well known databases are DB IP and MaxMind GeoIP.

To test if your setup works correctly, sites like mail-tester and learndmarc come in handy.

And of course, there’s likely more blocklists and providers that are worth mentioning here, so I’m happy for your feedback to update this article!