Reissuing a Web Certificate Broke SMTPS 465 Hostname Matching

I had just created an account for a customer on a shared host running DirectAdmin. The primary domain had another name configured as an alias (domain pointer) pointing to the same web space. I wanted the alias to work over https too, so I requested a new certificate containing all 4 web names. Every web check passed, and I thought the job was done. It was only when I worked through my routine mail port checks that I noticed mail.example.com:465 was returning the server's certificate instead of the domain's. If the customer had already configured a mail client to use SMTPS, they would have seen a certificate warning immediately.

All domain names, IP addresses, and usernames in this article are examples (example.com is the primary domain, example.net is the alias, and 203.0.113.10 is the server's public IP). The sequence of events, diagnostic commands, and results are real.

Symptoms: all 4 web names passed, but the mail port served the server's certificate

The server runs Debian with DirectAdmin, nginx and php-fpm for the web, and exim and dovecot for mail. When I created the account, DA automatically issued a wildcard certificate for *.example.com together with the apex. Everything worked from day one, including mail.

The problem started with a seemingly harmless requirement. The alias website uses the same certificate as the primary domain. For https://example.net to open without a warning, example.net must also appear in the certificate for example.com. So I requested a new one through CMD_API_SSL with request=letsencrypt, specifying 4 names: example.com, www.example.com, example.net, and www.example.net.

I checked the web names one by one, and they all passed.

IP=203.0.113.10
for h in example.com www.example.com example.net www.example.net; do
  printf '%-18s ' "$h"
  openssl s_client -connect $IP:443 -servername $h -verify_hostname $h </dev/null 2>/dev/null \
    | grep -E 'Verification'
done
# example.com        Verification: OK
# www.example.com    Verification: OK
# example.net        Verification: OK
# www.example.net    Verification: OK

If I had stopped there, I would have handed the job over. But when I ran the same command against the mail port, the result was a different story.

openssl s_client -connect 203.0.113.10:465 -servername mail.example.com \
  -verify_hostname mail.example.com </dev/null 2>/dev/null | grep -E 'subject=|Verification'
# subject=CN = host-a.example.org
# Verification error: hostname mismatch

The returned subject was the server's own hostname. This meant exim did not recognize mail.example.com, the name the client sent in SNI, so it served the server's default certificate. Encryption still worked, but the name did not match, which for most mail clients means a warning dialog or an outright refusal to send.

The file in the middle: /etc/virtual/snidomains

DirectAdmin's mail setup has a file called /etc/virtual/snidomains. It is a lookup table that maps the hostname sent by a client in SNI to the user and domain whose certificate should be used. As soon as I opened it, I saw the cause.

grep example /etc/virtual/snidomains
# หลังขอ cert 4 ชื่อ (ผมย่อแต่ละแถวให้เหลือ ชื่อใน SNI : user : โดเมนเจ้าของ cert
# หน้าตาจริงของแถวอาจต่างไปตามรุ่นของ DA สิ่งที่ต้องดูคือชื่อทางซ้ายสุด)
# example.com:webuser:example.com
# www.example.com:webuser:example.com
# example.net:webuser:example.com
# www.example.net:webuser:example.com

There were exactly 4 rows matching the 4 names I had supplied. No rows for mail. pop. smtp. ftp. remained. Every time a certificate is saved through this API, DA rebuilds that domain's entries in the file from the SAN names explicitly listed in the certificate. The original certificate that DA issued automatically covered the mail names. When I replaced it with one containing only web names, the entire set of mail name rows disappeared without any warning.

What makes this easy to miss is that the certificate request did not fail. Let's Encrypt issued what I requested, DA installed what it received, and the websites worked as intended. Every step reported success. What broke was something I had not requested but had previously had.

My first wrong guess: manually request a wildcard again

My first thought was that the old certificate was a wildcard and worked, so requesting a wildcard again should settle it. I ran DA's script manually.

cd /usr/local/directadmin/scripts
./letsencrypt.sh request '*.example.com,example.com'

The certificate was issued, and I checked the mail ports one by one.

IP=203.0.113.10; H=mail.example.com
for p in 465 993 995; do
  printf '%-4s ' "$p"
  openssl s_client -connect $IP:$p -servername $H -verify_hostname $H </dev/null 2>/dev/null \
    | grep -E 'Verification'
done
# 465  Verification error: hostname mismatch
# 993  Verification: OK
# 995  Verification: OK

dovecot on both 993 and 995 returned the correct certificate again, but exim on 465 was still wrong. My assumption that restoring the wildcard would restore everything was only half right. The reason is that DA's snidomains generator maps only explicitly listed names. It does not expand a wildcard into mail. and smtp. entries. The certificate does cover mail.example.com, but there is no row for that name in the table, so exim never reaches that certificate.

There is another detail to know before trying to combine the two approaches. If a wildcard is included together with a name it already covers, such as *.example.com alongside mail.example.com, Let's Encrypt rejects the request because the names are redundant. I had to choose one approach, and based on the results above, explicit names were the option that worked with exim.

The fix that worked: request through the API with the same names as auto-SSL

What I should have done from the start was check which names DA used when it issued the certificate automatically and copy that set: the domain, www., mail., pop., smtp., and ftp.. Then I could add the extra names I needed, which in this case were the alias apex and www., for a total of 8 names.

# ขอในฐานะเจ้าของโดเมน ผ่าน login key ของ admin (ค่า <LOGIN_KEY> อ่านจากไฟล์ที่ root เท่านั้นอ่านได้)
curl -s -u "admin|webuser:<LOGIN_KEY>" "https://host-a.example.org:2222/CMD_API_SSL" \
  --data-urlencode "domain=example.com" \
  --data-urlencode "action=save" \
  --data-urlencode "type=create" \
  --data-urlencode "request=letsencrypt" \
  --data-urlencode "name=example.com" \
  --data-urlencode "keysize=secp384r1" \
  --data-urlencode "encryption=sha256" \
  --data-urlencode "le_select0=example.com" \
  --data-urlencode "le_select1=www.example.com" \
  --data-urlencode "le_select2=mail.example.com" \
  --data-urlencode "le_select3=pop.example.com" \
  --data-urlencode "le_select4=smtp.example.com" \
  --data-urlencode "le_select5=ftp.example.com" \
  --data-urlencode "le_select6=example.net" \
  --data-urlencode "le_select7=www.example.net"

The parameter names for this API can vary by DA version. Before using this, check the documentation for the version you run, or open the SSL page in the panel and inspect what the form submits. I chose to request through the API instead of running the script directly because the API route lets DA write the entire set of snidomains entries itself, without any manual file edits.

Every name in the list must have DNS pointing to the actual server. Otherwise, validation for that name will fail and the entire certificate request will fail. If the domain is also behind the DNS provider's proxy, watch out for mail records, which must not be proxied. I covered that in Mail stopped working after moving a domain to Cloudflare. It is a different cause from this one, but the end result looks similar: mail does not work even though the websites are fine.

Once the request completed, I checked the same file again.

grep example /etc/virtual/snidomains | cut -d: -f1
# example.com
# www.example.com
# mail.example.com
# pop.example.com
# smtp.example.com
# ftp.example.com
# example.net
# www.example.net

Then I tested every port again, with the results shown in this table.

I noticed one other thing during the checks. webmail.example.com also returned a hostname mismatch, but when I checked, this name had no DNS record on this server, and the same was already true for every domain on the machine. It was not something this work had broken. I simply noted it so I would not chase the wrong problem.

A testing trap: 127.0.0.1 returns the default certificate every time

During the checks, I nearly reached another wrong conclusion. At first, I tested from the server itself against 127.0.0.1:443 and received the server's default certificate for every name. It looked as though the newly requested certificate had not been installed, but that was not true. nginx on this server binds each domain's vhost to the public IP, so requests arriving through loopback match no vhost and always fall through to the default.

# ผิด: ได้ cert default เสมอ ไม่ว่าจะติดตั้งถูกหรือไม่
openssl s_client -connect 127.0.0.1:443 -servername example.com </dev/null 2>/dev/null \
  | openssl x509 -noout -subject

# ถูก: ชี้ไปที่ IP สาธารณะ หรือบังคับ resolve ให้ curl
openssl s_client -connect 203.0.113.10:443 -servername example.com </dev/null 2>/dev/null \
  | openssl x509 -noout -subject -ext subjectAltName
curl -s --resolve example.com:443:203.0.113.10 -o /dev/null \
  -w '%{http_code} ssl_verify=%{ssl_verify_result}\n' https://example.com/

I had run into this same trap when moving an entire domain, which I described in A .com domain was seized: moving the entire site to .net on DirectAdmin. That time involved only the web side. This time there was another layer: the mail ports, where exactly the same principle applies. I need to set -servername to the name the customer will actually use and connect through the IP the customer will actually reach.

Back up before making changes, and check afterward

Before changing a certificate, I always back up the domain's four files along with snidomains. If the new request fails halfway through or produces a certificate missing some names, restoring the original files is much faster than requesting another certificate, and it does not use up the Let's Encrypt issuance quota.

D=/usr/local/directadmin/data/users/webuser/domains
B=/root/cert-backup-$(date +%F)
mkdir -p "$B" && chmod 700 "$B"
cp -a "$D"/example.com.cert "$D"/example.com.key \
      "$D"/example.com.cacert "$D"/example.com.cert.combined "$B"/
cp -a /etc/virtual/snidomains "$B"/
ls -l "$B"

This is the checklist I now use whenever I work on a domain's certificate on a DirectAdmin server.

1. Back up the domain's .cert .key .cacert .cert.combined and /etc/virtual/snidomains to a directory under /root with permissions set to 700.
2. Prepare the names: the domain, www., mail., pop., smtp., ftp., then add the alias apex and www. if there is an alias. Do not combine a wildcard with names it covers.
3. Request through CMD_API_SSL instead of running the script directly, so DA writes snidomains itself.
4. grep for the domain name in snidomains. All mail name rows must be present.
5. Run openssl s_client through the public IP on port 443 for every web name, and on 465, 993, 995 using the mail. name. Every result must show Verification: OK.
6. Record pre-existing mismatches separately, such as webmail. names with no DNS. Do not count them as breakage caused by this work.

The numbers from this job

The SAN list on a DirectAdmin server is more than a list of names covered by the certificate. It also supplies the data for the table exim uses to select a certificate. Requesting a new certificate with fewer names therefore amounts to unknowingly deleting rows from that table. It took me three certificate requests to get the right set, even though checking the names in the original certificate first would have let me finish in one attempt. If you manage a similar server, feel free to add the port-checking loop above to your own procedure. If I have left anything out or made a typo, I will come back and correct it later.