Stolen Mailbox Password Sent 1,786 Spam Emails Unseen by CSF/LFD

I recently had several customers on the same server report at once that they could not send email to recipients in the US, even though nothing had changed. I worked backward from the mail queue and eventually traced it to a single mailbox belonging to another customer, whose stolen password had been used to quietly send spam for four full weeks without a single response from CSF/LFD or DirectAdmin's Brute Force Monitor.

I think this case has more to teach than a routine brute-force attack, so I documented the mechanisms that kept it invisible, the commands I actually used to investigate, the command pitfalls that nearly led me to the wrong conclusion, and the blocklist removal sequence that must be followed in order.

All domains, mailbox names, and IP addresses in this article are fictional (using the standard TEST-NET ranges), but the volumes, behavior, and log line formats come from the real server.

The first visible symptoms were far downstream

The starting point was not a security alert, but a DirectAdmin warning about a backed-up mail queue and complaints from other customers who had done nothing wrong.

exim -bpc            # จำนวนเมลค้างคิว
exim -bp | head -40  # ค้างจากใคร ไปไหน

A look at the queue made it clear that the backlog came from a single sender, with all messages addressed to free email providers in the US. I then checked what those destinations were "saying" to us.

grep -ai 'found on one or more DNSBL' /var/log/exim/mainlog | tail -5
grep -aci 'found on one or more DNSBL' /var/log/exim/mainlog

554 ... <your-server-ip> found on one or more DNSBLs
    https://postmaster.<ผู้ให้บริการปลายทาง>/smtp-error-codes.php#BL000110

I counted 169 occurrences in a single day, and confirmed that the server's public IP (shared by the entire server) was already listed on both Spamhaus ZEN and cbl.abuseat.org. By the time I found out, the damage was already done.

One decisive question: did any foreign IP log in successfully?

Before going any further, I want to establish this question, because it separates "actually compromised" from "under attack but still intact," and jumping to conclusions here means wrongly blaming a customer.

The key is never to judge by the number of failed authentication attempts. Look only for successful authentication from networks that should not be there.

[email protected]
LOGS="/var/log/exim/mainlog.4 /var/log/exim/mainlog.3 /var/log/exim/mainlog.2 \
      /var/log/exim/mainlog.1 /var/log/exim/mainlog"

for L in $LOGS; do grep -aE " <= .*A=[a-z_]+:$ACCT" "$L"; done \
 | while IFS= read -r l; do
     ip=$(echo "$l" | grep -oE '\[[0-9.]+\]' | tail -1 | tr -d '[]')
     cc=$(geoiplookup "$ip" 2>/dev/null | grep -oE '[A-Z]{2},' | head -1 | tr -d ',')
     echo "$cc $ip"
   done | sort | uniq -c | sort -rn

Zero authentication failures from the very first attempt means the attacker was not guessing the password, but had the correct one from the start — the source was not the server, but the customer's endpoint or a password reused from another breached service.

Why the fully installed stack still could not see it

This is the core of the article. It was not a configuration mistake or something I forgot to enable, but a limitation of the tools themselves.

The entire set of CSF/LFD mail triggers — LF_SMTPAUTH, LF_POP3D, LF_IMAPD, LF_DISTATTACK, LF_DISTSMTP — and DirectAdmin's BFM all work the same way: count failed authentication attempts and compare them against a threshold. When the attacker logs in successfully every time, the counters never move, and not a single rule triggers.

The next layer was DirectAdmin's per-mailbox daily limit, which did exist and was working correctly, but the default of 200 messages/day in /etc/virtual/user_limit was too high to catch low-and-slow behavior. The attacker sent 50-75 messages a day, just 25-37% of the limit, and stayed quiet for a month.

Breaking the totals down by log period makes the slow burn clear: 327 / 324 / 522 / 397 / 216 messages. The last batch included the day the attacker increased the pace enough to hit the 200 limit for the first time, and only then did DirectAdmin display a warning.

grep -a 'emails have just been sent by' /var/log/directadmin/system.log | tail -20

2026:07:15-20:28:34: Warning: 200 emails have just been sent by
[email protected] (user=dauser).  Limit: 200

After the limit was reached, Exim issued another 16 temporary rejections, leaving 18 messages in the queue, all addressed to the same free email provider. That was the queue I found at the start of this article.

The other mailbox, abused from a cloud instance, let only about 205 messages through rather than over a thousand, because LFD triggered AUTHRELAY in time — the only component in the existing stack that actually detected a relay/outbound signal. If you have disabled it, I recommend turning it back on.

One more thing: the auto-whitelist system I previously wrote about for unblocking customer offices uses "successful login" as its deciding condition, which works safely for brute-force attacks but does nothing for accounts with stolen passwords, because the attacker also logs in successfully. I need to know the limits of my own tools, too.

Learn to read an exim log line first

All the evidence is in the <= lines of mainlog. Here is a real example (with values replaced).

grep -a "A=login:$ACCT" /var/log/exim/mainlog | head -5

2026-07-15 16:39:43 1xxxxx-000000xxxxx-xxxx <= [email protected]
  H=(mailhost) [203.0.113.10] P=esmtpsa X=TLS1.3:TLS_AES_256_GCM_SHA384:256
  A=login:[email protected] S=3549
  T="Immediate Opening - Part-Time & Flexible" for [email protected]

The parts to understand: A=login: means successful authentication, not an "attempt" (failures appear as authenticator failed ... ([email protected]), a completely different format) · P=esmtpsa means submission using both authentication and TLS · H=(...) is the name declared by the client and can be forged freely, while the last set of square brackets [...] contains the actual IP · T="..." is the subject, which can be used to group campaigns.

The commands I actually used to investigate

Everything was run as root directly on the mail server. The available history consisted only of mainlog and mainlog.1 through .4, roughly 4 weeks, so every figure in this article is limited by that retention period.

Find today's highest-volume authenticated senders — this should have been a cron job from the start.

grep -a "^$(date +%F)" /var/log/exim/mainlog \
  | grep -oE 'A=[a-z_]+:[^ ]+' | sed 's/A=[a-z_]*://' | sort | uniq -c | sort -rn | head

Compare each mailbox's sending volume against its own limit. DirectAdmin's usage file stores 1 character per message, so I count it with wc -c after removing newlines.

for u in /etc/virtual/*/usage/*; do
  n=$(tr -d '\n' < "$u" 2>/dev/null | wc -c); [ "$n" -gt 50 ] || continue
  box=$(echo "$u" | sed -E 's#/etc/virtual/([^/]+)/usage/(.*)#\2@\1#')
  lim=$(cat "$(dirname "$(dirname "$u")")/limit/$(basename "$u")" 2>/dev/null)
  printf '%-45s sent=%-5s limit=%s\n' "$box" "$n" "${lim:-default}"
done | sort -k2 -rn | head

Once I have a suspect, I check which campaigns it is sending, where they are going, and the volume in each log period.

# แคมเปญอะไรบ้าง
for L in $LOGS; do grep -aE " <= .*A=[a-z_]+:$ACCT" "$L"; done \
  | grep -oE 'T="[^"]+"' | sort | uniq -c | sort -rn | head

# ยิงไปโดเมนปลายทางไหนบ้าง
for L in $LOGS; do grep -aE " <= .*A=[a-z_]+:$ACCT" "$L"; done \
  | grep -oE 'for [^ ]+@[^ ]+' | sed 's/.*@//' | sort | uniq -c | sort -rn | head

# ปริมาณต่อไฟล์ log (เห็นการเผาช้า ๆ ข้ามสัปดาห์)
for L in $LOGS; do printf '%-32s %s\n' "$L" "$(grep -acE " <= .*A=[a-z_]+:$ACCT" "$L")"; done

For a per-account overview in a single OK/FAIL/country/rDNS table, use this. This is what prevents me from wrongly blaming a customer: a high OK count from an unfamiliar IP = compromised, while a high FAIL count with no OK at all = merely under attack.

[email protected]
LOG=/var/log/exim/mainlog

echo "OK=$(grep -c "A=[a-z_]*:$ACCT" $LOG)  FAIL=$(grep "set_id=$ACCT" $LOG | grep -c failed)"

( grep "A=[a-z_]*:$ACCT" $LOG | sed -nE 's/.*\[([0-9.]+)\].*/\1/p'
  grep "set_id=$ACCT" $LOG | grep failed | sed -nE 's/.*\[([0-9.]+)\].*/\1/p'
) | sort -u | while read ip; do
    ok=$(grep -F "$ip" $LOG | grep -c "A=[a-z_]*:$ACCT")
    fail=$(grep -F "$ip" $LOG | grep "set_id=$ACCT" | grep -c failed)
    cc=$(geoiplookup $ip | grep -oE '[A-Z]{2},' | head -1 | tr -d ',')
    rdns=$(getent hosts $ip | awk '{print $2}' | head -1)
    printf '%-16s OK=%-3s FAIL=%-4s %s %s\n' "$ip" "$ok" "$fail" "$cc" "$rdns"
done

And do not guess the domain owner's DirectAdmin username from the domain name. I have made that mistake before; it is better to ask the file directly.

OWNER=$(grep -rl '^example.com$' /usr/local/directadmin/data/users/*/domains.list \
        | sed -E 's#.*/users/([^/]+)/.*#\1#')
echo "$OWNER"

Command pitfalls that silently break the system

I think this is the most valuable part of the article, because every item here produces "no error" but returns the wrong answer, which I then trust completely.

The other four pitfalls are: mainlog uses ISO dates (2026-07-15), while /var/log/lfd.log uses syslog dates (Jul 15), so filtering with the wrong format returns nothing without an error · Always pass -a when using grep on exim logs, or an unusual byte can make it return Binary file matches without printing anything else · Do not match IPs by substring, because 74.125 also matches 174.125.x.x; escape the dots or anchor the match with square brackets · And when extracting set_id=, use [^ )]+ because it is inside parentheses.

For the DNSBL check I use regularly, reverse the octets of the server IP first (if the IP is 1.2.3.4, enter 4.3.2.1).

for bl in zen.spamhaus.org bl.spamcop.net b.barracudacentral.org \
          dnsbl.sorbs.net psbl.surriel.com cbl.abuseat.org; do
  printf '%-28s %s\n' "$bl" "$(dig +short 4.3.2.1.$bl A | tr '\n' ' ')"
done

Containment playbook — do not change this order

1. Spend 10 minutes confirming before acting: inspect the queue -> find the top senders -> ask the decisive question. If there are no successful logins from unfamiliar IPs, stop. Do not tell the customer they have been hacked yet.
2. Collect evidence before touching anything: raw mainlog lines + subjects + destinations + counts per log period. These will still exist after a password reset, but the queue and state will change.
3. Block the attacker's IP, then immediately reset the mailbox password. Always do both together; with a block alone, the attacker can switch IPs within a minute.
4. Remove queued spam, but inspect the list before deleting, because legitimate mail from other customers may be in the same queue.
5. Lower that mailbox's sending limit, and review the server-wide default as well.
6. Check the entire server for other mailboxes compromised in the same way, not just the domain that reported the problem.

# 1. บล็อก IP คนร้าย
csf -d 203.0.113.10 "spam via compromised [email protected] 2026-07-15"

# 2. เปลี่ยนรหัส mailbox ผ่าน DA API
OWNER=$(grep -rl '^example.com$' /usr/local/directadmin/data/users/*/domains.list \
        | sed -E 's#.*/users/([^/]+)/.*#\1#')
URL=$(/usr/local/directadmin/directadmin api-url --user="$OWNER")
curl -sk "${URL}/CMD_API_POP" \
  --data-urlencode action=modify \
  --data-urlencode domain=example.com \
  --data-urlencode user=user \
  --data-urlencode passwd="<new>" --data-urlencode passwd2="<new>"
# คาดหวัง: error=0&text=Account%20password%20changed

# 3. ยืนยันว่ารหัสเปลี่ยนจริง
ls -l /etc/virtual/example.com/last_password_change/user

# 4. ล้างสแปมในคิว — ดูก่อน ลบทีหลัง
exiqgrep -i -f 'example\.com'
exiqgrep -i -f 'example\.com' | xargs -r exim -Mrm

# 5. ลดเพดานของกล่องที่โดนลง (เคสนี้จาก 200 -> 50/วัน)
echo 50 > /etc/virtual/example.com/limit/user

The actual result was a queue reduced from 18 to 0, with not a single legitimate customer message deleted.

Get the delisting request accepted the first time

The most important rule is never request removal before cutting off the source. Submitting while spam is still flowing = immediate relisting, and some operators will make the next attempt harder. I therefore use a gate: no submission until all five conditions are met.

Once all five are met, submit to each operator in turn: Spamhaus (ZEN/CSS/PBL): go to check.spamhaus.org, enter the server IP, and follow the removal form · cbl.abuseat.org is now part of Spamhaus, so check and request removal through the same page; listings normally clear automatically once spam traffic stops · Major destination email providers each have their own form; providers returning 554 ... found on DNSBLs usually include an error-code page link in the rejection line, so follow that link directly · Other DNSBLs, such as spamcop / barracuda / sorbs / psbl: check first, and submit only if listed; many listings expire automatically within 24-48 hours if there is no new spam.

The forms all ask roughly the same things: state that a single mailbox was compromised, rather than this being the host's policy · Specify when the source was cut off and what actions have already been taken · Describe the measures to prevent recurrence · And do not include more customer information than necessary.

Do not stop after submitting. Repeat the DNSBL checks once a day until all listings clear, and watch for found on one or more DNSBL in mainlog alongside them. The latter is more representative of reality, because it comes from the actual destination, not a lookup.

Preventing recurrence: stop watching only failed logins

My conclusion from this case is that the change needed is not "stricter brute-force rules," because those rules were already working correctly. They were simply looking in a different direction from the problem. What needs to be added is outbound signals.

I added cron jobs to run the daily authenticated-sender counts and compare usage against limits, with alerts when a mailbox uses more than about 50% of its own limit, or when a mailbox that normally sends almost nothing suddenly appears among the top senders. The alert that would actually have caught this case on day one is a successful login from a country or ASN never before seen for that mailbox, because the attacker used a single IP in a single country for all four weeks.

The limits need reviewing too. A server-wide default of 200/day does not suit every mailbox. Receive-only mailboxes and system mailboxes should have very low limits, while mailboxes that actually send should have limits based on their real usage. Set whatever numbers suit your needs, but do not apply one value across the server and assume that gives you control.

I will close with the most painful part of this case — a shared IP means shared risk. One mailbox belonging to one customer left other tenants across the entire server unable to send email, even though they had done nothing wrong. The next steps should be to require app passwords or 2FA for submission, restrict submission countries for mailboxes used only domestically, and route outbound mail through a smarthost with its own rate limits. The actual source of the password leak, however, remains at the customer's endpoint; the server can only limit the damage. If I have missed anything in the explanation or made any typos, I will come back and correct them later.