DA Migration Finale — Security Fixes and Hardening After Stabilizing

Parts 1 through 4 of this series covered planning the move from CentOS 7 to Debian 12, restoring DirectAdmin across distros, fixing changes in the stack (pure-ftpd to proftpd, spamassassin to rspamd), and getting the server stable. If you have not read them yet, you can find them in the series index.

I deliberately saved this part for the end because it is not migration work as such, but security fixes and hardening I did after the server had fully stabilized. Some of these are real vulnerabilities caused by configs that many people have been writing incorrectly without realizing it. I think sharing these may be the most useful part of the series.

sudoers wildcard path traversal — the issue I most want to flag

If you still have sudoers entries containing the wildcard *, I recommend checking them urgently, because I initially had several lines written this way myself.

Look at this example entry — at first glance, it looks very safe because it restricts cat to files in a single directory.

user ALL=(root) NOPASSWD: /usr/bin/cat /var/spool/cron/crontabs/*

I had always understood * here to mean "any file in this folder." But when I tested it with the bot user granted this permission, the result gave me a chill.

sudo -u bot_user sudo -n cat /var/spool/cron/crontabs/../../../../etc/passwd
# RESULT: เนื้อหา /etc/passwd ออกมาเฉยเลย

It could read /etc/passwd, even though the entry clearly said it could only read within crontabs/.

The reason is that sudoers globbing differs from shell globbing. In a shell, an asterisk * does not match /, but in sudoers, an asterisk also matches /. So crontabs/* in sudoers also covers crontabs/../../../../etc/passwd, creating a path traversal that follows ../ out to read any file on the server as root.

The fix is to stop using a bare asterisk and replace it with a character class that excludes / and . entirely.

- user ALL=(root) NOPASSWD: /usr/bin/cat /var/spool/cron/crontabs/*
+ user ALL=(root) NOPASSWD: /usr/bin/cat /var/spool/cron/crontabs/[a-z][a-z0-9_-]*

[a-z][a-z0-9_-]* means the first character must be a lowercase a-z, followed only by alphanumeric characters, underscores, or dashes. Neither / nor . is in the allowed character set. With no / in the pattern, ../ traversal is impossible, closing this hole completely.

Once I found this, I applied the same pattern to every entry containing a wildcard on the system. Common ones included reading user usage, checking home directory sizes, and checking cert expiry dates.

/usr/bin/cat /usr/local/directadmin/data/users/[a-z][a-z0-9_-]*/user.usage
/usr/bin/du -sh /home/[a-z][a-z0-9_-]*/
/usr/bin/openssl x509 -enddate -noout -in /etc/letsencrypt/live/[a-z0-9][a-z0-9._-]*/cert.pem

For the cert line, I allow . and _ in the class ([a-z0-9._-]) because Let's Encrypt folder names are domains, which already contain dots. But I require the first character to be [a-z0-9] so it cannot start with a dot — blocking a leading . that could become ../. I initially had several bare * wildcards and only refactored them all out after learning about this.

OpenSSL SECLEVEL — a long-term tightening plan

In part 4, I initially enabled SECLEVEL=0 to get the web stack running because Debian 12 sets a much stricter default crypto policy than CentOS 7. Old certs that still had SHA1 in their certificate chains were rejected, leaving several users' websites unable to start. I temporarily lowered SECLEVEL to 0 to get everything working again first.

But SECLEVEL=0 is insecure mode: it accepts all weak crypto. It cannot stay this way long term; there needs to be a plan to raise it again. I laid out the following stages.

These are the steps I planned:

• Use a Let's Encrypt renewal hook to reissue every user's cert every 60 to 90 days. Newly issued certs will use SHA256 throughout the chain, with no SHA1 remaining.
• Track which certs still have SHA1 in their chains, using openssl x509 to verify each one so I know how many legacy certs remain.
• Once the SHA1 certs are gone, move to SECLEVEL=1, which still allows SHA1 in end-entity certs but does not allow SHA1 in CA certs.
• The long-term target is SECLEVEL=2, the Debian 12 default, which rejects SHA1 throughout the chain.

Mine has now been at SECLEVEL=0 for three months. Just one legacy cert has yet to be reissued, so I am nearly done. Once that cert expires, I will move to SECLEVEL=1 immediately, then work up to 2. The key is not to forget that insecure mode is enabled. Keep a checklist or reminder, or you could leave it on for a year without realizing it.

ModSecurity exclusions — load them before CRS to take effect in time

This trap had me puzzled for a while. DirectAdmin stores admin custom ModSec rules at /usr/local/directadmin/data/admin/modsecurity_rules, which looks like the right place to put exclusions. The problem is that this file loads after OWASP CRS.

With CRS loaded first, its rules have already fired, blocking the request before our exclusions have a chance to run. No matter how I added them, they had no effect because they were too late.

The fix is to put exclusions directly in a file that loads before CRS, named REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf following CRS's own convention. The number 900 puts it before the main rule set, which uses higher numbers.

cat > /etc/modsecurity.d/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf <<'EOF'
SecRule REMOTE_ADDR "@ipMatch 203.0.113.99" \
    "id:99001,phase:1,pass,nolog,ctl:ruleRemoveById=920350"

SecAction "id:99002,phase:1,pass,nolog,ctl:ruleRemoveById=920450"

SecRule REQUEST_URI "@beginsWith /phpMyAdmin/" \
    "id:99003,phase:1,pass,nolog,ctl:ruleRemoveById=932380"
EOF

nginx -t && systemctl reload nginx

Each line is a different exclusion — the first exempts a trusted IP from CRS rule 920350 (the placeholder 203.0.113.99 stands in for the real IP), the second disables rule 920450 unconditionally, and the third disables rule 932380 only for requests to phpMyAdmin. Remember that our rule id must not collide with CRS; do not use numbers in its reserved range. I chose the 99xxx range, which is definitely available. Always finish with nginx -t before reloading to avoid a syntax error taking down every website on the server.

Self-heal — make it survive DA updates on its own

The next problem is that a DA update or CustomBuild rebuild may wipe the entire /etc/modsecurity.d/ folder. The exclusions I carefully put there disappear, and I do not notice until legitimate requests start getting blocked. So I set up a daily cron job to check and restore them automatically.

cat > /usr/local/directadmin/scripts/custom/check_modsec_exclusions.sh <<'EOF'
#!/bin/bash
set -e
TARGET=/etc/modsecurity.d/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
if [ -f "$TARGET" ] && grep -q 'id:99001' "$TARGET" && \
   grep -q 'id:99002' "$TARGET" && grep -q 'id:99003' "$TARGET"; then
    exit 0
fi
# ถ้าไฟล์หายหรือ rule ไม่ครบ → เขียน exclusion กลับลงไปใหม่ตรงนี้
EOF
chmod 755 /usr/local/directadmin/scripts/custom/check_modsec_exclusions.sh

echo '20 4 * * * root /usr/local/directadmin/scripts/custom/check_modsec_exclusions.sh' \
    > /etc/cron.d/modsec-exclusions-check

The logic is simple: if the file exists and contains all three ids, exit 0 and do nothing. If the file is missing or the rules are incomplete, write it back. Schedule cron to run every day at 4:20 a.m. Whenever DA wipes it, the file will return within a day.

csfpost.sh drop-in — avoiding the silent path trap

In part 4, I encountered issue 16 of the series, involving CSF and csfpost.sh. CSF checks two paths, /usr/local/csf/bin/csfpost.sh and /etc/csf/csfpost.sh, but runs only one of them. If you happen to put rules in both places, the one at /etc/csf/csfpost.sh silently stops working, with no warning or error at all. The rules in it never run, and it took me a long time to work out why the firewall rules were not taking effect.

I fixed this permanently with a drop-in pattern. Instead of cramming rules into one long file, I made the main file do nothing but loop over and call individual files in a drop-in folder.

cat > /usr/local/csf/bin/csfpost.sh <<'EOF'
#!/bin/bash
set -u
DROPIN=/etc/csf/csfpost.d
[ -d "$DROPIN" ] || exit 0
for f in "$DROPIN"/*.sh; do
    [ -r "$f" ] || continue
    . "$f" || logger -t csfpost "FAIL: $f exited $?"
done
EOF
chmod 700 /usr/local/csf/bin/csfpost.sh

mkdir -p /etc/csf/csfpost.d
cat > /etc/csf/csfpost.d/10-allowout-da-users.sh <<'EOF'
for uid in 65534 1002 999; do
    iptables -C ALLOWOUT -p tcp -m owner --uid-owner $uid \
        -m conntrack --ctstate NEW -j ACCEPT 2>/dev/null \
        || iptables -I ALLOWOUT 1 -p tcp -m owner --uid-owner $uid \
           -m conntrack --ctstate NEW -j ACCEPT
done
EOF
chmod 755 /etc/csf/csfpost.d/*.sh

[ -f /etc/csf/csfpost.sh ] && mv /etc/csf/csfpost.sh /root/csfpost.sh.dead-bak

The main csfpost.sh loops over every .sh file in /etc/csf/csfpost.d/ and sources them one at a time. If one fails, it logs through logger without bringing down the whole set. The example drop-in allows outbound traffic for certain uids. Notice that I use iptables -C to check whether the rule already exists and insert it only if it does not. This is idempotent: running it repeatedly never adds duplicates. Finally, move the old /etc/csf/csfpost.sh file aside as a backup to prevent confusing shadowing.

The benefit of this pattern is that adding a new rule only requires dropping a new file into csfpost.d/, without editing existing files. There is no longer a risk of shadowing between the two paths, and rules can also be organized into files by purpose.

5 layers that survive DA updates — separate configs by what DA touches

After DA wiped things several times, I sat down and organized all my custom configs into 5 layers according to how DA preserves or overwrites them. With configs in the right layer, DA updates and CustomBuild rebuilds will not destroy anything.

• DA-preserved — /usr/local/directadmin/scripts/custom/ stores custom hooks that DA promises never to touch. Put anything that absolutely must persist here.
• Template overrides — /usr/local/directadmin/data/templates/custom/ overrides nginx/Apache vhost templates. When DA rewrites configs, it reads the templates here instead of the defaults.
• Numbered ini — Files named 99-custom-*.ini override PHP settings. The 99 prefix makes them load last, so their values override the defaults and survive ./build php_*.
• Self-healing cron — Files in /etc/cron.d/* call scripts in scripts/custom/ to re-apply rules daily (like the ModSec self-heal above), in case DA wipes anything.
• Post-build hooks — custombuild/custom/post.<target>.sh runs after CustomBuild finishes building. Use it for cleanup tasks such as removing listeners on the TAB port, and so on.

With all 5 layers in place, most of the worry about DA updates deleting our configs goes away, because each layer has its own mechanism for surviving.

2-Tier Incident Detection — catching both fast and slow failures

Another thing I set up after the server stabilized was two-tier incident detection, because failures come in two forms — ones that make noise immediately, and ones that quietly get worse over time. A single detector cannot cover both.

I deliberately made the recovery script cautious, rather than restarting things indiscriminately whenever it finds a problem, because DA build cycles routinely stop and start services. The core logic has four points:

• 5-second debounce — Do not act immediately on a signal. Wait, then re-check to filter out noise from DA build cycles that temporarily stop services.
• CustomBuild detection — Check with pgrep -af '/usr/local/directadmin/custombuild/build'. If a build is running, skip recovery entirely, because services going down during a build is normal.
• Strict pattern matching — The playbook returns non-zero if the error does not match a known pattern. Do not guess at a fix; leave unfamiliar problems for a human to inspect.
• No generic restart — There is no "let's try it" restart, because touching a service without knowing the root cause is risky and could make things worse.

Lessons from the whole series

To close the series, I want to list the main lessons from this cross-distro migration for anyone planning to do the same.

• Order matters — Fixes must happen in the right order. Mine was SECLEVEL first, then TAB-listen, then per-domain tests, then notifying users. Do not keep running ./build rewrite_confs repeatedly hoping it will fix the problem on its own. Fix the root cause first.
• Cross-distro migrations will bring layers of problems — Expect more than 10 issues, because OS defaults change, DA backward compatibility is not 100%, and the stack differs too (pure-ftpd to proftpd, spamassassin to rspamd). There is no way everything will migrate smoothly.
• Customer comms — Notify users at least 3 days in advance, send status updates every 30 minutes during cutover, and respond quickly during the first 24 to 48 hours. Do not promise "it will work immediately," because it never really is immediate.
• Document everything — Back up configs before every change, record the actual commands you run, and diff the system state before and after so you know what changed.
• Watch for accidental secret leaks — Put credentials in .gitignore, search-replace IPs/domains/usernames before every publication, and pass bot tokens through config on tmpfs rather than hardcoding them in files.

If you are moving across distros like this, I recommend taking snapshots/backups at every layer before starting. At least three layers — a PVE snapshot, vzdump, and a DA backup — so you can roll back at any level if something goes wrong.

This series has mainly been an overview. For deeper topics such as pigz wrapper benchmarks and safety analysis, DA dataskq architecture, the full csf-trusted-sweep, ZFS scrubs and long-term maintenance, the PHP 6-slot strategy, or ModSec3 + OWASP CRS tuning for Thai shops, let me know where you want more detail and I will write separate articles.

That completes the series. I will come back and fix anything missing or incorrect. Feel free to adapt it to your own server environment; the principles are the same throughout.