EST. 2026 — Notes from real server work SYSADMIN · SECURITY · AI · DIRECTADMIN Issue 01
THE SYSADMIN ALMANAC
Home / Tags / Linux
Linux

#Linux

20 articles
OS / Distro
VM Sees the Dongle but Has No Internet Without a Windows ECM Driver
Hardware

VM Sees the Dongle but Has No Internet Without a Windows ECM Driver

After passing a USB LTE dongle through to a Windows VM, the system sees the device but no network adapter appears. The cause is neither the SIM nor port passthrough, but the dongle exposing a CDC ECM interface that Windows has no built-in driver for, along with storage-mode pitfalls and dhclient overwriting the host's resolv.conf.

#proxmox#hardware#windows#linux
9 min read·21 Oct
The Monitor That Hammered Devices Until the Whole Farm Went Down
Linux

The Monitor That Hammered Devices Until the Whole Farm Went Down

A monitoring loop that runs every 30 seconds and tries to recover every device that fails a ping sounds useful, until one device hangs and gets hammered repeatedly until the whole bus goes down. This article walks through ruling out hardware, tracking what was accessing the devices, and the settings that fixed it.

#monitoring#linux#python#systemd
10 min read·17 Oct
Egress Firewall Breaks Domain Checks — Tests as root Pass
Security

Egress Firewall Breaks Domain Checks — Tests as root Pass

The domain order page reported every name as unavailable, even names I made up. The cause was an outbound DROP policy with port 43 missing from the allowed port list after an operating system migration. The trap was that tests as root passed every time. Testing must use the account that actually runs the app.

#firewall#csf#security#networking
9 min read·10 Oct
Web vs. Worker — mkdir 0700 Disables Inherited ACL Permissions
Linux

Web vs. Worker — mkdir 0700 Disables Inherited ACL Permissions

One app runs as two users, with php-fpm handling web requests and a separate account running the queue. Even with default ACLs fully configured, they still cannot write to each other's directories. The cause is mkdir with mode 0700, which sets a mask that wipes out the effective ACL permissions, plus a deletion trap that requires permissions on the directory, not the file.

#linux#permissions#posix-acl#php-fpm
9 min read·9 Oct
The Service Is Running but Dead — systemd restart Cannot Help
Linux

The Service Is Running but Dead — systemd restart Cannot Help

A long network outage broke DNS. A daemon holding a session with an external API failed to log in 194 times, then hung halfway through the process. It never exited, so the system never restarted it — why I stopped checking whether a process was alive and started measuring data freshness instead.

#systemd#linux#monitoring#alerting
9 min read·9 Oct
Tell a Broken Link from an ISP Outage Before Restarting Anything
Linux

Tell a Broken Link from an ISP Outage Before Restarting Anything

A typical watchdog restarts NetworkManager when ping fails, which is wrong half the time because no internet access may mean the ISP is down, not that the adapter has failed. This article distinguishes three states using nmcli, then escalates from gentle recovery to PCI remove and rescan without rebooting the machine.

#linux#networking#systemd#debian
9 min read·20 Sep
Read Device Uptime Without SNMP/SSH/API Using TCP Timestamps
Network

Read Device Uptime Without SNMP/SSH/API Using TCP Timestamps

Many IP cameras, door controllers, and embedded devices have no SNMP, SSH, or API for querying uptime. But if they return TCP timestamps, I can read their boot time remotely without logging in even once.

#networking#monitoring#linux#python
7 min read·20 Aug
The Web User's crontab Isn't the One Running Your Jobs
Linux

The Web User's crontab Isn't the One Running Your Jobs

Schedule changes saved through the web still leave jobs running at the old times because the script writes the web user's crontab, not the user’s crontab that cron actually reads. How to spot it in 3 commands, the sudoers drop-in I used, and how to design it so this cannot happen again.

#cron#linux#sysadmin#debugging
7 min read·19 Aug
One Query Hung for 54 Minutes, Causing Sitewide 504s via MyISAM Locks
Linux

One Query Hung for 54 Minutes, Causing Sitewide 504s via MyISAM Locks

A single 30 MB table caused 504s on every page across the site, even with 9.5 GB of RAM still free — I traced information_schema.PROCESSLIST to a single SELECT that had been stuck for 54 minutes on a MyISAM table, cascading into metadata locks across the server, with an immediate fix and ways to prevent it from happening again.

#mariadb#mysql#myisam#innodb
10 min read·6 Aug
nginx heap overflow spans 15 years—version checks aren't enough
Security

nginx heap overflow spans 15 years—version checks aren't enough

CVE-2026-42533 affects nginx versions 0.9.6 through 1.31.2, spanning 15 years, but actual exposure depends on the config, not the version. The problem is that news reports describe narrower conditions than the real ones. Checking against those reports can return "safe" even when the config still meets the conditions.

#security#nginx#cve#vulnerability
9 min read·16 Jul
Writing Scripts on Windows for Linux — Two File Cleanup Traps
Linux

Writing Scripts on Windows for Linux — Two File Cleanup Traps

Writing helper scripts on Windows and using scp to run them on Linux is routine work, but two things keep catching me out — d:\ paths embedded in scripts turn into strange filenames in /tmp that trigger the file monitor, and token/payload files get left behind on the server. This article lays out my rules for avoiding these mistakes.

#automation#windows#linux#sysadmin
6 min read·16 Jul
USB in WSL2 with usbipd-win — Real Hardware for Linux on Windows
Linux

USB in WSL2 with usbipd-win — Real Hardware for Linux on Windows

WSL2 cannot see USB devices because it is a VM without a USB controller — usbipd-win passes devices from Windows directly to the WSL kernel with 3 commands, list/bind/attach, plus lessons from actual use: attachments drop when unplugged, the CRLF trap, and USB modems taking over the default route

#usbip#wsl2#windows#linux
7 min read·11 Jul
libmodsecurity3 + nginx — only ctl:ruleEngine exclusions work
Security

libmodsecurity3 + nginx — only ctl:ruleEngine exclusions work

ModSecurity v3 (libmodsecurity3) + nginx + OWASP CRS — the ctl:ruleRemoveById taught in the guides does nothing and fails silently. This article covers what I found after trying every approach: ctl:ruleEngine works, scoped with a Host+URI chained rule, and the exclusion file must load before CRS.

#modsecurity#nginx#security#firewall
9 min read·10 Jul
Per-user Redis on shared hosting—and the maxmemory bug that hung sites
Linux

Per-user Redis on shared hosting—and the maxmemory bug that hung sites

Designing one Redis instance per user on shared hosting with a systemd template + cgroup — and lessons from a maxmemory bug where setting it above the cgroup limit caused intermittent WooCommerce hangs, plus the 80% rule and a guard script to prevent recurrence

#redis#cache#systemd#cgroup
10 min read·9 Jul
WordPress 500s, Empty debug.log — The Fatal Is in the Apache Log
DirectAdmin

WordPress 500s, Empty debug.log — The Fatal Is in the Apache Log

WordPress returns 500 only on portfolio pages, but debug.log is empty — because the fatal error during theme rendering happens before WP logging is ready. The answer is in the apache error log (proxy_fcgi), along with a fix for a child theme with a stale template copied from an older parent version.

#wordpress#php-fpm#debugging#logging
8 min read·8 Jul
AdSense Says ads.txt Is Missing: Tracing the Logs Layer by Layer
Network

AdSense Says ads.txt Is Missing: Tracing the Logs Layer by Layer

The AdSense Sites page reports ads.txt as "Not found" even though the file exists and is correct — a forensic investigation using curl + access logs to prove, layer by layer, whether Googlebot visited and what it received. Case closed: nothing was blocking it; Google's own status was stale.

#adsense#nginx#cloudflare#logging
9 min read·8 Jul
Recovering Old Website Images with Wayback Machine CDX API + Python
Backup

Recovering Old Website Images with Wayback Machine CDX API + Python

A website over a decade old had broken images throughout because it hotlinked to a dead CDN — I wrote a Python script to list images from the Wayback Machine CDX API and restore them to their original paths without touching a single character of HTML, with the full script I actually used.

#wayback#python#backup#disaster-recovery
9 min read·7 Jul
Legacy Apps Break After Migration — MariaDB Strict Mode Errors
Linux

Legacy Apps Break After Migration — MariaDB Strict Mode Errors

After moving a website from an old server to a new one, the registration form breaks even though the code has not changed — the cause is STRICT_TRANS_TABLES in the sql_mode of newer MariaDB versions, with fixes at 3 levels, from conf.d to per-app SET SESSION

#mariadb#mysql#migration#server-migration
7 min read·7 Jul
Lost the .com: Move to .net on DirectAdmin Without Data Loss
DirectAdmin

Lost the .com: Move to .net on DirectAdmin Without Data Loss

A .com domain expired and ended up with a domain squatter — how I moved the whole site (files/mail/DNS/SSL) to a new .net domain on DirectAdmin with a single CMD_CHANGE_DOMAIN command, plus 4 follow-up tasks that do not happen automatically

#directadmin#dns#ssl#letsencrypt
9 min read·6 Jul
Edit Videos with Claude Using Thai Prompts, No Video Editor Needed
AI

Edit Videos with Claude Using Thai Prompts, No Video Editor Needed

How I actually use Claude in VS Code to trim, resize, join, and add background music to videos by typing in Thai, without memorizing ffmpeg commands or opening a video editor. Includes getting AI to compose music, NVIDIA GPU acceleration, and Mac and Linux instructions for people who have never used Claude.

#claude-code#ffmpeg#video-editing#vscode
11 min read·19 Jun