EST. 2026 — Notes from real server work SYSADMIN · SECURITY · AI · DIRECTADMIN Issue 01
THE SYSADMIN ALMANAC
Home / Tags / Debian
Debian

#Debian

14 articles
OS / Distro
The Service Is Running but Dead — systemd restart Cannot Help
Linux

The Service Is Running but Dead — systemd restart Cannot Help

A long network outage broke DNS. A daemon holding a session with an external API failed to log in 194 times, then hung halfway through the process. It never exited, so the system never restarted it — why I stopped checking whether a process was alive and started measuring data freshness instead.

#systemd#linux#monitoring#alerting
9 min read·9 Oct
Tell a Broken Link from an ISP Outage Before Restarting Anything
Linux

Tell a Broken Link from an ISP Outage Before Restarting Anything

A typical watchdog restarts NetworkManager when ping fails, which is wrong half the time because no internet access may mean the ISP is down, not that the adapter has failed. This article distinguishes three states using nmcli, then escalates from gentle recovery to PCI remove and rescan without rebooting the machine.

#linux#networking#systemd#debian
9 min read·20 Sep
Every File Is Backed Up, but I Still Can't Rebuild the Server
Backup

Every File Is Backed Up, but I Still Can't Rebuild the Server

The database dump is complete, the code tar is complete, and all the secrets are there, but put them on a blank machine and the site will not come up, because the things that actually make the server work were never in the backup files. This article lists what disappears with the machine and the recovery sequence I wrote for my future self.

#backup#disaster-recovery#systemd#debian
11 min read·12 Sep
Backups Succeeded Nightly, but Offsite Uploads Failed for 3 Weeks
Backup

Backups Succeeded Nightly, but Offsite Uploads Failed for 3 Weeks

cron ran every night, local files were complete every day, and the log said done, but offsite storage had not changed for 3 weeks. The first 8 nights of backups were already gone for good because local rotation had deleted them first. This article examines the two gaps that let the failure stay silent for so long.

#backup#rclone#s3#cloudflare
10 min read·11 Sep
MariaDB on defaults — a 128M buffer pool on a 55 GB RAM server
Linux

MariaDB on defaults — a 128M buffer pool on a 55 GB RAM server

A 16 core server with 55 GB RAM and an SSD, but MariaDB still had the default 128 MB buffer pool. The result was 50 million disk reads in 5 days. The full before-and-after settings, service name and config path traps on DirectAdmin/Debian, rollback steps, and another 500 MyISAM tables waiting for conversion.

#mariadb#mysql#performance#tuning
10 min read·13 Aug
Read-only Debian shared-host security audit — avoid banning yourself
Security

Read-only Debian shared-host security audit — avoid banning yourself

Audit the security of a Debian server running DirectAdmin in read-only mode first — but the first trap is that scan traffic itself triggers CSF's brute-force rules and gets your own IP banned. This article lays out a safe audit sequence + areas to check in 2024-2026

#security#hardening#csf#cve
9 min read·16 Jul
Legacy Apps Break After Migration — MariaDB Strict Mode Errors
Linux

Legacy Apps Break After Migration — MariaDB Strict Mode Errors

After moving a website from an old server to a new one, the registration form breaks even though the code has not changed — the cause is STRICT_TRANS_TABLES in the sql_mode of newer MariaDB versions, with fixes at 3 levels, from conf.d to per-app SET SESSION

#mariadb#mysql#migration#server-migration
7 min read·7 Jul
Using Claude to Upgrade Proxmox 8 to 9 and Rebuild Windows VMs
Linux

Using Claude to Upgrade Proxmox 8 to 9 and Rebuild Windows VMs

A real case using VS Code + Claude to upgrade an entire Proxmox cluster from 8 to 9, then delete and rebuild all Windows VMs for VDI without ever SSHing into a host myself. I have laid out the commands and scripts Claude generated during the actual work as steps that a person or AI can read and run again.

#claude-code#ai#vscode#proxmox
13 min read·20 Jun
qcow2 or ZFS — Why I Switched After 12 Years on the Same Server
Linux

qcow2 or ZFS — Why I Switched After 12 Years on the Same Server

The reasons behind my storage choices for a Proxmox server that has run for 12 years without a reinstall — why I chose qcow2 in 2014, why I only moved to ZFS in 2026, and what actually changed the calculus.

#zfs#qcow2#proxmox#storage
12 min read·21 May
DirectAdmin from CentOS 7 to Debian 12 on Proxmox (Series Index)
DirectAdmin

DirectAdmin from CentOS 7 to Debian 12 on Proxmox (Series Index)

Notes on migrating a DirectAdmin shared host from CentOS 7 (EOL) to Debian 12 on Proxmox, serving 80+ customers, split into 5 parts with an overview of the process and links to every part

#migration#server-migration#directadmin#custombuild
8 min read·20 May
Moving DA from CentOS 7 to Debian 12, Part 4 — Cutover and 17 Issues
DirectAdmin

Moving DA from CentOS 7 to Debian 12, Part 4 — Cutover and 17 Issues

The actual cutover, where the near-0 downtime plan fell apart because of the DA Legacy license and slow qcow2 reads, ended with 1 hour of downtime and 17 issues in quick succession after the switch — critical web stack outages, medium issues affecting subsets of services, and minor debugging notes.

#cutover#migration#directadmin#qcow2
13 min read·20 May
DA Migration Part 3 — DirectAdmin + CustomBuild on Debian 12
DirectAdmin

DA Migration Part 3 — DirectAdmin + CustomBuild on Debian 12

Installing DirectAdmin through setup.sh, then assembling a stack with multiple PHP versions, MariaDB, ModSecurity, and rspamd using CustomBuild on a staging VM behind NAT, including pitfalls I ran into myself.

#directadmin#custombuild#php-fpm#mariadb
11 min read·20 May
Moving DA to Debian 12, Part 2 — Switching Storage to a ZFS Mirror
Linux

Moving DA to Debian 12, Part 2 — Switching Storage to a ZFS Mirror

Replace the three storage layers of mdadm RAID1 + ext4 + qcow2 with a single layer of ZFS mirror + raw zvol while the old VM stays online, by degrading RAID to a single disk.

#zfs#raid#storage#snapshot
11 min read·20 May
DA CentOS 7 → Debian 12 on Proxmox Part 1 — NAT pre-stage
DirectAdmin

DA CentOS 7 → Debian 12 on Proxmox Part 1 — NAT pre-stage

Preparing a new VM on an internal bridge with NAT while the old VM continues serving customers, reducing downtime when migrating DirectAdmin from CentOS 7 to Debian 12 on Proxmox.

#nat#proxmox#kvm#migration
11 min read·20 May