#S3 / Object Storage
2 articles
Backup / DR
One Codebase, Two Roles: Hosted Web, Worker at Home
The whole system used to run on one machine behind a home internet connection. When that machine went down, the team could not use anything. I split it into two roles using the same codebase — the web runs on an always-on host, while heavy work stays on the home machine, which pulls jobs itself, with an atomic job-claim table and an 18-step cutover plan with a rollback path.
Backups Succeeded Nightly, but Offsite Uploads Failed for 3 Weeks
cron ran every night, local files were complete every day, and the log said done, but offsite storage had not changed for 3 weeks. The first 8 nights of backups were already gone for good because local rotation had deleted them first. This article examines the two gaps that let the failure stay silent for so long.