A 25-Hour Power Outage Left Scheduled Jobs Missing

The machine I use for daily automated jobs happened to lose power three times in three days. The first two outages lasted a few minutes, but the last one lasted a full 25 hours. When power returned, I assumed everything would resume as usual. It turned out that only some of it did. Some jobs caught up on all their outstanding work, while another group simply lost a whole day without leaving any trace in the logs.

This article follows on from Running Automated Jobs on a Remote Windows Machine via SSH + Task Scheduler. That article covered setting up and launching jobs. This one addresses the question that came later: when the machine disappears for a day, where do the jobs that should have run during that time go?

First, Find Out Exactly When the Machine Went Down and Came Back

Before deciding which jobs went missing, I needed to establish exactly when the machine was unavailable. Windows already records two event IDs for this: 6008 (an unexpected shutdown) and 41 (the kernel did not shut down properly).

Get-WinEvent -FilterHashtable @{LogName='System'; Id=6008,41; StartTime=(Get-Date).AddDays(-14)} |
  Select-Object TimeCreated, Id, ProviderName |
  Format-Table -AutoSize

Pair these with the latest system startup time to see how long the outage lasted:

Get-WinEvent -FilterHashtable @{LogName='System'; Id=6005} -MaxEvents 10 |
  Select-Object TimeCreated
(Get-CimInstance Win32_OperatingSystem).LastBootUpTime

The Setting to Enable: StartWhenAvailable

This is the central point of the article. The default behavior of Windows Task Scheduler is if the machine is off at the scheduled time, discard that run without catching up. A job scheduled for three in the morning lost an entire day because the machine happened to be off. When the machine came back at six in the evening, it did nothing. It simply waited until three the next morning.

Check the whole machine for jobs that still have this setting disabled:

Get-ScheduledTask | Where-Object { $_.State -ne 'Disabled' } |
  Select-Object TaskName,
    @{n='StartWhenAvailable'; e={ $_.Settings.StartWhenAvailable }},
    @{n='Triggers'; e={ ($_.Triggers | ForEach-Object { $_.CimClass.CimClassName }) -join ',' }} |
  Where-Object { $_.StartWhenAvailable -eq $false } |
  Format-Table -AutoSize

Enable it one job at a time or for the whole set with Set-ScheduledTask — I recommend adding RestartCount and RestartInterval as well, in case a run fails because the network has not come back yet just after boot:

$names = @('JobA','JobB','JobC')
foreach ($n in $names) {
  $s = New-ScheduledTaskSettingsSet -StartWhenAvailable `
        -RestartCount 3 -RestartInterval (New-TimeSpan -Minutes 10) `
        -MultipleInstances IgnoreNew -ExecutionTimeLimit (New-TimeSpan -Hours 2)
  Set-ScheduledTask -TaskName $n -Settings $s | Out-Null
  Write-Output "[OK] $n"
}

Set this every time you create a new task with a time-based trigger. I have added it to my own checklist, because there is no way I will remember it when creating a new job late at night.

What Survived: A Dispatcher That Checks the Time Itself

Another group of jobs on the same machine was not tied directly to Windows triggers. I have a single task that wakes up every 20 minutes and opens a state file to check "which jobs are due?" If the current time is past a job's scheduled time, it launches that job immediately, regardless of how many hours have passed.

The state file is a single plain JSON file that records when each job last ran and when it is next scheduled:

{
  "jobA": { "every": "daily",  "at": "03:00", "last": "2026-08-29", "next": "2026-08-30T03:00" },
  "jobB": { "every": "daily",  "at": "15:00", "last": "2026-08-29", "next": "2026-08-30T15:00" },
  "jobC": { "every": "weekly", "dow": 2, "at": "06:30", "last": "2026-08-25", "next": "2026-09-01T06:30" }
}

The dispatcher is very short. Its core is a single line: now >= next:

import json, subprocess, datetime as dt
from pathlib import Path

STATE = Path(r"C:\jobs\sched_state.json")
RUNNERS = {"jobA": r"C:\jobs\run_a.bat",
           "jobB": r"C:\jobs\run_b.bat",
           "jobC": r"C:\jobs\run_c.bat"}

def main():
    st = json.loads(STATE.read_text(encoding="utf-8"))
    now = dt.datetime.now()
    changed = False
    for name, cfg in st.items():
        nxt = dt.datetime.fromisoformat(cfg["next"])
        if now < nxt:                       # ยังไม่ถึงคิว
            continue
        late = int((now - nxt).total_seconds() // 60)
        print("[RUN] %s (late %d min)" % (name, late), flush=True)
        rc = subprocess.run([RUNNERS[name]], capture_output=True, timeout=7200).returncode
        print("[%s] %s rc=%d" % ("OK" if rc == 0 else "ERR", name, rc), flush=True)
        cfg["last"] = now.date().isoformat()
        cfg["next"] = plan_next(cfg, now).isoformat(timespec="minutes")
        changed = True
    if changed:
        STATE.write_text(json.dumps(st, indent=2, ensure_ascii=False), encoding="utf-8")

if __name__ == "__main__":
    main()

Task Scheduler calls this dispatcher every 20 minutes, using a repeating trigger that is completely independent of the actual jobs' scheduled times. When the machine came back at six in the evening, the dispatcher's next run was at ten past six. It saw that the three-in-the-morning job had not run and launched it immediately. The logs confirmed that the daily jobs covered every day throughout the outage.

Stagger Job Start Times So They Do Not All Wake Up Together

Once everything comes back together after power returns, the next problem appears immediately. If five jobs are all overdue, the dispatcher launches them in the same minute. CPU, disk, and outbound bandwidth usage all spike at once, and the jobs may all fail together too.

I addressed this by giving each job its own spacing when calculating its next scheduled time, so the times do not coincide, and by limiting how many jobs the dispatcher can launch in a single run.

MAX_PER_TICK = 2          # หนึ่งรอบสั่งได้ไม่เกินสองงาน ที่เหลือรอรอบหน้า
SPREAD_MIN = {"jobA": 0, "jobB": 7, "jobC": 13}   # เยื้องเวลากันคนละไม่กี่นาที

def plan_next(cfg, now):
    base = dt.datetime.combine(now.date() + dt.timedelta(days=1),
                               dt.time.fromisoformat(cfg["at"]))
    return base + dt.timedelta(minutes=SPREAD_MIN.get(cfg["name"], 0))

There is only one reason for this: spread the load so the peaks do not overlap during recovery. For jobs that genuinely need to run in order, such as fetching data before publishing it, it is better to bundle them into a single job and call them sequentially inside it than to hope their timing lines up.

Do Not Reset State Across the Board

During recovery, I made one mistake that deserves a written reminder. I wrote a simple state-reset script that stamped today's date into last for every job at once, thinking I could give everything a fresh start.

The version I use now accepts pairs of job names and actual last-run dates, then updates only the specified jobs:

# python fix_state.py jobA=2026-08-29 jobC=2026-08-25
import sys, json
from pathlib import Path

STATE = Path(r"C:\jobs\sched_state.json")
st = json.loads(STATE.read_text(encoding="utf-8"))
for arg in sys.argv[1:]:
    name, _, day = arg.partition("=")
    if name not in st:
        sys.exit("unknown job: %s" % name)
    st[name]["last"] = day
    st[name].pop("next", None)          # ให้ตัวสั่งงานคำนวณใหม่เอง
    print("[OK] %s last=%s" % (name, day))
STATE.write_text(json.dumps(st, indent=2, ensure_ascii=False), encoding="utf-8")

Task State Is Not Process State

Another thing that led me to misread the situation during recovery was the assumption that disabling a task would also stop whatever it was running. That is not true.

# ฝั่ง task
schtasks /Query /TN JobA /FO LIST /V | Select-String 'Status|Last Run|Last Result|Next Run'
# ฝั่งโปรเซส
Get-CimInstance Win32_Process -Filter "Name='python.exe'" |
  Select-Object ProcessId, CreationDate, CommandLine | Format-List

Small Traps That Waste a Lot of Time

schtasks /change /st prompts for a password — For a task configured to run under a user account, changing its time with schtasks /change /tn JobA /st 04:00 prompts for that account's password. This is unusable from an unattended script. Use PowerShell instead and that resolves it.

Set-ScheduledTask -TaskName 'JobA' -Trigger (New-ScheduledTaskTrigger -Daily -At '04:00')

Check the right log file — I nearly concluded that one job was stuck because I had opened an old log in the parent folder, while the actual log had moved to a subfolder at some point without me realizing it. A more reliable approach is to look beyond the log's name and check when the job's output file was last modified.

Get-ChildItem C:\jobs\out -Recurse -File |
  Sort-Object LastWriteTime -Descending |
  Select-Object -First 5 LastWriteTime, Length, FullName

1. First, establish the outage window from Event 6008/41. Do not guess from memory.
2. Enable StartWhenAvailable for every task with a time-based trigger, then check the whole machine again.
3. For jobs that truly cannot afford missed runs, move them from direct triggers to a dispatcher that checks the time itself.
4. Restore state one job at a time using actual last-run dates. Never stamp the same value throughout the file.
5. Verify that jobs are actually running again from output file timestamps, not log entries.

Conclusion

I came away with three conclusions.

First — Task Scheduler handles the question "what should happen when the time comes?" well, but it cannot handle "what should happen when that time has already passed?" unless you tell it beforehand. The default is to discard the run.

Second — For jobs that cannot afford missed runs, tie them to the job's latest state rather than the clock. Record when they last ran and decide based on elapsed time. This approach tolerates the machine being unavailable for any length of time.

Third — During recovery, do not sweep the whole board with one broom. Jobs with longer intervals will be pushed back another cycle without anyone knowing, causing damage that remains invisible for weeks.

In fact, with power outages this frequent, I should already have a UPS. When a job dies halfway through a run after I have already paid for calls to an external service, that money is simply wasted. I am thinking about this too. Choose whichever approach works for you. If I have left anything out or made any typos, I will come back and fix them later.