CDP Still Responds but Commands Fail After Chrome Updates Itself

On the machine I use for automation, several jobs need a real browser because the target pages are rendered entirely with JavaScript. Plain HTTP requests return only an empty shell. My approach is to leave Chrome running in remote debugging mode and have scripts control it directly through the Chrome DevTools Protocol.

After using this setup for quite a while, I ran into two problems that hurt in different ways. The first was multiple jobs fighting over the same browser and mixing up their results. The second was Chrome updating itself one day and overwriting files that the existing process was still using, producing symptoms so misleading that I spent half a day chasing the wrong thing. This article covers both.

The most misleading symptoms I have seen

That morning, every job that needed a browser failed, but every health check I had said everything was fine.

curl -s http://127.0.0.1:9222/json/version
# {"Browser":"Chrome/152.0.7977.65","Protocol-Version":"1.3", ...}   ตอบปกติ

curl -s http://127.0.0.1:9222/json | head -20
# [{"description":"","id":"A1B2...","title":"","type":"page","url":"", ...}]

Look closely at the tab it returned: both title and url are empty, even though some page should have been left open. That was the only clue available from the start, but I did not pay much attention to it at the time.

The picture became clearer when I connected over WebSocket and tried issuing actual commands:

import json, websocket

ws = websocket.create_connection("ws://127.0.0.1:9222/devtools/page/A1B2...", timeout=20)
ws.send(json.dumps({"id": 1, "method": "Browser.getVersion"}))
print(ws.recv())                       # ตอบ

ws.send(json.dumps({"id": 2, "method": "Page.navigate",
                    "params": {"url": "https://example.com"}}))
print(ws.recv())                       # ตอบ

ws.send(json.dumps({"id": 3, "method": "Runtime.evaluate",
                    "params": {"expression": "1+1", "returnByValue": True}}))
print(ws.recv())                       # ค้างจนหมดเวลา ไม่มีอะไรกลับมาเลย

Browser.getVersion responded, and Page.navigate responded, but Page.enable and Runtime.evaluate stayed completely silent until they timed out.

First, prove the library is not at fault

Before accepting any conclusion, I tried three approaches against the same browser to rule out the libraries entirely.

# ทาง 1 - websocket-client ตามปกติ                       -> ค้าง
# ทาง 2 - websocket-client แบบปิด compression
ws = websocket.create_connection(url, timeout=20, suppress_origin=True,
                                 enable_multithread=True, compression=None)
# ทาง 3 - Playwright ต่อเข้า CDP ที่เปิดอยู่
from playwright.sync_api import sync_playwright
with sync_playwright() as p:
    b = p.chromium.connect_over_cdp("http://127.0.0.1:9222")
    page = b.contexts[0].pages[0]
    print(page.evaluate("1+1"))        # ค้างเหมือนกัน

All three failed in the same way, which meant the problem was in the browser, not in what was talking to it.

The real cause: the binary changed under a running process

The decisive check was comparing two versions — the version reported by the process that was still running, and the version of the file currently on disk.

# รุ่นของไฟล์บนดิสก์ตอนนี้
(Get-Item "C:\Program Files\Google\Chrome\Application\chrome.exe").VersionInfo.ProductVersion
# 152.0.7977.83

# รุ่นที่โปรเซสที่รันอยู่รายงานผ่าน CDP
(Invoke-RestMethod http://127.0.0.1:9222/json/version).Browser
# Chrome/152.0.7977.65

They were two minor versions apart. That was the whole answer. Chrome had updated itself and overwritten the files while the old process was still running. The old browser process remained in memory and could still accept commands, but it could no longer create new renderers because the binary it needed to launch had changed. The tabs I could see were empty shells, with an id but nothing inside.

The fix was to stop all the processes and start them again. That was it. No downgrade, no library fixes, no version pinning needed, because the files on disk already contained a complete new version. The only problem was the old process still running with a version that no longer existed on disk.

The trap that defeated the first fix: killing the wrong profile

The first time I tried to stop the processes, every symptom stayed the same because I targeted profile names from memory, and those did not match the actual ones.

# ดูของจริงก่อนว่าโปรเซสไหนถือพอร์ตไหน และใช้โปรไฟล์ไหน
Get-CimInstance Win32_Process -Filter "Name='chrome.exe'" |
  Where-Object { $_.CommandLine -match 'remote-debugging-port' } |
  Select-Object ProcessId,
    @{n='port';    e={ [regex]::Match($_.CommandLine,'remote-debugging-port=(\d+)').Groups[1].Value }},
    @{n='profile'; e={ [regex]::Match($_.CommandLine,'user-data-dir=([^"]+)').Groups[1].Value }} |
  Format-Table -AutoSize

# ปิดเฉพาะกลุ่มที่ใช้โปรไฟล์นั้นจริง ๆ
Get-CimInstance Win32_Process -Filter "Name='chrome.exe'" |
  Where-Object { $_.CommandLine -match 'user-data-dir=C:\\cdp-job-a' } |
  ForEach-Object { Stop-Process -Id $_.ProcessId -Force }

Health checks must issue real commands, not just check an open port

After that, I wrote a check and repair tool with two rules. First, it must always issue a command that actually needs the renderer. Second, if everything is healthy, it must do nothing; running it repeatedly must produce the same result.

import json, subprocess, sys, time, urllib.request, websocket

def evaluate_ok(port, timeout=15):
    """ตรวจของจริง - ต้องได้คำตอบจาก renderer ไม่ใช่แค่พอร์ตตอบ"""
    try:
        tabs = json.load(urllib.request.urlopen(
            "http://127.0.0.1:%d/json" % port, timeout=5))
        page = next(t for t in tabs if t.get("type") == "page")
        ws = websocket.create_connection(page["webSocketDebuggerUrl"], timeout=timeout)
        ws.send(json.dumps({"id": 1, "method": "Runtime.evaluate",
                            "params": {"expression": "1+1", "returnByValue": True}}))
        got = json.loads(ws.recv())
        ws.close()
        return got.get("result", {}).get("result", {}).get("value") == 2
    except Exception:
        return False

def repair(port, profile, launcher):
    if evaluate_ok(port):
        print("[OK] port %d healthy" % port)
        return 0
    print("[WARN] port %d answers but cannot evaluate - restarting" % port)
    kill_by_profile(profile)
    time.sleep(3)
    subprocess.run(["schtasks", "/Run", "/TN", launcher], capture_output=True)
    for _ in range(20):
        time.sleep(3)
        if evaluate_ok(port):
            print("[OK] port %d back" % port)
            return 0
    print("[ERR] port %d still dead" % port)
    return 1

I put this check as the first step of every job that needs a browser, before the actual work starts:

@echo off
set PY=%USERPROFILE%\anaconda3\envs\ml313\python.exe
"%PY%" C:\jobs\cdp_restart.py --port 9222
if errorlevel 1 (
  echo cdp 9222 unavailable, aborting >> C:\jobs\run.log
  exit /b 1
)
"%PY%" C:\jobs\collect.py >> C:\jobs\run.log 2>&1

Do not add it to the bat file that starts the browser itself, because that would create a loop of calls. I also leave it out of the bat files I use for manual testing, so it does not restart the browser I am currently watching.

An extra benefit of adding this check to every job was discovering that one port had simply been closed all day, even though three jobs used it. The browser had died silently during the day without anyone noticing. Now the check starts it back up automatically before each job begins.

Separate ports and profiles for each group of jobs

Another thing to sort out from the start is to keep multiple jobs from using the same browser. This has caught me three times already. One script would navigate away from a tab another job was using, and the resulting screenshot would show the other job's page in its entirety. Tracking down the cause was very difficult because neither script had anything wrong with its code.

The rule I use today is one job group, one port, one profile. I set these as environment variables in each group's scripts so I do not have to remember them.

@echo off
rem start-cdp-job-a.bat - เบราว์เซอร์ประจำกลุ่มงาน A
start "" "C:\Program Files\Google\Chrome\Application\chrome.exe" ^
  --user-data-dir=C:\cdp-job-a ^
  --remote-debugging-port=9222 ^
  --remote-allow-origins=* ^
  --no-first-run --no-default-browser-check ^
  --disable-features=Translate,MediaRouter

import os
os.environ.setdefault("CDP_PORT", "9222")          # กลุ่มงาน A
os.environ.setdefault("CDP_PROFILE", r"C:\cdp-job-a")

If you really must share a port, open your own tab

Sometimes sharing a browser with another job is unavoidable. The one rule that must never be broken is to leave existing tabs alone. Open a new tab of your own, and always close it when you are done.

import json, urllib.parse, urllib.request

BASE = "http://127.0.0.1:9222"

def new_tab(url="about:blank"):
    req = urllib.request.Request(BASE + "/json/new?" + urllib.parse.quote(url, safe=""),
                                 method="PUT")          # ต้องเป็น PUT
    return json.load(urllib.request.urlopen(req, timeout=10))

def close_tab(tab_id):
    urllib.request.urlopen(BASE + "/json/close/" + tab_id, timeout=10).read()

tab = new_tab("https://example.com")
try:
    ...                      # ทำงานบนแท็บของตัวเองเท่านั้น
finally:
    close_tab(tab["id"])     # ปิดทุกครั้ง ไม่งั้นแท็บค้างสะสมจนกินแรม

1. Give every job group its own port and profile. Put them in the scripts so there is nothing to remember.
2. If a port must be shared, open your own tab with PUT and always close it afterward.
3. Health checks must issue commands that need the renderer, not just check whether the port responds.
4. The repair tool must stop processes based on their actual command lines, not names from memory.
5. Put the check first in every job that needs a browser, except the one that starts the browser itself.

Conclusion

I came away with two conclusions.

First, a health check that does not do the same work as the real job is meaningless. An open port, a process that is still running, a page returning 200 — all of these only signal that something is there. They do not tell me whether it can do the work I need. If I am going to check, I should check with the commands the real job uses.

Second, resources shared without rules always turn into bugs that cannot be reproduced. A single browser receiving commands from multiple jobs at once is one example. The cost of separating them is a few hundred more megabytes of RAM and a little extra disk space, which is far cheaper than the time spent figuring out why a screenshot belongs to another job.

If you run multiple jobs that need a browser on one machine, take a look when you have time to see whether they currently share a port. If I have left gaps in the explanation or made any typos, I will come back and fix them later.