Thai Text Breaks Windows Scripts with Misleading Syntax Errors

I was writing a PowerShell script to configure the LAN adapter on a machine at a work site. I put Thai messages in Write-Host so the person running it could understand them. It passed validation, but when I double-clicked it to run it for real, I got Missing closing '}' in statement block, pointing to a line with nothing wrong on it. I spent a long time counting braces and revised the syntax several times. In the end, the cause was not in the characters I could see on screen, but in the file being saved as UTF-8 without a BOM.

After that, I went back through my old work notes and found 4 other variations of the same problem: .cmd files, sending JSON with curl, adb commands, and Python print. Every case followed the same pattern: the error pointed to syntax or the program itself, even though the cause was encoding. I have collected them here. I have replaced all filenames, destinations, and paths in this article with fictional values, but the error messages are real.

This is a separate layer from Windows paths left on a server and from garbled Thai data in a database. This is purely about the encoding of script files and the console on Windows.

Symptoms: Missing closing on a valid line

The script looked roughly like this, with nothing complicated.

# set-lan.ps1 - เซฟเป็น UTF-8 ไม่มี BOM (ค่าเริ่มต้นของ editor และเครื่องมือ AI ส่วนใหญ่)
param([string]$Adapter = 'Ethernet')

if (-not (Get-NetAdapter -Name $Adapter -ErrorAction SilentlyContinue)) {
    Write-Host "ไม่พบการ์ด LAN ชื่อ $Adapter"
    exit 1
}

Write-Host "กำลังตั้งค่าการ์ด LAN..."
New-NetIPAddress -InterfaceAlias $Adapter -IPAddress 10.10.10.50 -PrefixLength 24

Running it from the machine's regular console, or right-clicking and choosing Run, produced this message.

PS C:\work> .\set-lan.ps1
At C:\work\set-lan.ps1:4 char:...
+ if (-not (Get-NetAdapter -Name $Adapter -ErrorAction SilentlyContinue)) {
Missing closing '}' in statement block or type definition.
    + CategoryInfo          : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : MissingEndCurlyBrace

I guessed wrong twice. First, I thought a brace really was missing, so I counted pairs and moved the if block around. Second, I thought PowerShell 5.1 did not support some newer syntax, so I rewrote it using older syntax. Neither attempt fixed it.

What kept me looking in the wrong place was that I checked the script with pwsh, which is PowerShell 7, and it passed every time. But the machine that actually ran it opened .ps1 files with Windows PowerShell 5.1. I checked the right file with the wrong reader.

The cause: The same bytes, different mappings

One Thai character takes 3 bytes in UTF-8. The character ด is E0 B8 94. When PowerShell 5.1 finds no BOM, it reads the file as ANSI, which is CP874 on Thai Windows. This mapping uses one byte per character, so the three bytes become three characters: เ, ธ, and byte 0x94, which is a closing curly quotation mark in CP874.

PowerShell accepts curly quotation marks as string delimiters, just like straight quotation marks. When it encounters ” in the middle of "กำลังตั้งค่าการ์ด LAN...", the string closes at the character ด. The rest of the line is read as code, and the final straight quotation mark opens a new string that swallows all the following lines, including the closing brace. The parser therefore reports an unclosed block and points to the opening of the block, on a different line from the actual problem.

This can be demonstrated from the console without typing any Thai.

# ไบต์ UTF-8 ของ ด (U+0E14)
[System.Text.Encoding]::UTF8.GetBytes([string][char]0x0E14) | ForEach-Object { '{0:X2}' -f $_ }
# E0
# B8
# 94

# เอาสามไบต์เดิมมาอ่านด้วย CP874 แล้วดู code point ที่ได้
[System.Text.Encoding]::GetEncoding(874).GetString([byte[]](0xE0, 0xB8, 0x94)).ToCharArray() |
    ForEach-Object { 'U+{0:X4}' -f [int]$_ }
# U+0E40
# U+0E18
# U+201D

U+201D is a closing curly quotation mark. Thai text without ด or ณ in the string will not trigger this symptom. Some scripts therefore run while others fail, even though they were saved the same way, which makes encoding even less suspect.

Fixing .ps1: Add a BOM and check with the actual reader

The fix is one line: read the file and write it back as UTF-8 with a BOM.

$p = 'C:\work\set-lan.ps1'
[System.IO.File]::WriteAllText($p, [System.IO.File]::ReadAllText($p), [System.Text.UTF8Encoding]::new($true))

# ยืนยันสามไบต์แรก ต้องได้ EF BB BF
[System.IO.File]::ReadAllBytes($p)[0..2] | ForEach-Object { '{0:X2}' -f $_ }

Then check parsing with powershell.exe, not pwsh. I made a checker script whose contents are entirely ASCII so the checker itself would not run into the same problem.

# parsecheck.ps1 - ASCII only
param([Parameter(Mandatory = $true)][string]$Path)

$tokens = $null
$errors = $null
[System.Management.Automation.Language.Parser]::ParseFile($Path, [ref]$tokens, [ref]$errors) | Out-Null

foreach ($e in $errors) {
    '[ERR] {0}:{1} {2}' -f $e.Extent.StartLineNumber, $e.Extent.StartColumnNumber, $e.Message
}
if ($errors.Count -gt 0) { exit 1 }
'[OK] parse clean: ' + $Path

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\parsecheck.ps1 -Path C:\work\set-lan.ps1

ParseFile only parses the structure; it does not run the script. It can therefore check scripts that touch the network or system without side effects.

1. Always save .ps1 files containing non-ASCII characters as UTF-8 with a BOM. If the tool writing the file has no such option, convert it with the WriteAllText line immediately after writing.
2. Check that the first three bytes of the file are EF BB BF.
3. Run parsecheck.ps1 through powershell.exe -NoProfile, not pwsh.
4. If the script will be sent to someone else to double-click, test it once more using the same method they will actually use to open it.

While fixing this, I also found an actual difference between 5.1 and 7 unrelated to encoding: 5.1 does not accept a line break in the middle of an expression followed by a method call.

# 5.1 ไม่รับ: ขึ้นบรรทัดใหม่แล้วต่อด้วย ).IsInRole(...)
$isAdmin = ([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()
    ).IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

# เขียนแยกตัวแปร รับได้ทั้งสองรุ่น
$id = [Security.Principal.WindowsIdentity]::GetCurrent()
$principal = [Security.Principal.WindowsPrincipal]$id
$isAdmin = $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)

The .cmd case: Thai comments run as commands

This happened with a deployment file for a single-page website. It was a .cmd file with Thai text in REM and echo, running on Thai Windows with a console using CP874.

@echo off
REM ตั้งค่าปลายทาง
set HOST=example.com
set PORT=22
set KEY=C:\keys\deploy.key
REM ลบไฟล์เก่าก่อน
ssh -i "%KEY%" -p %PORT% webuser@%HOST% "rm -f /home/webuser/tmp/*.zip"
echo อัปโหลดเสร็จแล้ว

It produced three kinds of errors in a single run.

'ฟล์เก่าก่อน' is not recognized as an internal or external command,
operable program or batch file.
Bad port 'rm -f /home/webuser/tmp/*.zip'

The first line says the second half of a comment was run as a command. The second says the PORT variable is empty, shifting the ssh arguments over by one position so the remote command is read as a port number. Echoing the command reveals ssh -i "" -p followed by an empty value, meaning several set lines were swallowed as well.

cmd.exe reads batch files as bytes using the console's codepage, not as UTF-8. When it encounters the multibyte sequences for Thai characters, it splits lines in the wrong places. I did not trace it down to which byte shifted the line boundary. What I can confirm is the result: a comment line was split in half, and adjacent set lines did not execute.

My wrong guess in this case was to add chcp 65001 at the start of the file. It did not help because cmd parsed the file before the codepage change took effect.

Paths containing Thai work normally, provided they are quoted and invoked through cmd.exe /c.

& cmd.exe /c "C:\work\งานเว็บ\deploy.cmd"

The check I run before every batch file looks for bytes beyond ASCII and stops if it finds any.

# รันจาก Git Bash - เจอไบต์นอก ASCII ให้ถือว่าไฟล์ใช้ไม่ได้
if LC_ALL=C grep -nP '[^\x00-\x7F]' deploy.cmd; then
  echo "[ERR] deploy.cmd has non-ASCII bytes"; exit 1
fi
echo "[OK] deploy.cmd is ASCII only"

The curl and adb cases: Thai text lost along the way

The third case happened while testing an API that accepts Thai text. I sent JSON with curl --data, typing the text directly into the command line in Git Bash. The destination received ? characters throughout the text. I spent some time looking through the server code for a charset bug before noticing that sending it from an actual browser worked without any problems.

The text was converted through the console's codepage before reaching curl. Characters that could not be converted became ? before they even left the machine. It was a bug in the testing method, not the app. The fix was to keep Thai text out of the command line: write it to a UTF-8 file and let curl read the file.

# payload.json เซฟเป็น UTF-8 จาก editor ไม่ได้สร้างด้วย echo ใน shell
curl -sS -X POST https://example.com/api/message \
  -H 'Content-Type: application/json; charset=utf-8' \
  -H 'X-Token: <TOKEN>' \
  --data-binary @payload.json

# กรณีเป็นฟอร์ม ให้ curl อ่านค่าจากไฟล์แล้ว urlencode เอง
curl -sS -X POST https://example.com/comment --data-urlencode [email protected]

The fourth case involved adb during a UI test on an Android 13 device, when I tried to enter Thai text in an input field.

$ adb shell input text 'ทดสอบ'
java.lang.NullPointerException: Attempt to get length of null array

The error message did not mention encoding or characters at all. The input text command converts text into a sequence of keypresses through KeyCharacterMap.getEvents, which returns null for non-Latin characters. The following code reads the length of the null array to iterate over it, throwing an exception instead of saying it cannot type those characters. My workaround was to have the test enter an ASCII placeholder, and if I needed to check how Thai text actually rendered, I typed it on the device.

The Python case: Setting encoding in the script is too late

The last case was print crashing with a UnicodeEncodeError referring to the cp874 codec. I tried fixing it by adding os.environ['PYTHONIOENCODING'] = 'utf-8' as the first line of the script. It did not help because the interpreter had already opened stdout with its encoding before running my first line. The environment variable must be set when python is invoked.

# ตั้งตอนเรียก ไม่ใช่ตั้งในสคริปต์
PYTHONUTF8=1 PYTHONIOENCODING=utf-8 python report.py

# ทางที่ไม่ขึ้นกับ console เลย: เขียนผลลงไฟล์ UTF-8 แล้วเปิดอ่าน
PYTHONUTF8=1 python report.py > out.txt

# ในสคริปต์ ให้ระบุ encoding ทุกครั้งที่เปิดไฟล์ อย่าพึ่งค่าเริ่มต้นของเครื่อง
with open('out.txt', 'w', encoding='utf-8') as f:
    f.write(text)

When invoking it from PowerShell, set $env:PYTHONUTF8 = '1' first. I mainly choose to write to a file because the result is the same on every machine, regardless of which console is open.

Summary table: What each reader sees

The rules I use now

All these cases follow the same pattern. Thai text passes through an intermediary that interprets the bytes using its own mapping, and the error appears in the next layer, which knows nothing about encoding. The error message therefore talks about braces, ports, or null arrays instead.

I now follow three rules. First, .ps1 files containing Thai must have a BOM and pass a parse check with powershell.exe. Second, .cmd and .bat files must be entirely ASCII, without exceptions. Third, Thai text sent to other programs must go through UTF-8 files, not the command line. Anyone who still wants to put Thai text in batch files is welcome to, but I have stopped, because the time lost chasing misleading errors far outweighs the time saved reading comments in Thai. If I have left anything out or made a typo, I will come back and fix it later.