fix(bundler): degrade non-UTF-8 config reads into BundlerError - #3784
Open
jawwad-ali wants to merge 1 commit into
Open
fix(bundler): degrade non-UTF-8 config reads into BundlerError#3784jawwad-ali wants to merge 1 commit into
jawwad-ali wants to merge 1 commit into
Conversation
yamlio.py is the single chokepoint for every bundler read, and its module
docstring states the contract: "All reads/writes go through these functions so
that IO failures degrade into actionable BundlerError rather than raw
tracebacks."
Both readers catch only OSError, but `Path.read_text(encoding="utf-8")` and
`json.load()` raise UnicodeDecodeError on a non-UTF-8 file --
`issubclass(UnicodeDecodeError, OSError)` is False (its MRO is UnicodeError ->
ValueError). So the decode error escaped uncaught:
load_yaml: LEAKED UnicodeDecodeError -> 'utf-8' codec can't decode byte 0xff
load_json: LEAKED UnicodeDecodeError -> 'utf-8' codec can't decode byte 0xff
In load_json, json.JSONDecodeError does not help: it is a *sibling* of
UnicodeDecodeError, not a parent.
This is realistic rather than theoretical -- on Windows, PowerShell 5.1's
`Out-File` and `>` default to UTF-16, so a hand-edited
`.specify/bundle-catalogs.yml` or records file hits it.
Widen both read clauses to `(OSError, UnicodeError)`, matching the sibling
catalog readers (catalogs.py:101, workflows/catalog.py:336). JSONDecodeError
deliberately stays FIRST so malformed-but-decodable JSON keeps its more
specific "Invalid JSON" message; a regression test locks that ordering.
Write paths are unaffected -- verified that dump_yaml/dump_json do not leak
UnicodeEncodeError (both escape unencodable input), so this stays scoped to the
two read paths.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jawwad-ali
force-pushed
the
fix/yamlio-unicode-decode-guard
branch
from
July 28, 2026 14:56
8e22774 to
0c6f70b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
bundler/lib/yamlio.pyis the single chokepoint for every bundler read, and its module docstring states the contract explicitly:Both readers catch only
OSError. ButPath.read_text(encoding="utf-8")andjson.load()raiseUnicodeDecodeErroron a non-UTF-8 file, and:So the decode error escapes uncaught. Reproduced on
mainwith a UTF-16 file:In
load_json,json.JSONDecodeErrordoes not cover it — the two are siblingValueErrorsubclasses, neither subsumes the other.This is realistic rather than theoretical: on Windows, PowerShell 5.1's
Out-Fileand>default to UTF-16, so a hand-edited.specify/bundle-catalogs.ymlor records file lands here.Fix
Widen both read clauses to
(OSError, UnicodeError)— matching the sibling catalog readers (catalogs.py:101,workflows/catalog.py:336, which already use exactly this tuple).json.JSONDecodeErrordeliberately stays first inload_json, so malformed-but-decodable JSON keeps its more specificInvalid JSONmessage while a decode failure falls through to the read-error clause. A regression test locks that ordering.Scope: reads only. I verified the write paths do not leak
UnicodeEncodeError—dump_yaml/dump_jsonboth escape unencodable input (tested with a lone surrogate), so there is nothing to fix there.Verification
test_load_yaml_non_utf8_raises_bundler_errorandtest_load_json_non_utf8_raises_bundler_error(UTF-16 input): fail before with the rawUnicodeDecodeError, pass after.test_load_json_malformed_still_reports_invalid_json: clause-order guard — decodable-but-malformed JSON still reportsInvalid JSON.tests/unit/test_bundler_yamlio.py: 5 passed.tests/unit/+tests/contract/: 252 passed (2 pre-existing Windows symlink-privilege failures, unrelated — they need admin and pass on CI's Linux/macOS runners).uvx ruff@0.15.0 check src testsclean.No behaviour change for UTF-8 input: those reads take the identical path and return identical values.
AI-assisted: authored with Claude Code. I reproduced both leaks on
main, checkedissubclassrather than assuming, verified the write paths were safe before scoping the fix, and matched the existing sibling-reader tuple.