Repository navigation
Add COMPUTER_USE_ABLATE mechanism ablation switches - #5
Closed
Haoqing Wang (hqhq1025) wants to merge 13 commits into
Closed
Haoqing Wang (hqhq1025) wants to merge 13 commits into
Haoqing Wang (hqhq1025) wants to merge 13 commits into
Conversation
Parse and validate batching, wait, diff and guard; refuse to start on an unknown value, on guard outside Linux, and on guard without COMPUTER_USE_GUARD_LOG. The schema, tool descriptions and server instructions drop the fields and sentences of ablated mechanisms; with no ablation they are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
batching: an AX act takes one input step; wait steps may surround it (rule one_input_step). wait: wait steps are no longer an action type, and error texts no longer suggest them. diff: act runs as before but returns only its execution evidence; its reads are not recorded as delivered, so a later observe compares with what the model actually saw. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With guard ablated, the AX target identity, batch window, point owner and geometry checks and the GUI layout and zoom checks still decide, but only log the input they would stop to COMPUTER_USE_GUARD_LOG and let it through. A path with no live node still fails as stale. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--ablate (config key ablate) selects the removed mechanisms, passes them to the MCP server env of all four harnesses, gives the agent guides without the passages of the removed mechanisms, and records the set in result.json and meta.json. With guard ablated the runner collects the guest's guard log. configs/ablation holds the six OSWorld 1.0 groups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The semantic refusal fallback joins the other resolution code in element_resolver.py, and the GUI layout log line is built in guard_shadow.py, so the dispatch modules stay near the size limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Go: parsing and startup refusal, model-visible text of every ablation set, valid description examples, schema and decoder agreement under each ablation, the one-input-step rule, wait removal, act results without feedback, and the guard log. Provider: shadow mode logs and lets input through for target identity, batch window, point owner and GUI layout, and changes nothing without the ablation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
check_ablation_text.py starts the server for each COMPUTER_USE_ABLATE subset, reads instructions, tools and rejected calls, generates the guides, and fails on words or act examples of a removed mechanism. check_all.sh runs it with the candidate binary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deleting the passages of a removed mechanism left the agent to learn the changed rules from rejected calls. The guide now adds one sentence per ablation (batching, wait, diff), matching the AX tool description, and no new advice; guard adds none. Everything else in the guide is still only removed text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…left guard: the focus check of a focus-bound batch, the select range end and the combobox edit partner now go through the shadow, and a shadowed target keeps the window that holds it. The full arm no longer reads the tree for a shadow it does not use. wait: a rejected wait step points to a fresh observe instead of an input action guessed from its fields, and the no-tree window text no longer offers a wait. batching: input after close is refused as close-not-last without suggesting a wait, and no_focused_window no longer says to start a batch with activate. diff: act errors no longer carry a new wN, the values and windows a wait saw, or a promised observation; the tree reference no longer mentions diffs. A wait-only act without ref or window_ref no longer panics the server when the window list cannot be read. This crash predates the switches and hit every arm that keeps wait. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
batching removes "start the batch with activate"; diff removes the wait_timeout advice to read the returned state. check_ablation_text.py looks for both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The preflight probe sends one call per removed mechanism and requires each to be refused, and requires the server to refuse an unknown COMPUTER_USE_ABLATE value, so a build without the switches fails preparation instead of running full HCU under an ablation label. meta.json records the MCP binary's SHA-256. A task now fails before its agent starts when the gateway writes another token log than the run reads, which happened when ablation groups shared one gateway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Removing diff now answers what the diff contributes: an act still returns the state after its actions, as the whole current tree of its window, in place of the effect and windows report, the diff and the notes that compare with an earlier read. observe always repeats the tree instead of "Tree unchanged". The tree is read with the same post-action stabilization as a diff, so the ablation changes what is shown, not when it is read. The earlier variant returned only the execution evidence, which also removed every post-act observation; the error-path trimming it needed is reverted. effect_unknown stays in the evidence and a settle wait keeps its message: the first is execution evidence, the second belongs to wait. The guide and tool description say that an act result shows the whole tree instead of a report of what changed, and keep the advice to read the returned state. Without an ablation every model-visible byte is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Remove the batch-window sentence when batching is ablated, assert that the diff-ablation error keeps its cause, and check that read_text survives every ablation set.
Collaborator
Author
Haoqing Wang (hqhq1025)
deleted the
feature/hcu-ablation-switches-20261116
branch
November 16, 2026 18:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request is stacked on #4 and targets that branch; I will retarget it to
mainafter the fixes merge.It adds
COMPUTER_USE_ABLATE, a comma-separated switch that turns off one or more HCU mechanisms so that their effect can be measured. The default (unset) behavior is unchanged.batching: anactcall accepts one input step.wait:waitsteps are rejected and the model is sent to a fresh read.diff: the change report is replaced by the whole tree.guard: the input guards run in shadow mode. They log what they would block and let the input through. This needsCOMPUTER_USE_GUARD_LOG.An ablated server also stops describing what it removed. The server instructions, tool definitions, error results and the generated guides are checked for the words of each removed mechanism, and the server refuses to start on a misspelled switch.
The OSWorld runner takes
--ablate, records the setting in its results, probes that the guest server applies it, and exports the guard log.configs/ablationholds the six OSWorld 1.0 groups.Tests
make testas in the fix branch (the same sixtest_setup_canaryenvironment errors).scripts/maint/check_ablation_text.pypasses for every subset of the four switches and for the invalid values.