[ac-pplx]

What

scripts/split_rust_module.py splits a rustfmt-formatted Rust file into child modules from a short spec, and proves afterwards that the result is a pure move.

• list FILE prints each top-level item, and each member of an inherent impl, with the selector a spec uses.
• apply SPEC moves the selected items into new child modules. Each item keeps its doc comments and attributes, its original order, and its text. The parent keeps mod child; and a glob re-export, so every existing path still resolves and no caller changes. Each child starts with use super::*;, so the parent's imports, traits and macros resolve as before.
• verify [BASE] proves that BASE...HEAD's Rust changes are exactly what apply generates. It derives the spec from the diff (which items each new child module holds), regenerates the split from the merge base, rustfmts the result, and requires HEAD's parent and child files to equal it byte for byte. Any other Rust change fails it: a missing or altered mod/use line, a dropped or reordered item, an edit inside moved code or a comment, a symlink, or a diff with no split. So does a non-Rust change inside the split crate (a manifest, data or build input) unless --allow-non-rust is given; non-Rust changes elsewhere are listed. --print-spec shows the derived spec so it can be compared with the one in the PR.

A spec names items, not lines:

file = "crates/libs/x/src/foo.rs"
[[module]]
name = "pull"
doc = "Pull jobs: destination locks, cancellation cleanup and the pull itself."
items = ["struct DestinationPullGuard", "impl Drop for DestinationPullGuard",
         "impl TransferExecutor :: fn execute_pull*"]

impl <header> :: fn <name> moves chosen members of an inherent impl into a block with the same attributes and header in the child; the parent's block keeps the rest. Trait impls move whole.

Why

Large files are slow to review and change, for people and for agents. Splitting them by hand takes long and is hard to review. A split also goes stale against main within hours, and rebasing a 1 → N move is mostly manual, because git's rename detection doesn't follow it.

With this tool a split is a spec plus a regenerate. If main moves, rerun apply on the new main instead of resolving conflicts. Reviewers check the grouping in the spec and rerun verify, instead of reading thousands of moved lines.

verify checks the generator's output, not its semantics. Those rest on the edits below being reach-preserving, on the refusals, and on compiling and testing the result like any change.

The only edits it makes to moved text

Each keeps name resolution and reach exactly as they were:

Edit	Why
private / pub(self) → pub(super) on items, inherent-impl members and struct fields	One module deeper, pub(super) reaches exactly the old module and its descendants.
pub(super) → pub(in crate::<grandparent>) (or pub(crate))	Same reach as before.
super:: → super::super::	The code is one module deeper.
relative include!/include_str!/include_bytes! paths gain ../	Only when the children are one directory below the parent file (foo.rs → foo/).

It refuses, rather than guesses, when a move needs a hand edit. Most refusals are about macros, which the tool cannot expand:

• moving use, mod x;, extern blocks or macro_rules! items, and any split of a parent with a #[macro_use] item. Macros stay in the parent, above the new mod lines, so children still see them.
• moved code reaching a crate macro whose expansion names super/self paths or emits pub(super)/pub(self). This covers direct calls, calls through other macros, path-qualified calls and use .. as aliases.
• a macro invoked before the parent defines it, or defined twice;
• super passed in a user macro's arguments, bare or path-qualified. format!/assert!/tracing-style macros are fine, unless the crate defines its own macro of the same name;
• stringify! of super, and include!;
• a doc comment or #[doc] attribute in moved code naming a super::/self:: path: an intra-doc link would point one module off;
• top-level or impl-member macro invocations, unless the module sets allow_macro_items;
• relative visibility the edits cannot remap: pub(in super::..), a pub(super) nested inside a moved item, an inline module naming super, self::super, or a super:: path inside a string literal;
• include arguments other than a plain string literal;
• a parent file loaded through #[path], or #[path] inside moved code;
• a module name that is a keyword, collides with an item or import of the parent, or is already used as a path in the parent or its child modules;
• an attribute on the same line as its item or impl member;
• a carriage return, a src/bin/<name>.rs crate root, or a code block in a module doc.

Module options, each reproduced by verify:

• allow_unused_glob keeps use super::*; under #[allow(unused_imports)] for a module that needs nothing from its parent.
• allow_unused_reexport does the same for the parent's re-export when nothing outside the module names its items.
• allow_macro_items opts in to moving macro invocations.

apply also reports per new module the tracing events without an explicit target: (their target becomes the new module path), sql! locators, and lines matching an archcheck pattern_ratchet, so a PR can list them and move ratchet entries with the code.

Evidence

• scripts/tests/test_split_rust_module.py (108 tests):
  • the visibility table, file layouts and re-export reach;
  • every refusal above, and every change verify must reject;
  • a compile oracle: rustc --test -D warnings builds and runs a small crate before and after a split, with callers in the parent, a sibling and the grandparent reaching private, pub(super) and field-level items.
• Workspace compile run: all items of 19 large production files in 15 crates moved at once, 1,838 items in total.
  • cargo check --workspace --all-targets --all-features and CI's clippy invocations with -D warnings passed.
  • The --no-default-features builds of the 14 packages, doctests, and the nextest runs of four packages all matched main.
• Round trip: on the 102 largest production files, every item and impl member was moved, rustfmt'd and checked with verify. 92 regenerate identically; the other 10 stop at a refusal (macro invocations without the opt-in, include!, a parent macro emitting pub(super), nested relative visibility, an inline module naming super).
• Eight split PRs (node-ctl: split facets.rs into facet and local-state modules (pure move) #563–api: move six groups out of controllers/substrate/mod.rs (pure move) #569, spaced-host-client: move the client's methods into modules by guest area (pure move) #571) use it. Each regenerates byte-identically with this version.
• Review: four rounds by Codex and Claude. Rounds 1–3 found false accepts in an earlier verify design, which led to the regeneration check, and apply cases that changed meaning; each is fixed with a test. Round 4 found only macro forms the tool now refuses. A maintainer review then found seven non-blocking gaps; each is fixed with a test, and the eight split PRs still regenerate identically.

Limits

• It expects rustfmt layout: top-level items in column 0.
• Splitting a single large function is not a move, so it is out of scope.
• verify proves the commit is apply's rustfmt'd output for the derived spec. That apply preserves meaning rests on the edit rules, the refusals, and compiling and testing the result. Macros are where that argument is weakest, hence the conservative refusals.
• Macros defined outside the crate being split are not analyzed: other workspace crates, dependencies, proc and attribute macros. One that emits pub(super) or a super:: path would expand one module deeper without a refusal; compiling and testing the result is the check.
• What a module moved with allow_macro_items emits keeps the visibility the macro gives it; compile errors are the check.
• Log targets and source locations of moved code change. That is inherent to moving code; docs/rust/space-conventions.md ("Log targets") covers it, and apply counts them.

🤖 Generated with Claude Code

[the-gigi-pplx]

Review: 7 findings

Medium

1. scripts/split_rust_module.py:636: the refusal for super passed in a user macro's arguments can be bypassed two ways. The module docstring says this case is refused.

   • Path-qualified calls skip it. The pattern is (?<![\w:])(\w+)\s*!, so crate::m!(..), $crate::m!(..) and other_crate::m!(..) are never checked. Reproduced on a scratch crate. With #[macro_export] macro_rules! name_of { ($p:path) => { stringify!($p) } } in lib.rs, a moved crate::name_of!(super::Thing) was rewritten to crate::name_of!(super::super::Thing) and apply exited 0. The bare call name_of!(super::Thing) was refused, as intended. The result still compiles, but the string it produces changes.
   • A crate macro that shares a name with an entry in KNOWN_MACROS (line 557) is exempt by name alone. Reproduced: a local macro_rules! ensure { ($c:expr) => { if !$c { panic!("{}", stringify!($c)) } } }. A moved ensure!(super::LIMIT > 0) became ensure!(super::super::LIMIT > 0) with no refusal, which changes the panic text.

   verify can't catch either one, because the output is exactly what apply generates. To fix it, check the last path segment of a qualified call too, and apply the KNOWN_MACROS exemption only when the crate's own macro_rules! bodies (all_bodies) don't define that name. None of the split PRs that depend on this one (PR node-ctl: split facets.rs into facet and local-state modules (pure move) #563 to PR spaced-host-client: move the client's methods into modules by guest area (pure move) #571) rewrites a single super:: path, so none of them is affected.

Low

2. scripts/split_rust_module.py:238: a file that opens with a plain // comment (a license or SPDX line) and then //! docs isn't parsed as having a header. Both comments attach to the first item and move with it. Reproduced: splitting out fn a put // Copyright header. and //! Module docs. into p/bar.rs after use super::*;, which is E0753, and left the parent with no module docs. The compiler catches it, so it's loud rather than silent, but the header should stay with the parent. /*! .. */ inner docs aren't recognized either.
3. scripts/split_rust_module.py:653: super:: is rewritten only in masked code, where comments are blanked, so intra-doc links in moved doc comments keep their old text. Reproduced: a moved /// Returns the crate root's [`super::Thing`] stayed as written while the code beside it became super::super::Thing. One module deeper, the link resolves to the parent's Thing. That's a different item when the parent has one, and a broken link when it doesn't. Refusing (or deepening) super::/self:: in doc-link syntax would close it. None of the dependent splits has such a link; I grepped every new child file.
4. scripts/split_rust_module.py:822: DEFS has no fn, so in a parent made only of functions the anchor falls through to a later mod/use. Reproduced: with fn a, fn b, #[cfg(test)] mod tests;, splitting out fn a writes mod bar; and pub use bar::*; after mod tests; at the bottom of the file. Every regeneration will put them there again.
5. scripts/split_rust_module.py:1108: the comment says a fully moved split impl's docs go to "the first module (in spec order) that receives members". Lines 1114 and 1115 actually pick the module that receives the lowest-indexed member, which isn't the same when the spec lists modules in a different order from the members. Either the comment or the code should change, so the derived placement is predictable.
6. scripts/split_rust_module.py:1024: the relative-macro refusal only scans macro_rules! bodies under the parent's own crate. A macro from another workspace crate or a dependency, or a proc or attribute macro, that emits pub(super) or a super:: path expands one module deeper without a refusal. The docstring scopes the check to "any macro_rules! in the crate", so this matches what it promises. It's still worth listing among the things verify doesn't prove, next to the compile-and-test caveat. I read this one and didn't reproduce it.
7. verify exits 0 when the diff also changes non-Rust files. I appended a line to crates/libs/spaced/host/client/Cargo.toml on top of PR spaced-host-client: move the client's methods into modules by guest area (pure move) #571's head. verify printed not checked by verify (not Rust): …/Cargo.toml and exited 0. A manifest, feature or include_str! data change could then ride along in a "pure move" whose only evidence is the exit code. Exiting non-zero on non-Rust changes under the split crate unless a flag allows them would make the exit code mean what the body says it does.

Verified against dcfd1ae230db3453800e227e522be5b97995bbb8.

What I ran:

• pytest scripts/tests/test_split_rust_module.py: 98 passed.
• verify origin/main on all eight dependent split PRs: identical, exit 0 for each.
• Re-ran apply from each PR body's spec on its merge base: byte-identical to the head every time.
• Twelve hand mutations on PR spaced-host-client: move the client's methods into modules by guest area (pure move) #571's head: verify caught all eleven Rust ones. They were a comment edit in a moved item, a narrowed visibility, use super::{*};, an added item, a trailing comment, pub mod in the parent, an unrelated parent attribute, a new build.rs, a stray child file, a comment inside a body, and port → port: port. The twelfth, the Cargo.toml edit in finding 7, passed.
• Findings 1 to 4 were reproduced on a scratch crate and finding 7 on PR spaced-host-client: move the client's methods into modules by guest area (pure move) #571's head. Findings 5 and 6 come from reading the cited lines.

[the-gigi-pplx]

Approved. 7 non-blocking findings in the review comment above.

[ac-pplx]

Thanks for the careful read. All seven are fixed in a097b25, each with a test that fails on the previous head and passes now (108 tests):

1. super in macro arguments. The scan now matches the last path segment, so crate::m!, $crate::m! and other::m! are checked. The KNOWN_MACROS exemption applies only when no macro_rules! in the crate has that name; apply collects the crate's macro names before the transforms run.
2. Header. The header runs through the last //! line, /*! */ block or #![..] attribute before the first item, including the plain comments and blank lines above it. A license or SPDX line above the module docs stays with the parent.
3. Intra-doc links. apply refuses moved code whose doc comments (///, //!, /** */, /*! */) or #[doc] attributes name a super:: or self:: path. Plain comments and code are not affected; a test covers * super::V on a continuation line.
4. Declaration placement. With no type, const or impl in the parent, the mod lines go above the first fn or inline mod, not after a trailing mod tests;. fn is still not in DEFS, so parents that have definitions place them as before.
5. Lead-owner comment. It now says what the code does: the module that receives the impl's lowest-indexed moved member.
6. Macros from other crates. The docstring and the Limits section of the description now say that macros from other workspace crates, dependencies, and proc or attribute macros are not analyzed, and that compiling and testing is the check there.
7. Non-Rust changes. verify now fails on a non-Rust change inside the split crate (a manifest, data or build input) unless --allow-non-rust is given. Changes elsewhere are still listed.

The split PRs are unaffected: with this version, all eight (#563–#569, #571) still regenerate byte-identically at their current heads, none of their parents starts with a plain comment, and every non-Rust file they touch is outside the split crate.