Skip to content

A lockfile bump of one extension repository impacts every target of every module that depends on the extension's owner #603

Closed

Description

@macourteau

We run bazel-diff 70.0.1 (the Rust build) in query mode in CI, on a Bzlmod workspace with Bazel 9.3.1. Our Go dependencies come from gazelle's go_deps extension (gazelle 0.54.0, rules_go 0.65.0) and our JavaScript ones from the npm extension of aspect_rules_js. Every dependency bump from Renovate ends up rebuilding almost everything, and I traced it to the edges bazel-diff adds from a module to the repositories its extensions generate.

What we see

One Renovate change touched only go.mod and go.sum: a patch bump of go.yaml.in/yaml/v3. bazel-diff marked 4,485 targets as impacted, about 1,330 of them rules out of roughly 8,700 in the main repository. That included every copy_file, ts_project, js_binary, jest_test, proto_library and oci_* target, none of which depends on the Go module. The Go targets themselves were split correctly (12 go_library impacted, 31 not), so the extra targets come from somewhere else.

Running bazel-diff explain on one of them, a copy_file generated by rules_jest's jest_test macro, gives a single root cause:

//pkg:_tests_bazel_sequencer
IMPACTED (indirectly -- the change came from its dependencies)

Root causes: 1

  1. //external:in_yaml_go_yaml_v3
     the target's own hash changed
     7 hops (1 package boundary crossed)
     //external:in_yaml_go_yaml_v3 -> //external:bazel_gazelle -> //external:rules_android+
       -> //external:rules_jvm_external+ -> //external:com_google_protobuf
       -> //external:aspect_rules_js -> //external:aspect_rules_jest -> //pkg:_tests_bazel_sequencer

Why it happens

The first hop is the edge lower_repositories adds from a module to every repository its extensions generate. gazelle owns go_deps, so //external:bazel_gazelle depends on the Go repository that changed. Every hop after that is an ordinary module dependency: protobuf reaches gazelle through rules_android, rules_js depends on protobuf, and rules_jest depends on rules_js. So every module with a bazel_dep path to gazelle changes whenever any Go dependency does, and in a typical Bzlmod workspace that is most of the module graph.

The npm extension does the same

A pnpm-lock.yaml change bumping a single package (user-agents from 2.1.228 to 2.1.229) marked 4,288 targets. bazel query 'rdeps(//..., //:node_modules/user-agents)' finds 66. explain on the same copy_file reports four root causes, all reaching it through //external:aspect_rules_js:

Root causes: 4

  1. //external:aspect_rules_js
     the target's own hash changed
     //external:aspect_rules_js -> //external:aspect_rules_jest -> //pkg:_tests_bazel_sequencer

  2. //external:aspect_rules_js++npm+npm__user-agents__2.1.229
     new target in the final revision
     //external:aspect_rules_js++npm+npm__user-agents__2.1.229 -> //external:aspect_rules_js
       -> //external:aspect_rules_jest -> //pkg:_tests_bazel_sequencer

  3. //external:aspect_rules_js++npm+npm__user-agents__2.1.229__links
     new target in the final revision
     (same path as 2)

  4. //external:npm
     the target's own hash changed
     //external:npm -> //external:aspect_rules_js -> //external:aspect_rules_jest -> //pkg:_tests_bazel_sequencer

The names of npm repositories carry the package version, so every bump adds and removes repositories, and //external:aspect_rules_js changes along with them. From there every module that depends on rules_js (rules_jest, rules_ts and so on) carries the change to each target that uses one of its rules.

What I tried

I first tried --labelOnlyModuleExtensions=@bazel_gazelle//:extensions.bzl%go_deps from #558 and #559. bazel-diff accepts it without a warning, but the impacted set and the explain path stay the same. As far as I can tell from #559, the view of the owning module without the extension's repositories is only used by targets inside those repositories, while a module that depends on the owner (rules_android+ above) still reaches its full node.

--isolatedDependencyRoot doesn't fit either, since it needs one root import covering every generated repository, and go_deps exposes each Go module through its own use_repo entry.

--useCquery does solve it. The Go bump impacts only its real dependents, and the npm bump impacts exactly the 66 rule targets above plus their generated files. The cost is that hashing takes 4 to 7 times as long on the same runners (62 to 72 seconds for the base revision instead of 13 to 18, and 10 to 15 seconds for the current one instead of 1 to 2), and the result follows a single host platform. That's what we're using for now, but query mode would be our preference.

What we would like

When an extension is listed in --labelOnlyModuleExtensions, a module that depends on the extension's owner could reach the owner through the view #559 introduced, the one without the listed extension's repositories. A Go or npm bump would then only impact the targets that depend on the bumped repository by label, instead of every target of every module above the owner in the module graph.

I think the flag's existing assertion, that the repositories use each other only through labels and never by resolving a toolchain from one another, already covers this, because neither go_repository nor npm_import repositories register toolchains, so a dependent module can't reach them through toolchain resolution either. If that stretches the flag too far, a separate opt-in would work just as well for us.

Reproducing

  1. Take a Bzlmod workspace with bazel_deps on gazelle, rules_go, protobuf and aspect_rules_js, with go_deps.from_file(go_mod = "//:go.mod"), one go_library that depends on a go_deps repository, and any target that uses a rules_js or bazel_lib rule, such as a copy_file.
  2. Run bazel-diff generate-hashes -d edges.json at a base revision, bump the Go module's patch version in go.mod and go.sum, and hash again.
  3. get-impacted-targets lists the copy_file, and explain attributes it to the Go repository through //external:<gazelle's apparent name> and the module dependencies, as above.

I haven't reduced this to a standalone repository yet, but I can if that would help.

Activity

  1. tinder-maxwellelliott commented on Nov 2, 2026

    @tinder-maxwellelliott
    Collaborator

    Does this PR capture this issue fully? #608

  2. macourteau commented on Nov 3, 2026

    @macourteau
    Author

    Thanks for putting the reproducer together so quickly! It captures the Go half of what I'm seeing, a module that only depends on the extension's owner (like rules_android depending on gazelle), but I don't think it covers the npm half, and that part may need more than the fix proposed there.

    With npm, the owner (rules_js) is also the module whose rules our targets use directly. Here's explain for a js_binary that has nothing to do with the bumped package:

    //foo:bar
    IMPACTED (indirectly -- the change came from its dependencies)
    
    Root causes: 4
    
      1. //external:aspect_rules_js
         the target's own hash changed
         //external:aspect_rules_js -> //foo:bar
    
      2. //external:aspect_rules_js++npm+npm__user-agents__2.1.229
         new target in the final revision
         //external:aspect_rules_js++npm+npm__user-agents__2.1.229 -> //external:aspect_rules_js -> //foo:bar
    
      3. //external:aspect_rules_js++npm+npm__user-agents__2.1.229__links
         new target in the final revision
         (same path as 2)
    
      4. //external:npm
         the target's own hash changed
         //external:npm -> //external:aspect_rules_js -> //foo:bar
    

    There's no dependent module on that path, so routing dependent modules through the view wouldn't change it. In the fixture, ext's crate rule is only used inside the spokes, so the root module has no target like this one.

    A few smaller differences, in case they matter for the fix:

    • npm repository names carry the package version, so a bump removes npm__user-agents__2.1.228 and adds npm__user-agents__2.1.229 rather than changing a spoke in place. That's why //external:aspect_rules_js itself shows up as changed: its inputs changed. In the fixture, spoke b keeps its name.
    • The test passes --fineGrainedHashExternalRepos for the hub and spokes. We don't fine-grain any repository (the numbers in the report are without it), so it would be good to know the fix holds in that configuration too.
    • Our Go path crosses several modules (rules_jest to rules_js to protobuf to rules_android to gazelle). One hop is probably enough to pin it if every edge to the owner goes through the view, but I wanted to mention it.

    So to cover both cases, I'd suggest adding to the fixture a root target that uses a rule from ext directly, a spoke whose name changes with its version, and a run without --fineGrainedHashExternalRepos.

  3. tinder-maxwellelliott commented on Nov 3, 2026

    @tinder-maxwellelliott
    Collaborator

    Thanks for the detailed write-up. A minimal reproducer is up in #608.

    Fixture (tests/resources/workspaces/extension_owner_dependent_module/):

    • Module ext owns an extension that generates a hub @crates and one spoke repository per entry of versions.json. This stands in for gazelle's go_deps or rules_js's npm.
    • Module dependent only has a bazel_dep on ext, like rules_js → protobuf → rules_android → gazelle.
    • Root targets: //:uses_a and //:uses_b use the spokes; //:uses_dependent uses a rule of dependent and names nothing in @crates (your copy_file).

    Test (external::label_only_extension_bump_does_not_reach_modules_depending_on_the_owner) bumps spoke b and checks:

    1. Without --labelOnlyModuleExtensions, //:uses_dependent is impacted. This is today's conservative behavior.
    2. With --labelOnlyModuleExtensions=@ext//:crates.bzl%crates, only //:uses_b should be impacted. This fails today, matching what you saw with go_deps:
      left:  {"//:uses_b", "//:uses_dependent"}
      right: {"//:uses_b"}
      
    3. A change to ext's own files must still impact //:uses_dependent, so the fix can't just drop the dependent → ext edge.

    Your diagnosis is right: in lower_repositories, //external:ext depends on every repository its extensions generate, and the label-only view from #559 is only used by targets inside those repositories. //external:dependent still depends on the full owner node. The likely fix is the one you proposed: when an extension is listed in --labelOnlyModuleExtensions, route a dependent module's edge to the owner through that same view.

    No standalone repository needed on your side; the fixture reproduces it on Bazel 8.6.0+.

  4. macourteau commented on Nov 3, 2026

    @macourteau
    Author

    Thanks! That matches the Go case. The npm case from my previous comment isn't in the fixture yet though: a root target that uses a rule from the owner itself (like a js_binary from rules_js, which owns npm) reaches the full owner node directly, with no dependent module in between, so routing dependent modules through the view wouldn't change it. Would you like that tracked in #608 (say a //:uses_ext target in the fixture) or as a separate issue? And is the fix expected to hold without --fineGrainedHashExternalRepos, which we don't use?

  5. added a commit that references this issue on Nov 3, 2026
  6. tinder-maxwellelliott commented on Nov 3, 2026

    @tinder-maxwellelliott
    Collaborator

    Thanks, you're right on all counts. #608 is extended to cover both halves:

    • npm half: //:uses_ext uses the owner's own rule directly (your js_binary → rules_js), with no dependent module on the path.
    • Versioned spokes: spokes are named after their version (crates__b__1_0 → crates__b__2_0), so a bump removes one repository and adds another, like npm__user-agents__2.1.228 → 2.1.229.
    • Multi-hop Go half: //:uses_dependent's module reaches the owner through two bazel_dep hops.
    • No fine-graining: the main check runs with only --labelOnlyModuleExtensions, your configuration. A second run fine-grains just the @crates hub. (A versioned spoke can't be listed in --fineGrainedHashExternalRepos, since the flag applies to both revisions and Bazel rejects a repository that doesn't exist at one of them.)

    Results today with the extension listed:

    Configuration Expected Today
    label-only, no fine-graining {uses_a, uses_b} {uses_a, uses_b, uses_dependent, uses_ext}
    label-only + @crates fine-grained {uses_b} {uses_b, uses_dependent, uses_ext}

    A change to the owner's own files still has to reach both uses_ext and uses_dependent, and it does today.

    So, as you said, routing only dependent modules through the #559 view won't be enough. For a listed extension, every edge to the owner from outside its generated repositories has to go through that view, including a main-repo target's direct label on the owner's rule.

  7. added a commit that references this issue on Nov 3, 2026
    bd2033b
  8. macourteau commented on Nov 3, 2026

    @macourteau
    Author

    Thanks! The extended fixture matches both cases we're seeing. Looking forward to the fix.

    EDIT: tested with #608 patched in and confirmed to work on our workspace: a Go bump now impacts only its 8 real dependents (4,485 before) and an npm bump only its real dependents (552 targets, 4,288 before).

  9. added a commit that references this issue on Nov 4, 2026
    0ae54f5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions