Skip to content

Repository hashes depend on the checkout path when a repository is built from a workspace directory #554

Closed

Description

@Spikhalskiy

Some repository rules build a repository from a directory inside the workspace and take that directory as an absolute path. Examples are rules_rs local crates and Gazelle's go_repository(local_path = ...). bazel-diff hashes the path, so the same commit hashes differently in every checkout. Diffing hashes generated at two paths reports every dependent of that repository as impacted. So does any serve cache shared between checkouts, whose key doesn't include the path. #544 was the same problem for file://%workspace% registry URLs. #545 fixed that case by leaving integrity-pinned URLs out of the hash, but a path attribute has no integrity.

We hit this with a few vendored Rust crates and two in-repo Go modules. Several checkouts share one hash cache, and every diff across checkouts selected a few thousand targets. Fine-graining those repositories doesn't help, because the module that owns the extension still depends on the path.

Mechanism

repository_target (src/bazel.rs) copies the repository's attributes verbatim into the //external:<repo> rule it hashes (repository_definition_hash does the same for a module's own definition). Bazel prints the attribute as an absolute path:

path = "/<checkout>/vendored",

So //external:vendored changes with the checkout path. //external:ext has an edge to each repository its extension generates, so it changes too, and so does every target that uses either one. With --fineGrainedHashExternalRepos=@vendored, targets that use @vendored stop depending on its //external node, but targets that use anything else from ext still change.

Reproduction

https://gist.github.com/Spikhalskiy/0f2888ae49ca73153662bce83158b61f

The script writes a workspace with a module ext (a local_path_override) that provides @ext//:tool and an extension that builds @vendored from the workspace's vendored/ directory. //:consumer uses @vendored and //:tool_user uses @ext//:tool. The script copies the workspace to a second path and runs generate-hashes in both and get-impacted-targets between them, without and with fine-graining @vendored. With bazel-diff 65.0.0 and Bazel 9.3.0, it prints:

Identical checkouts at two paths (expected: none):
//:consumer
//:tool_user
Identical checkouts at two paths, --fineGrainedHashExternalRepos=@vendored (expected: none):
//:tool_user

Suggested fix

Before hashing repository attributes, replace the workspace root with a placeholder such as %workspace%. Check both the path as given and its canonical form, since Bazel prints /private/var/folders/... on macOS for a workspace given as /var/folders/.... Paths outside the workspace keep hashing as they are, and an edit inside the directory is still detected when the repository is fine-grained, because its files are hashed by content. With that, both runs above print nothing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions