feat: add optional unique mock roots by ddstreet · Pull Request #400 · microsoft/azure-linux-dev-tools

Dan Streetman (ddstreet) · 2026-10-22T01:12:19Z

Add the global --unique-mockroot flag and pass mock a stable --uniqueext for each runner. Limit extensions to 24 characters with component labels of up to 12 characters and a six-character random suffix. Use bld, rndr, qry, and shl for build, render, query, and shell actions. Preserve the extension across runner clones and cleanup without changing default root naming.

Propagate component labels through builds, rendering, queries, custom source preparation, and advanced mock commands. Add lifecycle and label boundary tests, update user and agent guidance, and refresh generated CLI docs and help snapshots.

Quality checks and affected unit packages pass. Help snapshots were refreshed from verified CLI output.

Copilot

🔵 Needs a closer look

Root initialization, reuse, and cleanup across workflows need human validation with real mock beyond the command-capture tests.

No open issues 🎉

What changed in this PR

Adds opt-in mock-root isolation to azldev while preserving default root naming. Each runner receives a bounded, labeled extension that remains stable across commands, clones, and cleanup.

Changes:

Adds global --unique-mockroot support with action/component labels and a six-character random suffix.
Propagates labels through build, render, query, source-preparation, and advanced mock workflows.
Adds lifecycle and boundary tests, updates user and agent guidance, and refreshes CLI documentation and help snapshots.

File	Description
`scenario/__snapshots__/TestSnapshotsContainer_help_stdout_1.snap`	Adds flag to container help snapshot.
`scenario/__snapshots__/TestSnapshotsContainer_--help_stdout_1.snap`	Adds flag to container `--help` snapshot.
`scenario/__snapshots__/TestSnapshotsContainer_--bogus-flag_stderr_1.snap`	Updates container error-help snapshot.
`scenario/__snapshots__/TestSnapshots_help_stdout_1.snap`	Adds flag to help snapshot.
`scenario/__snapshots__/TestSnapshots_--help_with_color_stdout_1.snap`	Updates colored help snapshot.
`scenario/__snapshots__/TestSnapshots_--help_stdout_1.snap`	Adds flag to `--help` snapshot.
`scenario/__snapshots__/TestSnapshots_--bogus-flag_stderr_1.snap`	Updates error-help snapshot.
`internal/rpm/mock/uniqueroot.go`	Generates bounded, sanitized root extensions.
`internal/rpm/mock/uniqueroot_test.go`	Tests labels, lifecycle stability, and defaults.
`internal/rpm/mock/mock.go`	Stores, clones, and passes `--uniqueext`.
`internal/app/azldev/env.go`	Adds root-label context helpers.
`internal/app/azldev/env_test.go`	Tests context propagation and isolation.
`internal/app/azldev/core/workdir/workdir.go`	Labels component build environments.
`internal/app/azldev/core/workdir/workdir_test.go`	Tests component-specific root extensions.
`internal/app/azldev/cmds/component/update.go`	Passes component labels and worker context.
`internal/app/azldev/cmds/component/render.go`	Labels batch rendering and source preparation.
`internal/app/azldev/cmds/component/refreshupstreamcommit.go`	Passes component labels and worker context.
`internal/app/azldev/cmds/component/query.go`	Labels query processor roots.
`internal/app/azldev/cmds/component/preparesources.go`	Labels source-preparation roots.
`internal/app/azldev/cmds/component/diffsources.go`	Labels source-diff roots.
`internal/app/azldev/cmds/component/build.go`	Labels batch and individual builds.
`internal/app/azldev/cmds/advanced/mock.go`	Labels advanced builds and shell sessions.
`internal/app/azldev/cmds/advanced/mock_test.go`	Tests advanced command extension propagation.
`internal/app/azldev/app.go`	Registers and applies the global flag.
`internal/app/azldev/app_test.go`	Tests flag wiring and default behavior.
`internal/app/azldev/agentskill/content/mock.md.tmpl`	Explains unique shell roots and re-entry.
`internal/app/azldev/agentskill/content/build-component.md.tmpl`	Documents isolated builds and inspection.
`internal/app/azldev/agentskill/agentskill_test.go`	Checks emitted unique-root guidance.
`docs/user/reference/cli/azldev.md`	Documents the global flag.
`docs/user/reference/cli/azldev_version.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_repo.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_repo_query.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_repo_compare.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_project.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_project_new.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_project_init.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_package.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_package_list.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image_test.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image_list.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image_inject-files.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image_customize.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image_build.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_image_boot.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_docs.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_docs_markdown.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_docs_agent.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_docs_agent_show.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_docs_agent_install.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_config.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_config_generate-schema.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_config_dump.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_update.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_test.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_render.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_query.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_prepare-sources.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_metadata.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_list.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_history.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_diff-sources.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_changed.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_build.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_component_add.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_completion.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_completion_zsh.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_completion_powershell.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_completion_fish.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_completion_bash.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced_wget.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced_mock.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced_mock_shell.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced_mock_build-rpms.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced_mcp.md`	Lists inherited flag.
`docs/user/reference/cli/azldev_advanced_download-sources.md`	Lists inherited flag.
`docs/user/how-to/build-component.md`	Explains isolation, labels, and preserved-root inspection.

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Package-free unique mock shells fail because their chroots are not initialized, and the stale MCP tool-schema snapshot causes TestMCPServerMode to fail.

2 open issues

_{Review effort: Balanced}

Dan Streetman (ddstreet) · 2026-11-10T18:53:43Z

+	app.cmd.PersistentFlags().BoolVar(&app.uniqueMockRoot, "unique-mockroot", false,
+		"use unique mock roots labeled with the action, components, and a random suffix (at most 24 characters)")


Copilot can you check the latest commit to see if this is addressed?

Antonio Salinas (Tonisal-byte) · 2026-10-28T23:05:25Z

+
 	args = append(args, "--configdir", filepath.Dir(r.mockConfigPath))
+	if r.uniqueExtension != "" {
+		args = append(args, "--uniqueext", r.uniqueExtension)


Is leaving the unique bootstrap roots behind intentional? With bootstrap enabled, --uniqueext gives the bootstrap root a unique name too, but the normal cleanup path still uses --scrub chroot, which only removes the main build root. Previously the retained bootstrap could be reused; with a fresh extension for each runner, wouldn’t these accumulate across components and subsequent commands? Should we also clean up the unique bootstrap root while preserving shared caches, or is there another cleanup mechanism I’m missing?

wouldn’t these accumulate across components

yes, and mock --scrub-all-chroots doesn't work for any of our mockroots because we aren't doing things correctly and placing our mock config into /etc/mock, so mock doesn't recognize any of our mockroots.

i'll get copilot to address this while using azldev, but directly calling mock will still be basically broken for anything involving azl4 (without specifying multiple parameters to point to our in-repo mock files)

PTAL to see if copilot correctly fixed your feedback about removing both of the mockroots

Previously the retained bootstrap could be reused

i think the correct way to do this is for us to publish an image and use that, instead of (re)generating a bootstrap mockroot from rpms. that should be part of our work to update our mock configuration upstream.

Add the global --unique-mockroot flag and pass mock a stable --uniqueext for each runner. Limit extensions to 24 characters with component labels of up to 12 characters and a six-character random suffix. Use bld, rndr, qry, and shl for build, render, query, and shell actions. Preserve the extension across runner clones and cleanup without changing default root naming. Propagate component labels through builds, rendering, queries, custom source preparation, and advanced mock commands. Add lifecycle and label boundary tests, update user and agent guidance, and refresh generated CLI docs and help snapshots. Quality checks and affected unit packages pass. Help snapshots were refreshed from verified CLI output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 086236b0-c4cd-4289-ace7-c82d78940883

Copilot

🔵 Needs a closer look

Using --unique-mockroot without --add-package enters shell mode without initializing the new root, preventing the advertised fresh session from opening.

2 open issues

_{Review effort: Balanced}

Address review feedback on unique mock roots: - Scrub the bootstrap root along with the main chroot when a unique extension is in use, since mock gives the bootstrap root a unique name too and nothing would reuse or remove it later. - Initialize the fresh unique root before entering `mock --shell` when no packages are being installed, as mock requires an initialized chroot for shell mode. - Add the unique-mockroot flag to the MCP tools/list snapshot. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6a5e36c4-2d25-42f4-8047-ba75293d4a6c

Copilot

🔵 Needs a closer look

Human assessment is needed because repeated context layering in the shared MCP environment can increase memory use and context-operation costs over the server’s lifetime.

1 open issue

Refresh MCP snapshot to include the unique-mockroot flag

1 resolved since last review

Initialize unique mockroot before opening a package-free shell

_{Review effort: Balanced}

Regenerate TestMCPServerMode_1.snap.json so the unique-mockroot property is in the order emitted by the MCP tool schema. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 077c559f-4c80-4bff-86f8-1841925e81b3

Copilot

🔵 Needs a closer look

Human assessment is needed because an unpublished critical finding indicates unique-root cleanup can leave roots behind and turn otherwise successful builds into failures.

1 open issue

Refresh MCP snapshot to include the unique-mockroot flag

_{Review effort: Balanced}

When --unique-mockroot is set, 'component build' now builds multiple components in parallel, each with its own mock processor and mock root, bounded by the CPU-bound concurrency limit. Builds stay sequential for a single component or when --local-repo-with-publish is used, since publishing requires ordered builds and mutates shared repo metadata. Add env.UniqueMockRootEnabled() and mock.UniqueRootsEnabled(), update the flag help text, and regenerate CLI docs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 36bd7f34-6b81-42da-b6a1-af2040d23322

Dan Streetman (ddstreet) · 2026-11-10T18:52:56Z

added commit to parallellize actions if --unique-mockroot is specified

Copilot

🟡 Changes recommended

Unresolved issues include races in the shared event listener, cancellation paths that can skip cleanup or report incomplete builds as successful, and help/MCP snapshots that mismatch the final flag description.

5 open issues

_{Review effort: Balanced}

+		"use unique mock roots labeled with the action, components, and a random suffix (at most 24 characters); "+
+			"allows 'component build' to build multiple components concurrently")


+		func(_ context.Context, component components.Component) buildOutcome {
+			// workerEnv (captured) carries the cancellation context.
+			//nolint:contextcheck // env carries the ctx
+			componentResults, buildErr := BuildComponent(workerEnv, component, workDirFactory, options)


+				if !options.ContinueOnError {
+					cancel()


Protect shared event nesting from concurrent workers, return cancellation when queued builds are skipped, and run chroot cleanup with bounded, cancellation-detached contexts. Refresh CLI and MCP snapshots for the updated --unique-mockroot description. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4b65bb80-fb20-4e66-aaac-2bf4ae60835c

Copilot

🔵 Needs a closer look

Human assessment is needed to evaluate shared-SRPM corruption risks, potentially excessive nested parallelism, and missing workflow tests for the new concurrent build path.

4 open issues

1 resolved since last review

Return an error when cancellation skips queued builds

_{Review effort: Balanced}

Probe mock help for bootstrap-cache support before scrubbing unique roots. On older versions, scrub only the main root and remove the validated unique bootstrap directory with sudo after mock exits, preserving shared caches. Cover capability detection, custom root paths, cleanup failures, and cache preservation with regression tests, and update user and agent guidance. Validation: mage all passed with refreshed Docker group membership; mage mutationDiff HEAD killed all 24 mutants with none surviving or uncovered. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1dd8217a-74b0-49ad-9004-917ec834302f

Copilot

🟡 Changes recommended

With --unique-mockroot --dry-run, empty command output causes ScrubRoot path validation to fail instead of previewing cleanup; capability and path discovery must be skipped during dry runs.

5 open issues

_{Review effort: Balanced}

Skip mock capability and root-path discovery when previewing unique-root cleanup. Keep the wrapped scrub command so dry runs log cleanup without requiring output from commands that are not executed. Preserve dry-run behavior across runner clones, add regression coverage for default and private base directories, and document the cleanup preview. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: fc0e0213-1257-4970-b4a6-e865ac20b402

Copilot

🔵 Needs a closer look

Human assessment is needed because concurrent builders can share and scrub the same overridden root directory, risking build interference, while parallel scheduling and failure handling lack behavioral tests.

4 open issues

1 resolved since last review

Dry-run root discovery fails before cleanup preview

_{Review effort: Balanced}

Pass --disable-plugin yum_cache whenever mock uses --uniqueext, leaving non-unique commands unchanged. Cover the unique-root lifecycle and update user and agent documentation in both lockfile modes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e25b5788-5449-4db1-ae9d-5c235e69dbc5

Copilot

🔵 Needs a closer look

Human assessment is needed because explicit rootdir overrides can cause concurrent workers to share and scrub the same directory, while the new orchestration lacks regression coverage.

4 open issues

_{Review effort: Balanced}

Dan Streetman (ddstreet) · 2026-11-11T04:13:26Z

Nan Liu (@liunan-ms) I don't think I have the ACL for Copilot to actually respond to me, and i'm pretty sure most of its earlier complaints have been resolved...can you ask it to please recheck all its open issues to see if any still need attention?

Nan Liu (liunan-ms)

The changes look good, confirmed that all Copilot comments are addressed in the code.

Copilot AI balanced review requested due to automatic review settings October 22, 2026 01:12

Copilot started reviewing on behalf of Dan Streetman (ddstreet) October 22, 2026 01:12

View session

Dan Streetman (ddstreetmicrosoft) mentioned this pull request Oct 22, 2026

feat(mock): scope build environments to configuration directories microsoft/azurelinux#19368

Closed

Copilot AI reviewed Oct 22, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings October 27, 2026 20:56

Copilot started reviewing on behalf of Dan Streetman (ddstreet) October 27, 2026 20:57

View session

Copilot AI reviewed Oct 27, 2026

View reviewed changes

Antonio Salinas (Tonisal-byte) reviewed Oct 28, 2026

View reviewed changes

Dan Streetman (ddstreet) force-pushed the mock-uniqueext branch from 22447c3 to c16a09e Compare November 3, 2026 19:07

Copilot AI balanced review requested due to automatic review settings November 3, 2026 19:07

Copilot started reviewing on behalf of Dan Streetman (ddstreet) November 3, 2026 19:21

View session

Copilot AI reviewed Nov 3, 2026

View reviewed changes

Copilot AI balanced review requested due to automatic review settings November 10, 2026 15:52

Copilot started reviewing on behalf of Dan Streetman (ddstreet) November 10, 2026 15:52

View session