Skip to content

test: set each environment name once, so a running command never reads a blank API key - #37

Closed
nroi wants to merge 1 commit into
mainfrom
fix/testacc-set-env-once
Closed

nroi wants to merge 1 commit into
mainfrom
fix/testacc-set-env-once

Conversation

@nroi

@nroi nroi commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Written by AI agent

The acceptance suite's environ() now holds each name once with its last value, so the concurrent tfstate execs no longer read a blank MESHSTACK_API_KEY while a later start sets up its environment.

How flaky it was, and how I measured it

Before After
meshfed-release CI, concurrent subtest since 2026-10-07 2 of 49 runs failed (job 115331305948, job 115838981323) no CI runs yet
Throwaway test, 4 commands at once, 2000 times: blank API key or secret 1291 of 8000 commands 0 of 8000
The same, with os.CopyFS before each start as in the subtest 1454 of 8000 0 of 8000
The same, blank MESHSTACK_WORKSPACE (the CLI then answers 404) 274 and 560 of 8000 0 of 8000
TestAccTfstate against the local stack passes

The race detector stays silent here: every os.Getenv and os.Setenv takes the environment lock, and only the sequence of two Setenv calls on one name is not atomic.

The throwaway test (not committed)
func TestEnvRaceRepro(t *testing.T) {
	os.Setenv("MESHSTACK_API_KEY", "6f1c1f8e-1f6e-4d7e-9d39-2a8f3f3f3f3f")
	os.Setenv("MESHSTACK_API_SECRET", "secret")
	block := "00000000-0000-0000-0000-000000000001"
	server := fakemeshstack.Start(t, fakemeshstack.Options{
		ApiKeys:  []fakemeshstack.ApiKey{{ClientId: os.Getenv("MESHSTACK_API_KEY"), ClientSecret: "secret"}},
		TfStates: map[fakemeshstack.TfState][]byte{{Workspace: "race-ws", BuildingBlockUuid: block}: []byte(`{"version":4,"serial":1,"lineage":"l","resources":[]}`)},
	})
	blank := 0
	for range 2000 {
		c := newCLI(t, server.URL).withApiKey()
		c.setEnv(envWorkspace, "race-ws")
		var runs []*cliRun
		for range 4 {
			runs = append(runs, c.start("", "buildingblock", "tfstate", "show", block))
		}
		for _, run := range runs {
			if err := run.wait(); err != nil && (strings.Contains(err.Error(), "an API key needs") || strings.Contains(err.Error(), "selects no credential")) {
				blank++
			}
		}
	}
	t.Logf("%d of 8000 commands read a blank API key or secret", blank)
}

Put it in cmd/internal/testacc/ and run go test ./cmd/internal/testacc/ -run TestEnvRaceRepro -v without MESHSTACK_CLI_TEST_ACC. It needs no local stack.

🤖 Generated with Claude Code

…s a blank API key

The concurrent tfstate execs failed now and then with "an API key needs
MESHSTACK_API_KEY and MESHSTACK_API_SECRET together". Each start blanked the
key and set it again a moment later, while the execs started before it were
already reading their settings.

Before: 2 of 49 runs of the concurrent subtest failed this way in
meshfed-release CI since 2026-10-07 (jobs 115331305948 and 115838981323).
A throwaway test that started 4 commands at once through start(), against
internal/testutil/fakemeshstack, 2000 times, saw 1291 of 8000 commands read
a blank API key or secret, and 274 a blank MESHSTACK_WORKSPACE. With an
os.CopyFS before each start, as the subtest does, the counts were 1454 and
560.

After: the same test saw 0 of 8000 commands read a blank value in both
variants, and the whole TestAccTfstate passes against the local stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant