Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,3 +282,9 @@ GitHub.com.
Rootform Action source is licensed under [Apache License 2.0](LICENSE).
Rootform binary release terms are separate and ship with each distribution
archive.

Reports are requested with `--details`: file outputs and enabled artifacts
contain complete CLI evidence. Summary and the optional PR comment preserve
CLI summaries and verdicts, fold secondary details, and link the full report
when GitHub inline limits require a shorter display. No second analysis is
performed to produce the inline display.
227 changes: 127 additions & 100 deletions dist/shared/index.js

Large diffs are not rendered by default.

8 changes: 5 additions & 3 deletions docs/adr/006-github-native-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,11 @@ races. GitHub does not support conditional unsafe REST mutations for comments,
so check-then-PATCH alone cannot provide an atomic guarantee. Do not pretend
otherwise or add a second comment/lock API.

Summary and comment carry CLI Markdown plus run/version/evidence links. Reports
above GitHub limits are preserved intact as files; inline presentation explains
its omission and links evidence. No semantic truncation/reconstruction.
Summary and comment carry CLI Markdown plus run/version/evidence links. Markdown
outputs always request `--details`; complete files stay available even when
GitHub's inline limit requires omitting exhaustive disclosure contents. Primary
CLI summaries and verdicts remain visible when they fit, with a complete-report
link. No conclusions or counts are reconstructed from JSON.

## Versioned Action refs

Expand Down
8 changes: 8 additions & 0 deletions scripts/integration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ function run(name: string, kind: BusinessKind, values: Partial<RunOptions>, expe
assert.equal(result.failure, undefined, `${name}: ${result.failure?.message}`);
assert.ok(result.paths.form, `${name}: missing Form`);
for (const path of Object.values(result.paths)) assert.ok(path && statSync(path).size > 0);
if (result.paths.report) {
const report = readFileSync(result.paths.report, "utf8");
assert.ok(!report.includes("Each list above shows at most"), `${name}: compact CLI preview`);
assert.ok(
!/\d+ of \d+ (?:entries|evaluations) shown/u.test(report),
`${name}: incomplete Markdown list`,
);
}
evidence.push({
name,
exitCode: result.exitCode ?? null,
Expand Down
13 changes: 13 additions & 0 deletions specs/006-github-native-actions/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,16 @@ consumer. Exact CLI pins are distinct from Action Git refs. Public workflow
qualification uses synthetic data only. The owner authorizes this replacement
surface; release refs may first be created only after reviewed green merged
source. No existing tag is moved by this change.

## Complete review clarification

- Owner approval: @soulbah — 2026-10-02.
- GitHub Actions request `--details` for every Markdown-producing run/check.
Full reports remain in step outputs and enabled artifacts; long lists use
disclosure rather than a CLI preview instruction in GitHub.
- Review composition preserves CLI statements, tables and recorded verdicts,
uses one H2 identity, H3 blocks and H4 subsections, and separates major blocks.
It does not derive conclusions from Form or Policy JSON.
- Inline size limits may omit secondary/exhaustive detail, with an explicit
complete-report link, while retaining CLI-written primary summaries and
verdicts. File outputs and artifacts always retain the complete report.
7 changes: 7 additions & 0 deletions specs/006-github-native-actions/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,10 @@
- T005 public qualification and lead review: Complete. Delivery tracked by Action PR 19 and create-only first-v1 workflow; both must complete before the delivery claim. Requirements 2–10.

Proof and exact identities are in evidence/qualification.md. Merge/publication status is visible in the linked PR and workflow; no existing tag is moved.

- T006 complete review reporting: implementation and local validation complete.
Markdown-producing calls request `--details`; composition preserves CLI
statements within one identity and collapsed audit block. Runtime tests,
ten published-binary integrations, full gate and bundle reproduction pass.
Final public consumer proof is tracked in
[qualification PR 3](https://github.com/rootform-dev/action-qualification/pull/3).
57 changes: 55 additions & 2 deletions src/report.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ test("forwards exact CLI Markdown without adding derived counts", () => {
const markdown = "## Rootform check\n\nNo policy findings.\n";

expect(renderReport({ version: "0.1.0", markdown })).toBe(
"# Rootform\n\nCLI `0.1.0`.\n\n## Rootform check\n\nNo policy findings.\n",
"## Rootform\n\nCLI `0.1.0`.\n\n## Rootform check\n\nNo policy findings.\n",
);
});

Expand All @@ -29,7 +29,7 @@ test("combines CLI reports with section framing while preserving their content",
expect(combinedPath).toBe(join(directory, "report.md"));
if (!combinedPath) throw new Error("Combined report missing");
expect(readFileSync(combinedPath, "utf8")).toBe(
`## Architecture\n\n${analysis.trimEnd()}\n\n## Policies\n\n${policy.trimEnd()}\n`,
`## Rootform\n\n${analysis.trimEnd()}\n\n---\n\n${policy.trimEnd()}\n`,
);
} finally {
rmSync(directory, { force: true, recursive: true });
Expand Down Expand Up @@ -60,3 +60,56 @@ test("omits an oversized CLI report whole and preserves evidence links", () => {
expect(rendered).not.toContain("oversized-row");
expect(rendered).not.toContain("CLI-BODY-END");
});

test("current CLI composition keeps one identity and audit details after both result blocks", () => {
const dir = mkdtempSync(join(tmpdir(), "rootform-current-review-"));
const architecture =
"## Rootform\n\n### Architecture\n\n**76 instances added.**\n\n#### Planned changes\n\n| Change | Instances |\n| --- | ---: |\n| + Added | 76 |\n\n---\n\n### Details\n\n<details>\n<summary>Provenance</summary>\n\n- **Input:** `form.json`\n\n</details>\n";
const policies =
"## Rootform\n\n### Policies\n\n> [!CAUTION]\n> **Verdict: VIOLATED**\n\n#### Planned architecture\n\n1 evaluation violated.\n\n---\n\n### Details\n\n<details>\n<summary>Provenance</summary>\n\n- **Origin:** Plan\n\n</details>\n";
try {
const a = join(dir, "analysis.md"),
p = join(dir, "check.md");
writeFileSync(a, architecture);
writeFileSync(p, policies);
const path = combineReports([a, p], dir);
if (!path) throw new Error("missing report");
const report = readFileSync(path, "utf8");
expect(report.match(/^## Rootform$/gmu)).toHaveLength(1);
expect(report.match(/^### /gmu)).toHaveLength(3);
expect(report.match(/^---$/gmu)).toHaveLength(2);
expect(report.indexOf("### Policies")).toBeLessThan(report.indexOf("### Details"));
expect(report).toContain("| + Added | 76 |");
expect(report).toContain("> **Verdict: VIOLATED**");
expect(report).toContain("- **Origin:** Plan");
} finally {
rmSync(dir, { force: true, recursive: true });
}
});

test("oversize exhaustive details shrink inline while CLI verdict and counts remain", () => {
const markdown = `## Rootform\n\n### Policies\n\n> [!CAUTION]\n> **Verdict: VIOLATED**\n\n#### Planned architecture\n\n1 Policy selected. 1200 evaluations violated.\n\n<details>\n<summary>Violated: 1200 evaluations</summary>\n\n${"- `object.resource`\n".repeat(1200)}\n</details>\n`;
const rendered = renderReport({
version: "0.1.0",
markdown,
artifactUrl: "https://github.com/rootform-dev/action/actions/runs/7/artifacts/11",
limit: 1000,
});
expect(Buffer.byteLength(rendered)).toBeLessThanOrEqual(1000);
expect(rendered).toContain("> **Verdict: VIOLATED**");
expect(rendered).toContain("1200 evaluations violated.");
expect(rendered).toContain("<summary>Violated: 1200 evaluations</summary>");
expect(rendered).toContain("Complete details remain available");
expect(markdown.match(/object.resource/gu)).toHaveLength(1200);
});

test("legacy check framing preserves evaluated sides without status headings", () => {
const markdown =
"## Rootform Policies\n\n**Overall verdict: INDETERMINATE**\n\n### Before\n\n#### `sample.policy.requirement`\n\n**Indeterminate: 1 evaluation**\n\n### After\n\nAll evaluations passed.\n\n### Provenance\n\n- **Input:** `comparison.json`\n";
const rendered = renderReport({ version: "0.1.0", markdown });
expect(rendered).toContain("### Policies\n\n> [!WARNING]");
expect(rendered).toContain("#### Before");
expect(rendered).toContain("**`sample.policy.requirement`**");
expect(rendered).toContain("### Details");
expect(rendered).not.toContain("#####");
});
106 changes: 93 additions & 13 deletions src/report.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,94 @@ import { join } from "node:path";

export const REPORT_MARKER = "<!-- rootform-action-report:v1 -->";

// Framing reads Markdown structure only. CLI statements and tables stay exact.
function parts(markdown: string): { body: string; provenance: string } {
let body = markdown.trimEnd();
let provenance = "";
const current = body.startsWith("## Rootform\n");
const legacy = /^## Rootform (architecture|Policies)\n/u.exec(body);
if (!current && !legacy) return { body, provenance };
body = body.replace(/^## Rootform[^\n]*\n\n/u, "");
const boundary = current ? /\n---\n\n### Details\n\n/u : /\n### Provenance\n\n/u;
const match = boundary.exec(body);
if (match?.index !== undefined) {
provenance = body.slice(match.index + match[0].length).trim();
body = body.slice(0, match.index).trimEnd();
provenance = provenance
.replace(/^<details>\n<summary>Provenance<\/summary>\n\n/u, "")
.replace(/\n<\/details>$/u, "");
}
if (legacy) {
body = body.replace(/^### (Before|After)$/gmu, "#### $1");
body = body.replace(/^### (.+)$/gmu, "#### $1");
// Policy identities/outcomes are content, not navigation.
body = body.replace(
/^#### (`[^\n]+`|Incomplete coverage|Without target|Not evaluated)$/gmu,
"**$1**",
);
body = `### ${legacy[1] === "Policies" ? "Policies" : "Architecture"}\n\n${body}`;
body = body.replace(
/^\*\*((?:Overall verdict: )?(?:VIOLATED|INDETERMINATE|NOT EVALUATED)[^\n]*)\*\*$/gmu,
(line, words: string) =>
`> [!${words.includes("VIOLATED") ? "CAUTION" : "WARNING"}]\n> ${line}`,
);
}
return { body, provenance };
}

function compose(documents: string[]): string {
const blocks = documents.map(parts);
const bodies = blocks.map((block) => block.body).join("\n\n---\n\n");
const provenance = blocks
.filter((block) => block.provenance)
.map((block) => block.provenance)
.join("\n\n");
return `## Rootform\n\n${bodies}${provenance ? `\n\n---\n\n### Details\n\n<details>\n<summary>Provenance</summary>\n\n${provenance}\n\n</details>` : ""}\n`;
}

export function combineReports(paths: string[], directory: string): string | undefined {
const reports = paths.filter((path) => existsSync(path));
if (reports.length === 0) return undefined;
if (reports.length === 1) return reports[0];
const path = join(directory, "report.md");
writeFileSync(
path,
reports
.map(
(file, index) =>
`## ${index === 0 ? "Architecture" : "Policies"}\n\n${readFileSync(file, "utf8").trimEnd()}\n`,
)
.join("\n"),
{ flag: "wx" },
);
writeFileSync(path, compose(reports.map((file) => readFileSync(file, "utf8"))), { flag: "wx" });
return path;
}

function omitFoldedDetails(markdown: string, notice: string): string {
const lines: string[] = [];
let depth = 0;
for (const line of markdown.split("\n")) {
if (line === "<details>") {
if (depth++ === 0) lines.push(line);
} else if (line === "</details>" && depth > 0) {
if (--depth === 0) lines.push("", notice, "", line);
} else if (depth === 0 || (depth === 1 && line.startsWith("<summary>"))) {
lines.push(line);
}
}
return lines.join("\n");
}

function primaryStatements(markdown: string): string {
const lines: string[] = [];
let keeping = true;
let subsection = false;
for (const line of markdown.split("\n")) {
if (line.startsWith("### ")) {
keeping = true;
subsection = false;
}
if (line.startsWith("#### ")) {
if (subsection) keeping = false;
subsection = true;
}
if (line === "<details>" || /^\*\*`/u.test(line)) keeping = false;
if (keeping) lines.push(line);
}
return lines.join("\n").trimEnd();
}

export function renderReport(options: {
version: string;
markdown?: string;
Expand All @@ -29,7 +99,6 @@ export function renderReport(options: {
commentState?: string;
limit?: number;
}): string {
const identity = `# Rootform\n\nCLI \`${options.version}\`.\n\n`;
const links = [
options.workflowUrl ? `[Workflow run](${options.workflowUrl})` : "",
options.artifactUrl ? `[Download evidence](${options.artifactUrl})` : "",
Expand All @@ -39,7 +108,18 @@ export function renderReport(options: {
const footer = `${links ? `\n\n${links}` : ""}${options.commentState ? `\n\nPR report: ${options.commentState}.` : ""}\n`;
const markdown =
options.markdown ?? "No CLI report was produced. See the failed step diagnostic.";
const full = `${identity}${markdown.trimEnd()}${footer}`;
if (Buffer.byteLength(full, "utf8") <= (options.limit ?? 900 * 1024)) return full;
const body = markdown.startsWith("## Rootform\n") ? markdown : compose([markdown]);
const identity = `## Rootform\n\nCLI \`${options.version}\`.\n\n`;
const framed = (content: string) =>
`${identity}${content.replace(/^## Rootform\n\n/u, "").trimEnd()}${footer}`;
const limit = options.limit ?? 900 * 1024;
const full = framed(body);
if (Buffer.byteLength(full, "utf8") <= limit) return full;
const notice = `Complete details remain available through the report step output${options.artifactUrl ? ` and [evidence artifact](${options.artifactUrl})` : ""}.`;
const folded = framed(omitFoldedDetails(body, notice));
if (Buffer.byteLength(folded, "utf8") <= limit) return folded;
const primary = framed(`${primaryStatements(body)}\n\n${notice}`);
if (Buffer.byteLength(primary, "utf8") <= limit && /^### (Architecture|Policies)$/mu.test(body))
return primary;
return `${identity}CLI report exceeds GitHub's inline size limit. Its complete file remains available through the report step output${options.artifactUrl ? " and evidence artifact" : ""}.${footer}`;
}
2 changes: 2 additions & 0 deletions src/run.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ describe("current CLI command transport", () => {
expect(f.commands[0]?.slice(0, 4)).toEqual(["rootform", "run", "plan.json", "--no-serve"]);
expect(f.commands[0]?.filter((v) => v === "-o")).toHaveLength(3);
expect(f.commands.flat()).not.toContain("--plan");
expect(f.commands[0]).toContain("--details");
expect(f.result.exitCode).toBeUndefined();
});

Expand Down Expand Up @@ -101,6 +102,7 @@ describe("current CLI command transport", () => {
test("raw check analyzes once then evaluates once with three outputs", () => {
const f = scenario({ kind: "check", side: "after", policies: ["security/*"] });
expect(f.commands.map((c) => c[1])).toEqual(["run", "check"]);
expect(f.commands[1]).toContain("--details");
expect(f.commands[0]?.filter((v) => v === "-o")).toHaveLength(1);
expect(f.commands[1]?.filter((v) => v === "-o")).toHaveLength(3);
expect(f.commands[1]?.[2]).toBe("form.json");
Expand Down
10 changes: 7 additions & 3 deletions src/run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ export const runCommand: CommandRunner = (command, cwd) => {
env: { ...cliEnvironment(), CI: "true", NO_COLOR: "1" },
maxBuffer: 16 * 1024 * 1024,
timeout: 300_000,
stdio: ["ignore", "pipe", "pipe"],
stdio: ["ignore", command[1] === "run" || command[1] === "check" ? "ignore" : "pipe", "pipe"],
windowsHide: true,
});
if (result.error) throw new Error("Rootform process failed or exceeded its execution limit");
if (result.status === null) throw new Error("Rootform process ended without an exit status");
return { exitCode: result.status, stderr: result.stderr, stdout: result.stdout };
return { exitCode: result.status, stderr: result.stderr, stdout: result.stdout ?? undefined };
};

// This transport hint reads no architecture or Policy result. The CLI still
Expand Down Expand Up @@ -95,7 +95,11 @@ export function runBusiness(options: RunOptions): ExecutionResult {
const pair = (file: string | undefined) =>
file ? ["--plan-file", file, "--require-enrichment"] : [];
const invoke = (args: string[], outputs: string[], check = false, cwd = options.workspace) => {
const result = runner([options.binary, ...args, "--no-pager", "--color", "never"], cwd);
const detailFlags = outputs.some((path) => path.endsWith(".md")) ? ["--details"] : [];
const result = runner(
[options.binary, ...args, ...detailFlags, "--no-pager", "--color", "never"],
cwd,
);
if (check) exitCode = result.exitCode;
if (result.exitCode !== 0) {
throw new RootformCommandError(
Expand Down
Loading