Skip to content

feat(factory): add Factory Pi workflow skill - #7

Merged
0xnotdev merged 13 commits into
mainfrom
fm/factory-cp04-skill-correction
Sep 28, 2026
Merged

0xnotdev merged 13 commits into
mainfrom
fm/factory-cp04-skill-correction

Conversation

@0xnotdev

Copy link
Copy Markdown
Owner

Intent

Deliver a corrected replacement for Factory CP-04 PR #6 on a new branch at exact PR6 head, preserving CP-04 skill/probe/F-09 proofs and real CP-05 dogfood. Correct the false skill command boundary to accurately list landed evidence TASK-ID and status --json, while continuing to forbid absent /factory, factory run, and factory inspect. Correct now-false final-command claims in docs/probes/CP-04.md so the proof reflects the final tree. Keep the final diff from current landed main CP-04-only: BUILD_PLAN.md, docs/probes/CP-04.md, skills/factory/SKILL.md; do not change CP-05 code, Firstmate state, other projects, credentials, deployment, new features, Herdr lifecycle, or merge. Preserve CP-05 contract SHA-256 34864d74fd6a831ef57b9e85fea124bd93598a674201c680bab018ce5e6c7646. Independently validate npm ci, format, lint, typecheck, full tests, Pi 0.85.1 discovery and explicit/automatic skill invocation, installed CLI syntax, and read-only real task closure cross-check. Target current landed main. Run full no-mistakes review, test, document, lint, push, PR, and CI pipeline without skips, --yes, force-push, merge, or deployment; use agent:pi in this isolated NM_HOME.

What Changed

  • Added a skills/factory Pi skill that documents the Factory workflow, safe preview/apply process, and current CLI command boundary.
  • Added the CP-04 probe report covering skill discovery/invocation, F-09 dogfood evidence, CP-05 closure cross-checks, and final validation notes.
  • Marked the CP-04 BUILD_PLAN checklist items complete.

Risk Assessment

✅ Low: The change is tightly scoped to CP-04 documentation and skill instructions, preserves the required three-file diff and CP-05 digest, and the corrected provenance now separates historical 57785cf validation from later command-boundary probes.

Testing

I installed dependencies, built the CLI, verified the final diff is CP-04-only, exercised the end-user CLI surfaces for evidence/status plus absent run/inspect commands, confirmed the CP-05 contract digest, ran focused evidence/status tests, and validated Pi 0.85.1 skill discovery plus explicit and automatic Factory skill invocation; all targeted checks passed and transient dist/node_modules outputs were removed.

Evidence: Factory CLI and targeted test transcript
$ git rev-parse HEAD
7b49889aefc46f853091006632c41f1a63546102

$ git diff --name-only 3bc6e46c6cfb0f78b33103eacb86d9ea0006c15f..HEAD
BUILD_PLAN.md
docs/probes/CP-04.md
skills/factory/SKILL.md

$ node dist/src/cli.js --help
usage: factory <doctor|init|validate|context TASK-ID|sync --dry-run|--apply|evidence TASK-ID|status> [--root <project>] [--json]

Commands:
  doctor --home <home>       Read-only integration boundary probe
  init                       Create .factory directories and ignore runtime state
  validate                   Check versioned contracts and dependency graph
  context TASK-ID            Write a bounded attributed context pack and receipt
  sync --dry-run --home HOME Preview create-only backlog reconciliation
  sync --apply --home HOME   Publish after a fresh conflict-free preflight
  evidence TASK-ID           Validate task evidence against contract and Git SHA
  status [--home HOME]       Report backlog and product completion separately

Options:
  --root <path>              Project root (default: cwd)
  --home <path>              Explicit canonical Firstmate home
  --ctx-bin <path>           Override ctx executable
  --tasks-bin <path>         Override tasks-axi executable
  --token-budget <count>     Override the task token budget
  --byte-budget <count>      Override the task byte ceiling
  --json                     Print one schema_version=1 JSON object

$ node dist/src/cli.js evidence CP-05 --root . --json
{
  "schema_version": 1,
  "ok": true,
  "command": "evidence",
  "task_id": "CP-05",
  "receipt_path": ".factory/state/evidence/CP-05.json",
  "status": "unverified",
  "valid": false,
  "contract_sha256": "34864d74fd6a831ef57b9e85fea124bd93598a674201c680bab018ce5e6c7646",
  "tested_commit": null,
  "release_candidate": "7b49889aefc46f853091006632c41f1a63546102",
  "checks": [],
  "issues": [
    {
      "code": "RECEIPT_MISSING",
      "message": "No task evidence receipt has been recorded"
    }
  ]
}

$ node dist/src/cli.js status --root . --json
{
  "schema_version": 1,
  "ok": true,
  "command": "status",
  "root": "/home/ansh/.treehouse/Factory-86709f/5/Factory/.factory/state/nm-cp04-replacement/home/worktrees/5325f1d9fdcd/01M3MFHX55CT4S5ZR6S57YGXFA",
  "backlog": {
    "home": null,
    "available": false,
    "summary": {
      "queued": 0,
      "in_flight": 0,
      "done": 0,
      "unverified": 1
    },
    "all_tasks_done": false,
    "tasks": [
      {
        "task_id": "CP-05",
        "published_id": "factory-factory-cp-05",
        "state": null,
        "status": "unverified"
      }
    ]
  },
  "task_evidence": {
    "summary": {
      "passed": 0,
      "failed": 0,
      "stale": 0,
      "unverified": 1
    },
    "tasks": [
      {
        "schema_version": 1,
        "ok": true,
        "command": "evidence",
        "task_id": "CP-05",
        "receipt_path": ".factory/state/evidence/CP-05.json",
        "status": "unverified",
        "valid": false,
        "contract_sha256": "34864d74fd6a831ef57b9e85fea124bd93598a674201c680bab018ce5e6c7646",
        "tested_commit": null,
        "release_candidate": "7b49889aefc46f853091006632c41f1a63546102",
        "checks": [],
        "issues": [
          {
            "code": "RECEIPT_MISSING",
            "message": "No task evidence receipt has been recorded"
          }
        ]
      }
    ]
  },
  "product": {
    "status": "NOT COMPLETE",
    "release_candidate": "7b49889aefc46f853091006632c41f1a63546102",
    "completion_sha256": "ee8ba8379d82bc92331878a219542a424fa3ae46df30893e27ca56645b0921dc",
    "summary": {
      "passed": 0,
      "failed": 0,
      "stale": 0,
      "unverified": 2
    },
    "conditions": [
      {
        "id": "P-03",
        "outcome": "Task evidence is linked to the exact task contract and tested Git SHA, and a failed or stale result cannot be marked passed.",
        "method": "executable",
        "receipt_path": ".factory/state/completion/P-03.json",
        "status": "unverified",
        "tested_commit": null,
        "issues": [
          {
            "code": "RECEIPT_MISSING",
            "message": "No project-condition receipt has been recorded"
          }
        ]
      },
      {
        "id": "P-04",
        "outcome": "Product completion reports independently from backlog status and remains incomplete when a project-level check fails after every task closes.",
        "method": "executable",
        "receipt_path": ".factory/state/completion/P-04.json",
        "status": "unverified",
        "tested_commit": null,
        "issues": [
          {
            "code": "RECEIPT_MISSING",
            "message": "No project-condition receipt has been recorded"
          }
        ]
      }
    ]
  }
}

$ sha256sum .factory/tasks/CP-05.yaml
34864d74fd6a831ef57b9e85fea124bd93598a674201c680bab018ce5e6c7646  .factory/tasks/CP-05.yaml

$ node --test dist/test/evidence.test.js dist/test/status.test.js
✔ valid task evidence is bound to its exact contract and Git SHA (587.379322ms)
✔ dirty task contract cannot pass at unchanged HEAD (547.933062ms)
✔ prose-only pass cannot produce valid passing task evidence (585.224697ms)
✔ failed exit cannot produce valid passing task evidence (583.077914ms)
✔ missing artifact cannot produce valid passing task evidence (548.110999ms)
✔ changed contract cannot produce valid passing task evidence (540.647301ms)
✔ changed release SHA cannot produce valid passing task evidence (572.590918ms)
✔ status reports backlog progress separately from every product evidence condition (720.859923ms)
✔ a failed project check keeps all-closed work NOT COMPLETE (687.201323ms)
✔ dirty completion contract cannot pass at unchanged HEAD (717.76873ms)
ℹ tests 10
ℹ suites 0
ℹ pass 10
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 4089.505903

$ node dist/src/cli.js run --root . --json (expected unsupported)
{
  "schema_version": 1,
  "ok": false,
  "error": {
    "code": "INVALID_COMMAND",
    "message": "Expected command: doctor, init, validate, context, sync, evidence, or status",
    "details": {
      "command": "run"
    }
  }
}
exit=2

$ node dist/src/cli.js inspect --root . --json (expected unsupported)
{
  "schema_version": 1,
  "ok": false,
  "error": {
    "code": "INVALID_COMMAND",
    "message": "Expected command: doctor, init, validate, context, sync, evidence, or status",
    "details": {
      "command": "inspect"
    }
  }
}
exit=2
Evidence: Pi RPC skill discovery output
{"type":"response","command":"get_commands","success":true,"data":{"commands":[{"name":"llama","description":"Manage llama.cpp router models","source":"extension","sourceInfo":{"path":"<inline:llama.cpp>","source":"inline","scope":"temporary","origin":"top-level"}},{"name":"skill:factory","description":"Use Factory to turn reviewed project intent into validated task contracts, bounded source-attributed context, and a safe Firstmate backlog preview. Use when initializing or validating a Factory project, preparing worker context, or previewing and publishing tasks to a named Firstmate home.","source":"skill","sourceInfo":{"path":"/home/ansh/.treehouse/Factory-86709f/5/Factory/.factory/state/nm-cp04-replacement/home/worktrees/5325f1d9fdcd/01M3MFHX55CT4S5ZR6S57YGXFA/skills/factory/SKILL.md","source":"local","scope":"temporary","origin":"top-level","baseDir":"/home/ansh/.treehouse/Factory-86709f/5/Factory/.factory/state/nm-cp04-replacement/home/worktrees/5325f1d9fdcd/01M3MFHX55CT4S5ZR6S57YGXFA/skills/factory"}}]}}
{"type":"response","command":"abort","success":true}
Evidence: Explicit /skill:factory invocation output
Implemented: `doctor`, `init`, `validate`, `context TASK-ID`, `sync --dry-run|--apply`, `evidence TASK-ID`, `status`; forbidden inventions: `/factory`, `factory run`, `factory inspect`.
Evidence: Automatic Factory skill invocation summary
{
  "command": "PI_OFFLINE=1 pi --offline --approve --no-context-files --no-session --tools read --no-skills --skill skills/factory/SKILL.md --provider openai-codex --model gpt-5 --mode json -p <automatic Factory prompt>",
  "read_paths": [
    "/home/ansh/.treehouse/Factory-86709f/5/Factory/.factory/state/nm-cp04-replacement/home/worktrees/5325f1d9fdcd/01M3MFHX55CT4S5ZR6S57YGXFA/skills/factory/SKILL.md",
    "package.json",
    "skills/factory/docs/CONTRACTS.md",
    "skills/factory/docs/WORKFLOW.md",
    "/home/ansh/.treehouse/Factory-86709f/5/Factory/.factory/state/nm-cp04-replacement/home/worktrees/5325f1d9fdcd/01M3MFHX55CT4S5ZR6S57YGXFA/docs/CONTRACTS.md",
    "/home/ansh/.treehouse/Factory-86709f/5/Factory/.factory/state/nm-cp04-replacement/home/worktrees/5325f1d9fdcd/01M3MFHX55CT4S5ZR6S57YGXFA/docs/WORKFLOW.md",
    "src/cli.ts"
  ],
  "final_text": "For this Factory source checkout, build then use:\n\n`` `sh\nnpm run build\nnode dist/src/cli.js doctor --root <project-root> --home <named-firstmate-home> --json\nnode dist/src/cli.js init --root <project-root> --json\nnode dist/src/cli.js validate --root <project-root> --json\nnode dist/src/cli.js context <TASK-ID> --root <project-root> --json\nnode dist/src/cli.js sync --dry-run --root <project-root> --home <named-firstmate-home> --json\nnode dist/src/cli.js sync --apply --root <project-root> --home <named-firstmate-home> --json\nnode dist/src/cli.js evidence <TASK-ID> --root <project-root> --json\nnode dist/src/cli.js status --root <project-root> --home <named-firstmate-home> --json\n`` `\n\nDo **not** invent or use:\n\n`` `sh\n/factory ...\nfactory run ...\nfactory inspect ...\n`` `\n\n`sync --apply` is only valid after reviewing the exact named-home `sync --dry-run` result."
}

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 1 issue found → auto-fixed ✅
  • 🚨 docs/probes/CP-04.md:477 - The final validation section is still anchored to dogfood candidate 57785cf, but it now claims Pi invocations at that SHA returned evidence TASK-ID and status --json; at 57785cf the skill still listed only doctor/init/validate/context/sync and explicitly forbade evidence/status, so this proof record does not reflect an actually validated tree. Update the proof to either cite a validation run against the corrected target commit or separate the historical 57785cf result from the later source correction.

🔧 Fix: Correct CP-04 validation provenance
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • git diff --name-only 3bc6e46c6cfb0f78b33103eacb86d9ea0006c15f..HEAD
  • npm ci
  • npm run build
  • node dist/src/cli.js --help
  • node dist/src/cli.js evidence CP-05 --root . --json
  • node dist/src/cli.js status --root . --json
  • node dist/src/cli.js run --root . --json and node dist/src/cli.js inspect --root . --json expecting unsupported commands
  • sha256sum .factory/tasks/CP-05.yaml
  • node --test dist/test/evidence.test.js dist/test/status.test.js
  • printf ... | PI_OFFLINE=1 pi --offline --approve --no-session --no-context-files --no-extensions --no-prompt-templates --no-themes --no-skills --skill skills/factory --mode rpc
  • PI_OFFLINE=1 pi --offline --approve --no-context-files --no-session --no-tools --no-skills --skill skills/factory/SKILL.md --provider openai-codex --model gpt-5 -p '/skill:factory ...'
  • PI_OFFLINE=1 pi --offline --approve --no-context-files --no-session --tools read --no-skills --skill skills/factory/SKILL.md --provider openai-codex --model gpt-5 --mode json -p <automatic Factory prompt>
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@0xnotdev
0xnotdev merged commit 8337ab7 into main Sep 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant