Skip to content

Fix RSI training guest execution - #163

Merged
w4ffl35 merged 1 commit into
masterfrom
fix/rsi-training-guest-diagnostics
Oct 14, 2026
Merged

w4ffl35 merged 1 commit into
masterfrom
fix/rsi-training-guest-diagnostics

Conversation

@w4ffl35

@w4ffl35 w4ffl35 commented Oct 14, 2026

Copy link
Copy Markdown
Member

Unblocks #9's live QLoRA run. Three guest-execution defects found while running the documented bounded command:

  1. Training image lacked a C compiler toolchain: torch's triton JIT compiles a CUDA driver shim at forward time. Added gcc, libc6-dev, python3-dev.
  2. Guest command pointed at /opt/headlesscode-rsi-training/*.py, unreachable under the sandbox path policy (only /workspace + system paths are open). The scripts already ship in the job snapshot from the frozen base commit, so run them from /workspace/scripts/rsi-training/ (stronger provenance: bound to the job digest). Image no longer copies the scripts.
  3. PEFT writes an incidental README.md into the adapter dir, which the strict eval-guest extractor rejects. Trainer now removes it and asserts the exact contracted 2-file set.
  4. Also: omitted-result error now includes the guest stdout tail (was exit code only, undebuggable).

Verified: manual guest training completes 8/8 steps; fleet training completes with adapter artifact. Paired eval pending a run whose base commit includes the trainer fix.

… error

torch's triton JIT needs a C compiler at forward time; without it QLoRA training fails in-guest. Also surface the guest stdout tail when a model job omits its result file, instead of reporting only the exit code.
@w4ffl35
w4ffl35 merged commit 14cd53b into master Oct 14, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant