Skip to content

Add --skills-dir, --skills and --methodology to the CLI - #1779

Merged
bearsyankees merged 4 commits into
mainfrom
feat/str-1649-skills-cli-flags
Nov 19, 2026
Merged

bearsyankees merged 4 commits into
mainfrom
feat/str-1649-skills-cli-flags

Conversation

@bearsyankees

@bearsyankees bearsyankees commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds two things the CLI did not have: a way to use your own skills without editing the installed package, and a way to give a testing methodology to every agent in a scan.

Before this change, a custom skill had to be put in strix/skills/ inside the install. A cross-cutting methodology (coordination rules, shared artifacts, completion criteria) had no good home: as a skill, it is only a name and a one-line description in a catalog, and an agent may never pick it.

New flags

Flag What it does
--skills-dir PATH (repeatable), STRIX_SKILLS_DIR Adds directories with extra skills in the same <category>/<name>.md layout as the built-in skills. The agents find them in the skill catalog. A custom skill with the same relative path as a built-in skill replaces it. STRIX_SKILLS_DIR takes several directories, separated by os.pathsep.
--skills a,b Loads the named skills into the root agent before the scan starts.
--methodology FILE_OR_SKILL (repeatable) Takes a Markdown file or a skill name. Strix puts the text in the system prompt of the root agent and of every child agent.
strix -n -t https://app.example.com \
  --skills-dir ./my-skills \
  --methodology ./authz-differential.md

How a methodology reaches every agent

  • resolve_methodology(ref) reads the file or the skill. A missing or empty file is a CLI error.
  • render_system_prompt(..., methodology=[{name, content}]) renders a <methodology> block, one tag per entry. The block tells agents to follow its coordination rules, to pass what a child needs in the delegated task, and to load or assign the skills it names. It also states that the methodology does not change the authorized scope.
  • The runner and the agent factory pass the same methodology to each child agent, so it does not depend on the model selecting a skill.
  • The run state stores the methodology, so a resumed scan keeps it.
  • methodology_tag(name) makes the tag name safe. An entry named methodology renders as <methodology_rules> so it does not nest inside the wrapper tag.

When to use which

  • --instruction / --instruction-file: facts about this engagement (targets, credentials, focus).
  • --methodology: how to do the work, reused across engagements, seen by every agent.
  • A skill: reference knowledge for one agent (a vulnerability class, a technology, a tool).

docs/advanced/skills.mdx (new), docs/usage/cli.mdx, docs/usage/instructions.mdx and docs/advanced/configuration.mdx document this.

Tests

tests/test_cli_skills.py (20 tests) covers flag parsing, directory precedence, STRIX_SKILLS_DIR, methodology resolution and errors, the rendered prompt for root and child agents, and the tag name rule. A real scan with --methodology and --skills-dir against a local test API showed the methodology in the root and child system prompts.

Closes STR-1649

Link to Devin session: https://app.devin.ai/sessions/b45989020f3943e3b150469e7bcf97ec
Open in Devin Desktop: https://app.devin.ai/desktop/session/b45989020f3943e3b150469e7bcf97ec?variant=devin
Requested by: @bearsyankees

--skills loads named skills into the root agent before the scan starts
and is persisted in run.json so --resume keeps them. --skills-dir and
STRIX_SKILLS_DIR add extra skill roots ahead of the packaged set so
installed builds can add or override skills without rebuilding. Docs
explain when to use an instruction file versus a skill.
@greptile-apps

greptile-apps Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 3/5

[Medium impact] The changes affect scan setup and the prompts used by Strix agents.

Fix the lost resume guidance and empty-prompt failure before merging.

Findings

  1. P1 Custom skills vanish on resume ▶
  2. P1 Skill name empties the prompt ▶
  3. P2 Distinct methodologies get dropped ▶
  4. P2 Saved skill directories are ignored ▶
Fix with agent prompt
### Issue 1
strix/interface/cli_args.py:571-573
**Custom skills vanish on resume.** A run started with `--skills-dir ./my-skills --skills custom/foo` saves the skill name but not its directory. A later `--resume RUN_NAME` in a new process restores the name without restoring the search path. The rebuilt prompt silently skips the skill; overrides of built-in skills silently revert to the packaged version. Save and restore the custom directories, or save the skill contents, before rebuilding resumed agents.

### Issue 2
strix/agents/prompt.py:138-142
**Skill name empties the prompt.** Loading a custom skill named `methodology` puts that key in `skill_content`. The new explicit `methodology=` argument then conflicts with `**skill_content`, raising `TypeError` even without `--methodology`. The exception handler returns an empty string, which the factory uses as the agent's instructions. Keep skill bodies separate from the template's named arguments so a valid skill name cannot erase the entire system prompt.

### Issue 3
strix/interface/cli_args.py:395-396
**Distinct methodologies get dropped.** `resolve_methodology` reduces file paths and qualified skill names to a short display name. This check then silently drops any later item with that name. For example, `--methodology ./team/methodology.md --methodology ./project/methodology.md` applies only the first file, even when their contents differ. Deduplicate by the resolved source, not the display tag, or reject the collision explicitly.

### Issue 4
strix/skills/__init__.py:64-66
**Saved skill directories are ignored.** The configuration guide supports settings through the JSON file's `env` map, but `env_skill_dirs` reads only `os.environ`. The configuration loader neither exports file values there nor recognizes `STRIX_SKILLS_DIR` as a settings field. Thus `{"env":{"STRIX_SKILLS_DIR":"/my/skills"}}` has no effect: custom names are rejected and built-in overrides are not used. Connect this option to the settings loader, or clearly document it as environment-only.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds command-line options to preload skills, use custom skill directories, and apply shared methodologies to scan agents.

  • Operators can choose skills for the root agent and add their own skill folders.
  • A scan methodology now guides the root agent and every child agent.
  • Resumed scans can reuse their saved skills and methodology.

Reviews (1) · Last reviewed commit: "Add --methodology to apply a file or ski..." · Reviewed by Greptile

Comment thread strix/interface/cli_args.py Resolved
Comment thread strix/agents/prompt.py Resolved
Comment thread strix/interface/cli_args.py Resolved
Comment thread strix/skills/__init__.py Resolved
@arcanist

arcanist Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Ready to merge Re-run Arcanist review

PR summary

Operators could not preselect root-agent skills or add skill directories from the CLI. They can now load named skills, search custom directories before packaged skills, and retain skill selections for resumed runs. Scan-wide methodologies previously had no dedicated CLI input. Operators can now apply repeated file or skill methodologies to root and child agents and retain them for resumed runs.

Review history (2)
  • Ready to merge
  • Ready to merge (this review)
🟡 Medium blast radius. If this breaks (2):
  1. Agents follow incomplete or misframed rules across a scan. You can turn off the methodology flag.
  2. Agents get the wrong skill or methodology content, or miss it entirely, while the rest of the scan settings stay in place.
Flow diagram
%%{init: {"theme": "neutral"}}%%
flowchart TD
  n0["Scan runner"]
  n1["Pass methodology to root agent"]
  n0 --> n1
  n2["Build agent instructions"]
  n1 --> n2
  n3{"Methodology supplied?"}
  n2 --> n3
  n4["Add methodology instructions"]
  n3 -->|"Yes"| n4
  n5["Instructions return to agent"]
  n4 --> n5
  n3 -->|"No methodology block"| n5
Loading
✅ Ruled out (2)
  • Could an interrupted run resume without its saved skills and methodologies?
    No, resume rejects that run because no agent snapshot exists yet, and runs with a snapshot have already saved both selections.
  • Can a methodology named after the wrapper break the prompt tags?
    No, every entry is normalized again before rendering, and a blank or wrapper-matching name becomes a distinct rules tag, so the outer section stays balanced.

A file named methodology.md rendered as nested <methodology> tags in the
system prompt. Such an entry now uses the tag methodology_rules.
@devin-ai-integration devin-ai-integration Bot changed the title Feat/str 1649 skills cli flags Add --skills-dir, --skills and --methodology to the CLI Nov 19, 2026
@bearsyankees
bearsyankees merged commit 8db295a into main Nov 19, 2026
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant