Skip to content

Fix: run-tests rejects unknown options before the -- separator - #144

Open
rquidute wants to merge 5 commits into
project-chip:v2.16-cli-developfrom
rquidute:fix/1215-cli-reject-unknown-options
Open

rquidute wants to merge 5 commits into
project-chip:v2.16-cli-developfrom
rquidute:fix/1215-cli-reject-unknown-options

Conversation

@rquidute

Copy link
Copy Markdown
Contributor

Summary

th-cli run-tests silently accepted any unknown option and forwarded it to the Python test runner as an extra test parameter. A typo such as --prompt_timeout (instead of --prompt-timeout) only failed inside the SDK container, and the error was visible only in the backend logs.

Closes #1215

Changes

  • run-tests now splits its arguments at a literal --. Options before it are parsed strictly, so unknown options fail with exit code 2, for example No such option '--prompt_timeout'. Did you mean '--prompt-timeout'?
  • Arguments after -- are still forwarded unchanged as test parameters (-- --int-arg PIXIT.X:1).
  • Stray positional arguments before -- are now errors instead of being ignored.
  • Removed ignore_unknown_options and allow_extra_args from the command's context settings.

Behaviour change

Passing test parameters without -- (e.g. run-tests -t X --int-arg PIXIT.X:1) now errors. The help text and docs describe only the -- form.

Tests

Added three tests in tests/test_run_tests.py: unknown option rejected with suggestion, stray positional argument rejected, and args after -- still forwarded. Five other tests in that file (config-file related) fail identically on the unchanged branch.

Not covered here: surfacing SDK startup errors in the CLI output (needs a backend change).

run-tests accepted any unknown option and forwarded it to the Python
test runner as an extra test parameter, so a typo such as
--prompt_timeout (instead of --prompt-timeout) only failed inside the
SDK container, with the error visible only in the backend logs.

Parse options before a literal -- strictly, so unknown options fail
with "No such option" and a suggestion. Arguments after -- are still
forwarded as test parameters.

Closes #1215
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Explain Change →

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 03519487-312f-40c7-9d6d-9a1ec783dbf6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
Finishing Touches

Fix failing CI checks

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@mergify

mergify Bot commented Nov 18, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue). If it does not match the queue conditions yet, it enters the queue once it does.

  • Queue this pull request

@rquidute rquidute self-assigned this Nov 18, 2026
Suite updates carry an errors list (e.g. a failed suite setup), but the CLI
only printed the suite title and state, so the reason was visible only in the
backend log. Print each suite error once.

Related to #1215
Everything after -- is passed to the test runner, so an option of run-tests
itself there (e.g. -- --prompt-timeout 5, or the --prompt_timeout typo) was
either rejected by the runner deep inside the run or silently ignored. Fail
up front and tell the user to pass it before the separator. The option names
come from the command definition, so they cannot drift.

Related to #1215
Comment thread th_cli/commands/run_tests.py Resolved
A long error glued to the header with ' - ' is hard to read. Put multi-line
API error content below the header, and indent every line of a multi-line
suite error.

Related to #1215
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

[Bug] th-cli run-tests silently accepts unknown options (e.g. --prompt_timeout) and fails later inside the test

2 participants