[bhushan4work]

What does this change?

Adds icontains, regex, and equals text assertions to chatcops test, including input validation, automatic not- support, unit tests, end-to-end coverage, and documentation.

Closes #411

How I tested this

• pnpm --filter @chatcops/cli exec vitest run tests/test-command.test.ts (28/28 tests passed)
• Standalone YAML end-to-end evaluation using all three assertions and their not- variants (1 passed)
• pnpm typecheck
• pnpm lint
• pnpm test:coverage (132 test files, 1,688+ tests passed)

AI assistance

none

Checklist

• This PR targets dev
• I was assigned the linked issue
• pnpm -r build, pnpm test and pnpm -r typecheck pass locally
• Tests added or updated where it makes sense
• For a bug fix: the new test fails without my change
• Changeset added (pnpm changeset) if this changes core, widget or server
• Docs updated if behavior or options changed

[anurag629]

Thanks, this is clean, and the bad-value tests are thorough (not- with a bad value still fails, good). Locally merged with dev: build, typecheck, 67 CLI tests, lint and docs:check pass.

One small thing from #411: a pattern that doesn't compile. Right now readRegex (packages/cli/src/test/assertions/text.ts) passes new RegExp(...) straight through, so regex: "(" fails with the engine's own text ("Invalid regular expression: /(/: Unterminated group"), which doesn't say which assertion it came from. Please wrap both new RegExp calls, catch the SyntaxError, and throw something like regex: "(" isn't a valid regular expression (Unterminated group). Then add a test for that and for invalid flags ({ pattern: 'a', flags: 'zz' }), which throws the same way.

[anurag629]

Thanks for the quick turnaround @bhushan4work. The regex fix is what I asked for: both constructors are wrapped, bad flags get their own message, and not-regex with a bad pattern still fails (test-command.test.ts:429-458). Those tests fail with the old text.ts. Merged with current dev, build, typecheck, test:coverage, lint (no new diagnostics), api:check, publint and docs:check pass.

One more thing before I merge, in equals (packages/cli/src/test/assertions/text.ts:78). Only the reply is trimmed. A long expected reply is most naturally written as a block scalar:

- equals: |
    The Team plan is $19.

YAML gives that value a trailing \n, so it never equals the trimmed reply. equals would always fail, which someone would notice, but not-equals would always pass, which nobody would. Please trim value too (turn.text.trim() === value.trim()) and add a test with a trailing newline for both equals and not-equals.

Two small ones while you're in there:

• website/src/content/docs/core/testing.mdx:63 now reads "Each assertion has one key, judge has a model grade the reply..." since contains moved into the list. It needs a period after "one key".
• The flags message repeats itself: flags "zz" aren't valid regular expression flags (Invalid flags supplied to RegExp constructor 'zz'). I'd drop the parenthetical for flags. If you check the flags first with new RegExp('', flags) and then compile the pattern, the nested try at text.ts:46-62 goes away, and so do the throw error lines at 36 and 61, which can't be reached.

Heads up: #446 touches the same import block in assertions/index.ts, so whichever of the two lands second needs a small rebase there.

[anurag629]

@bhushan4work #446 is merged, so this one now conflicts in the import lines of packages/cli/src/test/assertions/index.ts. Keep both imports when you rebase, along with the equals changes from my last review.

[anurag629]

Thanks @bhushan4work, all of it is in. equals trims both sides, with the trailing newline cases for equals and not-equals (they fail with the old line). Flags are checked first, the nested try and dead rethrows are gone, the flags message no longer repeats itself, and the docs sentence is fixed. With current dev merged in (including #446), build, typecheck, test:coverage, lint (same as dev), api:check and docs:check pass, and text.ts is at 100% of lines. Merging once CI is green.

[anurag629]

Merged, thanks @bhushan4work.