Repository navigation
chat eval (#1078): gold set, offline scorer and first staging baseline - #1219
PRAteek-singHWY wants to merge 5 commits into
Conversation
24 questions in the ask_regressions format (the OWASP#962 threat-modeling pair, password storage, AI prompt injection, two abstain cases), each with the CRE ids a good answer should surface. CRE ids are checked against the CRE table; the two AI ids come from OWASP#1124 and are newer than the Aug cache. ask_eval/gold.py scores a /rest/v1/completion response with no LLM and no network: CRE recall, required cite groups through multicite (SAMM must be cited when the question names SAMM), abstain correctness, and [n] markers that point past the returned passages. to_phase6_cases feeds the same gold into phase6_eval's hit@k so the Ragas/DeepEval bridge runs on real cases. Faithfulness beyond marker validity needs an LLM judge and is left out.
…cores them Live mode asks a running OpenCRE each gold question and writes the answers to JSONL, so the run can be re-scored later without paying for the LLM again. Recorded mode scores a JSONL with no network. --baseline compares two runs and lists which cases flipped (current chat vs OWASP#1161 top-k). Failed requests and missing rows are reported apart and kept out of the metrics, so a 503 does not read as a wrong answer.
First live run against staging.opencre.org (24 questions) showed two scorer bugs and two gold rows that were too narrow: - [n] markers point into owasp_agent.citations on staging, not into table, so every marker was flagged as bad. Count citations when present. - staging prefixes replies with "Answer: " and keeps accurate=true on refusals, so a correct refusal was scored as an answer. Strip the label. - secrets_in_code accepts 508-702 (Use key vaults), dependencies accepts 715-223 / 715-334. Both are correct CREs the first gold missed; noted in the rows. Runner sends a named User-Agent: Cloudflare in front of opencre.org answers urllib's default agent with 403 / error 1010.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
One improvement that came to mind while going through the scorer: would it be useful to track CRE precision and recall separately? Currently, 'cre_hit' only checks whether at least one expected CRE ID appears, so it doesn't tell us whether the response also retrieved irrelevant CREs which excludes extra unnecessary info being passed into the answers. Having both metrics could give us a better picture of retrieval quality. What's your take on it? |
Hi @SomyaJain0308 Good idea, added |
Share of returned CREs the gold row lists. Reported only, not part of pass/fail, since gold rows list accepted CREs rather than every correct sibling. Staging run 2026-11-09: 0.705. Suggested by @SomyaJain0308.
…verdict - Gold set 24 -> 29: two compound questions where both halves must be cited (password storage + credential stuffing, session cookies + XSS) and three more refusals (future Top 10 edition, off topic, unknown CVE). - application/utils/ask_eval/faithfulness.py: hermetic side of the faithfulness eval. Picks the judgeable answers, pulls the evidence passages, counts passages cut at the API's 500 char cap, summarises judge verdicts. Tested with a fake judge. - scripts/chat_eval_faithfulness.py: optional local Ragas faithfulness run over a recording, listing the claims the judge found unsupported. Not in make test or any requirements file. - docs/chat_eval.md: metrics, how to run, staging results (26/29), the evidence truncation finding (62% of passages cut) and the verdict: hermetic scorer in make test, Ragas as an optional local script, not CI.
Part of #1078.
Adds a 24-question gold set for chat (in the ask_regressions format) and a small scorer: right CRE found, SAMM cited when the question asks for SAMM, refuses when it should, and every [n] points at a real source. No LLM, so it runs in make test.
scripts/chat_eval_gold.py --liveasks a running OpenCRE the questions and saves the answers. After that, scoring is offline.Ran it on staging.opencre.org: 22/24. Misses: the 2031 Global AppSec question gets the 2025 DC event instead of "I don't know", and the SMS 2FA answer is right but misses CRE 354-752.
For "does the source really support the sentence" you need an LLM judge, so I'd keep Ragas as an optional local command, not CI.
@SomyaJain0308, review and more questions welcome.