Repository navigation
[bot] Merge master/84eedf18 into rel/dev - #1939
Merged
Merged
Conversation
The document skill scored new drafts only: a run that edited the document open in the editor failed on base_document_id, so a dataset could not check the copilot's edits at all. An item can now state `rework` with the base_document_id of the open document. The run must then report that id and reworks_open_document, and it may also have to keep listed page ids (a hidden page counts as lost), keep listed charts on their pages, leave a stated title, and keep the document's content.filters. Each is a strict check published only when the item states it. A new draft must not claim reworks_open_document either: that, and a base_document_id on a new draft, now fail document_new_draft rather than document_not_saved, which keeps meaning only that the draft is not saved yet. A rework is scored on aac-v2 documents only: a v1 draft fails it once, saying so, rather than through every check. v1 is on its way out, and its parts carry the Report names the wire check already flags. A malformed rework is rejected before any request, and so is one whose base_document_id is not the id of user_context.view.document: otherwise every run would fail on base_document_id and read as a copilot regression. Expected filters are compared as sent, keyed by filter id, so they have to be copied from a real aac-v2 run. jira: QA-30024 risk: low Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011CpUh9hFjL8SbdpFxBGRXp
A general question about the document open in user_context ran in the organization's default document format. On an organization that defaults to aac-v1, the agent got report_builder instead of document_builder and could read only the context summary of an aac-v2 document, so it could not say which charts the document shows. evaluate_agentic_general_question and run_agentic_general_question now take document_format, as the document skill does. It is sent on every turn, recorded in the dataset-run metadata, and an unknown value is rejected before any request. None keeps the server default. jira: QA-30024 risk: low Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011CpUh9hFjL8SbdpFxBGRXp
…coring feat(gooddata-eval): evaluate the copilot with a document open
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## rel/dev #1939 +/- ##
===========================================
+ Coverage 86.24% 86.31% +0.07%
===========================================
Files 346 346
Lines 26261 26397 +136
===========================================
+ Hits 22649 22785 +136
Misses 3612 3612 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🚀 Automated PR to perform merge from master into rel/dev with changes up to 84eedf1 (created by https://github.com/gooddata/gooddata-python-sdk/actions/runs/42960722333).