Skip to content

fix(skills): route decorator table classes away from database-interface - #75

Merged
Upd4ting merged 1 commit into
mainfrom
fix/skill-eval-decorators-neighbour
Nov 6, 2026
Merged

Upd4ting merged 1 commit into
mainfrom
fix/skill-eval-decorators-neighbour

Conversation

@Upd4ting

@Upd4ting Upd4ting commented Nov 6, 2026

Copy link
Copy Markdown
Member

Why

The eval case database-interface-decorators-neighbour flipped between runs with no description change. This PR makes it pass on every run. The case asks for a decorator-based table class and expects database-decorators-interface without database-interface. Sonnet often loaded both, because this skill's description claims "Table" and "define a database schema" and does not mention its neighbour.

Goal

The neighbour case passes on every run. Prompts that database-interface owns keep loading it.

Done means

  • The database-interface description routes declarations of decorator table and model classes to database-decorators-interface, and keeps AQL queries on those tables.
  • Repeated local evals of the neighbour case pass on every run, where main flips.
  • No case of either skill regresses against main, and the eval report scored against main is PASS.
  • pnpm run checks passes, and CI is green.

Change

One line in skills/database-interface/SKILL.md, the description:

  • "define a database schema" becomes "define an AQL schema". The neighbour already claims "register a schema".
  • A new sentence says the skill is not for declaring decorator-based table or model classes (RegisterTable, Field, Index, Fixture, Encrypted, Hashed, BasicDataModel) and points to database-decorators-interface. All seven are real exports of that package. Another sentence keeps AQL queries on their tables here, because BasicDataModel exposes an AQL table.

The description is 889 characters, under the 1024 limit.

Verification

Local evals with scripts/evals.mjs of the AntelopeJS plugin 0.7.0, published corpus plus this package's skills. The 10 cases of database-interface and database-decorators-interface ran on claude-opus-5-5-low and claude-sonnet-5-5-low.

Description Runs per model Neighbour case, Opus Neighbour case, Sonnet
main 7 7/7 4/7
this PR 5 5/5 5/5
  • Every run of this PR scored PASS, and the run scored --against a main run is PASS.
  • database-interface-duplicate-key loaded database-interface in every run, so listing Index does not pull that case away.
  • Opus fails one answers case in most runs, on this PR and on main alike. It is database-decorators-interface-modifiers, where the name check counts a relative import path (../db/user) as an invented name. That case belongs to the neighbour package and this change does not touch it.
  • codex-gpt was not run: the Codex CLI is not installed on the machine that ran these evals. The skill-eval gate run should cover it.
  • pnpm run checks: Checks: 8 passed, 0 failed.

@Upd4ting
Upd4ting merged commit 7ae3041 into main Nov 6, 2026
8 checks passed
@Upd4ting
Upd4ting deleted the fix/skill-eval-decorators-neighbour branch November 6, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant