Repository navigation
fix Harbor oracle reward=1 path (engines, Playwright, fonts, CI, prisma, rsbuild) - #17
Merged
Merged
Conversation
…s, CI) - Pin CI/release runners to ubuntu-24.04 (Playwright unsupported on 26.04) - Per-repo Harbor Node images from engines.node (22 for ^22 caret, 24 for >=24) - Install Playwright Chromium in images for browser-mode hidden-tests - Prefetch next/font Google Fonts into Next mock map (fixes Docker ETIMEDOUT) - Ignore next-env.d.ts / tsbuildinfo in only-changed; local validate asserts Node - Add oracle solution for rr7 hydration-heading-viewport-attr - Batch runner for control×32 + agentic×32 Harbor oracle matrix Assisted-by: Composer Co-authored-by: Jason Zhang <xzha4350@gmail.com>
Root *.tsx includes were pulling in benchmark/xstack archive fixtures and failing `pnpm exec biome lint .` on CI. Assisted-by: Composer Co-authored-by: Jason Zhang <xzha4350@gmail.com>
next-turbopack-boilerplate gitignores next-env.d.ts; without `npx next typegen` after install, `@typecheck` fails on image imports. Assisted-by: Composer Co-authored-by: Jason Zhang <xzha4350@gmail.com>
Epic Stack images need .env + prisma migrate/generate --sql so typecheck and vitest grade. Rsdoctor skips root prepare (full monorepo build) and builds the client graph after install. Local validate-tasks --oracle mirrors postInstall via ensureRepoReady. Assisted-by: Cursor Agent Co-authored-by: Jason Zhang <xzha4350@gmail.com>
Injected bugs (e.g. broken import) must not run during image build steps; baseline still includes the post-setup tree for agents. Assisted-by: Cursor Agent Co-authored-by: Jason Zhang <xzha4350@gmail.com>
TanStack Start example depends on unpublished @tanstack/react-start@^1.169.6; preInstall pins npm 1.169.5. Portainer frozen install hits pnpmfileChecksum mismatch from configDependencies — use --no-frozen-lockfile. Batch runner gains --skip-ok to reuse existing reward=1 jobs. Assisted-by: Cursor Agent Co-authored-by: Jason Zhang <xzha4350@gmail.com>
…yers Clone/install/postInstall/fonts/typegen/Playwright live in agentic-react/base-<repo>:<cacheKey> (keyed by Dockerfile + SHA + lockfiles). Task Dockerfiles FROM that base and only apply setup edits + baseline; agentic overlay stays a thin layer on the task image. prepare-variants and run-oracle-batch build/cache bases before trials. Assisted-by: Cursor Agent Co-authored-by: Jason Zhang <xzha4350@gmail.com>
…he timings All 64 Harbor oracle trials reward=1 with shared repo bases. environment_setup drops from tens–hundreds of seconds to ~2–7s per task after the base is cached. Assisted-by: Cursor Agent Co-authored-by: Jason Zhang <xzha4350@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes so Harbor oracle reaches reward 1 on all 32 control + 32 agentic tasks, plus CI runner pin and shared per-repo base images.
Harbor / grading
harbor.nodeImage(Node 22 vs 24) fromengines.node(jsoncrack / Next 16 need ≥24)ensurePlaywrightBrowsers/assertHostNodeForReponext typegenin repo basespostInstallcopies.env,prisma migrate deploy+generate --sqlignoreInstallScripts+ client graph build; setup edits after base installpreInstallpins unpublished@tanstack/react-start@^1.169.6→ npm1.169.5--no-frozen-lockfile(pnpmfileChecksum vs configDependencies)agentic-react/base-<repo>:<cacheKey>(SHA + Dockerfile + lockfiles); thin task + agentic overlay layersvalidate-tasks --oraclemirrors viaensureRepoReadyRepo CI
.github/workflowsruns-onto ubuntu-24.04benchmark/**Results
64/64 Harbor oracle reward=1 (control × 32 + agentic × 32). CSVs:
experiments/oracle-rewards.csv,experiments/oracle-timings.csv.Repo base cache (mean environment_setup)
astro-web-checknext-turbopack-boilerplatenext-webpack-jsoncrackrr7-epic-stackrsbuild-rsdoctortanstack-start-trellauxvite-shadcn-adminwebpack-portainerControl (32)
astro-web-check__island-results-not-hydratedastro-web-check__runtime-server-status-timingsastro-web-check__ssr-home-background-windowastro-web-check__ui-server-status-labelsnext-turbopack-boilerplate__hydration-locale-switchernext-turbopack-boilerplate__ssr-current-count-crashnext-turbopack-boilerplate__ui-counter-button-colornext-turbopack-boilerplate__validation-counter-rangenext-webpack-jsoncrack__build-webpack-wasm-confignext-webpack-jsoncrack__hydration-hero-ctanext-webpack-jsoncrack__runtime-editor-title-crashnext-webpack-jsoncrack__ui-bottom-bar-valid-labelrr7-epic-stack__hydration-heading-viewport-attrrr7-epic-stack__ssr-loader-missing-user-404rr7-epic-stack__state-double-check-buttonrr7-epic-stack__ui-marketing-headingrsbuild-rsdoctor__build-broken-importrsbuild-rsdoctor__logic-windows-asset-pathsrsbuild-rsdoctor__state-bundle-size-sort-orderrsbuild-rsdoctor__ui-plugins-page-labelstanstack-start-trellaux__hydration-home-updated-attanstack-start-trellaux__state-column-card-ordertanstack-start-trellaux__type-board-loader-paramtanstack-start-trellaux__ui-card-remove-order-badgevite-shadcn-admin__runtime-tasks-status-crashvite-shadcn-admin__state-signout-keeps-uservite-shadcn-admin__type-refactor-column-header-labelvite-shadcn-admin__ui-dashboard-export-buttonwebpack-portainer__build-webpack-alias-resolutionwebpack-portainer__state-expandable-message-togglewebpack-portainer__type-loading-button-root-causewebpack-portainer__ui-add-button-default-labelAgentic (32)
astro-web-check__island-results-not-hydratedastro-web-check__runtime-server-status-timingsastro-web-check__ssr-home-background-windowastro-web-check__ui-server-status-labelsnext-turbopack-boilerplate__hydration-locale-switchernext-turbopack-boilerplate__ssr-current-count-crashnext-turbopack-boilerplate__ui-counter-button-colornext-turbopack-boilerplate__validation-counter-rangenext-webpack-jsoncrack__build-webpack-wasm-confignext-webpack-jsoncrack__hydration-hero-ctanext-webpack-jsoncrack__runtime-editor-title-crashnext-webpack-jsoncrack__ui-bottom-bar-valid-labelrr7-epic-stack__hydration-heading-viewport-attrrr7-epic-stack__ssr-loader-missing-user-404rr7-epic-stack__state-double-check-buttonrr7-epic-stack__ui-marketing-headingrsbuild-rsdoctor__build-broken-importrsbuild-rsdoctor__logic-windows-asset-pathsrsbuild-rsdoctor__state-bundle-size-sort-orderrsbuild-rsdoctor__ui-plugins-page-labelstanstack-start-trellaux__hydration-home-updated-attanstack-start-trellaux__state-column-card-ordertanstack-start-trellaux__type-board-loader-paramtanstack-start-trellaux__ui-card-remove-order-badgevite-shadcn-admin__runtime-tasks-status-crashvite-shadcn-admin__state-signout-keeps-uservite-shadcn-admin__type-refactor-column-header-labelvite-shadcn-admin__ui-dashboard-export-buttonwebpack-portainer__build-webpack-alias-resolutionwebpack-portainer__state-expandable-message-togglewebpack-portainer__type-loading-button-root-causewebpack-portainer__ui-add-button-default-labelNo model trials. No secrets. Nothing still failing.