Skip to content

feat(browser-actions): add high-fidelity video capture - #2563

Merged
chubes4 merged 2 commits into
mainfrom
fix/issue-2559-wp-codebox
Oct 3, 2026
Merged

chubes4 merged 2 commits into
mainfrom
fix/issue-2559-wp-codebox

Conversation

@chubes4

@chubes4 chubes4 commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds opt-in high-quality video capture for wordpress.browser-actions. Instead of Playwright's realtime recorder, it captures device-scale PNG frames and encodes a VP9/WebM video once at the end, preserving the existing marker timeline and reporting dimensions and fps.

What changed

  • Added the validated video-quality=default|high argument; the default recording path remains unchanged.
  • High mode captures viewport screenshots at device scale on a 10 fps schedule and encodes them with ffmpeg using VP9 at CRF 18.
  • High-mode video metadata reports viewport × deviceScaleFactor dimensions and 10 fps; marker data continues to use the recording's monotonic timeline.
  • Added browser regression coverage decoding a 540×960 viewport at DPR 2 and verifying 1080×1920 output, fps, and marker metadata.

How to test

  1. Run npm ci && npm run build && npm run test:browser-video-capture; expect passes as recorded by Cook's deterministic gate.

Compatibility

Opt-in and backward-compatible: existing captures retain the Playwright recorder behavior. High mode requires ffmpeg available on PATH, via FFMPEG_PATH, or in the Playwright browser cache. It produces VP9 WebM and captures at 10 fps.

Evidence

  • Verified command npm ci && npm run build && npm run test:browser-video-capture: status=succeeded; candidate commit 70085e33ad7c3d1e4c7e93ccdec526644421115d, tree 7c5b520ea581dbb4c1ba9ec9a6572e75befb360b, fingerprint 6dddb0c9d676d3e15a85054dd1a1ff80592f6e20c761726ef7978d6a326dfc07; durable gate gate-1
  • Base unchanged since verification: main remains at 0faffbf.
  • CI expected: Homeboy CI after push
  • Candidate adoption provenance: the candidate was promoted from the recorded Cook task execution.
  • Cook deterministic verification: 1 gate(s) completed green.
  • Durable run execution: Succeeded
  • Reviewer-resolvable evidence from source_refs[0]: browser-actions: crisp high-DPI video capture (record at device pixels, not 1 Mbps CSS-pixel VP8) #2559
  • Task objective: Goal: an opt-in high-fidelity video capture mode for wordpress.browser-actions that records at device pixels (viewport x deviceScaleFactor) and encodes once at the end at high quality, with no realtime-encoder warm-up, keeping the markers timeline aligned. Start in packages/runtime-playground/src/browser-actions-runner.ts (recordVideo setup ~L224, video adoption ~L564, arg parsing ~L863-903), packages/runtime-core/src/command-registry.ts for the new arg, and tests/browser-actions-video-capture.browser.test.ts. Plain Playwright 1.61.1 recordVideo cannot do this (Chromium screencast is CSS-pixel), so use CDP Page.startScreencast with maxWidth/maxHeight at device pixels or timed screenshots, then ffmpeg (Playwright ships one under ms-playwright/ffmpeg-*; fall back to PATH).
  • Verified candidate scope: 3 changed file(s): packages/runtime-core/src/command-registry.ts, packages/runtime-playground/src/browser-actions-runner.ts, tests/browser-actions-video-capture.browser.test.ts.
  • Verified finalization base: main at 0faffbf
  • deterministic gate passed: sh -lc npm ci && npm run build && npm run test:browser-video-capture (Passed)

AI assistance

  • AI assistance: Yes
  • Tool(s): Homeboy (opencode)
  • Model: openai/gpt-6-luna
  • Used for: [operator-only reference omitted]

@chubes4

chubes4 commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

Finalized by Extra Chill Bot: CI's browser job failed with spawn ffmpeg ENOENT because the high-quality encoder only tried system ffmpeg. Now it resolves FFMPEG_PATH → Playwright's bundled ffmpeg (ms-playwright/ffmpeg-*) → PATH. Frames are piped as JPEG over pipe:0 and encoded as VP8 at high quality, which the minimal bundled build supports (it has no VP9/H.264, PNG decoder, or - protocol alias). The test decodes with the same resolved ffmpeg and asserts a 1080x1920 VP8 stream. All 5 video tests pass locally with and without a system ffmpeg on PATH; npm run build is clean.

homeboy-ci Bot added 2 commits October 3, 2026 17:11
…dled ffmpeg

The CI runner has no system ffmpeg, so video-quality=high failed with spawn ffmpeg ENOENT.
Resolve FFMPEG_PATH, then Playwright's bundled ffmpeg, then PATH (checked for executability).
Pipe JPEG frames as MJPEG over pipe:0 and encode VP8, the formats the minimal bundled build supports.
@chubes4
chubes4 force-pushed the fix/issue-2559-wp-codebox branch from 368df63 to 65fd27a Compare October 3, 2026 17:12
@chubes4
chubes4 merged commit 3e01210 into main Oct 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant