feat(studio)!: audio scenes replace mixes by kixelated · Pull Request #77 · moq-dev/studio

kixelated · 2026-11-07T05:00:08Z

Finishes quest/m0/scenes/audio.md, part of the scenes epic. It lands audio scenes and deletes the quest. Pan, the one piece left, moves out of the epic to quest/m0/audio-pan.md, which waits on a moq pin.

Summary

Audio scenes replace mixes. A collection's [[audio_scenes]] each hold sounds (sounds) and other audio scenes (scenes). moq-audio's bus has no nesting, so the studio flattens them: each audio scene's bus Mix routes the union of sounds it reaches, each sound once. Validation refuses:
- an audio scene that holds itself, even through other scenes;
- an unknown sound or scene;
- a source with no audio;
- anything other than exactly one program and at most one recorded.
Pickers. Each published scene picks the audio scene it carries (audio = "..." on the scene, defaulting to the one on program). Each recording picks one too (Output panel, recorded = true). Switching either one, or editing an audio scene, follows a mix::Pick on a block boundary, so audio is never dropped or repeated.
Published scenes encode their own audio. A published scene's broadcast has its own Opus track, encoded from its picked audio scene. moq-dev/studio#71 instead listed the program's audio by reference. Switching a published scene's audio scene therefore never changes the track its viewers play.
UI.
- The Audio panel is now a mixer. Each sound's strip has gain, mute, delay, a meter, and a send to each audio scene. Each audio scene has a strip with sends to the others, and a send that would loop is disabled.
- The Scenes panel lists audio scenes, tagged audio, after the picture scenes. You can add, rename, reorder, and remove them there, and each published scene gets an audio-scene picker.
Health and Output.
- A sound stalls or stops the health check when the audio scene on program, or any published scene's audio scene, plays it.
- The problem names the broadcasts that play it, such as "scene tall plays silence for it", rather than always the program.
- A browser source uploaded from memory rather than copied on the GPU (moq-dev/studio#76) names the broadcasts that show it, such as "Source chat, which scene tall shows, is uploaded from memory...".
- Output names the audio scene each published broadcast carries and shows its audio errors.
Unchanged. Guest mix-minus return feeds still follow the audio scene on program, minus the guest. A source's sync delay is still its strip's delay, applied in every audio scene that holds it.
Docs.
- The README's sections on audio scenes, panels, published scenes, guests, the scene file, recording, and health are updated.
- quest/m2/publish-independently.md now refers to the audio scene on program.
Quests.
- quest/m0/scenes/audio.md is deleted. The scenes epic keeps only its end-to-end test and is now sized [M].
- New quest/m0/audio-pan.md [S] adds the Audio panel's pan and [audio.<source>] pan. It requires the new quest/m0/pin-audio-pan.md [XS], which waits on moq's Pan a bus source, planned in moq-dev/moq#8133.

Breaking changes

Scene file: [mixes] and [audio.<source>].exclude are gone, replaced by [[audio_scenes]] and a scene's audio. A file without [[audio_scenes]] gets one empty Program audio scene, so its sources are silent until they are sent to it.
moq-scene: Mixes became AudioScene, and Collection.mixes became Collection.audio_scenes. Change::Mixes became Change::AudioScenes. New Draft methods: hear, set_audio_scenes, rename_audio_scene, and remove_audio_scene. New Collection helpers: heard, nests, program_audio, recorded_audio, and published_audio.

Decisions

Settled by the maintainer on 2026-11-07 (comment):

Pan, which is blocked because moq-audio's bus Strip has no pan:
- ✅ Plan a moq-dev/moq quest for pan plus a studio pin quest, and move studio pan out of the scenes epic so it doesn't hold up the epic's end-to-end test.
- Plan the moq quest and the pin, but keep pan in the epic.
- Drop pan.
Strip settings:
- ✅ Keep [audio.<source>] for gain, mute, and delay, dropping only exclude.
- Give each audio scene its own gain and mute per sound.
Published scenes' audio:
- ✅ Each published scene encodes its own picked audio scene.
- Point at the program's audio track.
A scene file with no [[audio_scenes]]:
- ✅ Gets one empty "Program" audio scene.
- Gets a default holding every audio source.
Guests:
- ✅ A joining guest is in no audio scene, and taking them off the air removes their sends.
- A joining guest is sent to the program's audio scene, and taking them off the air mutes them.

Choices made without asking

Inclusion lists, not exclusion lists. Audio scenes are their own [[audio_scenes]] array, with a namespace separate from picture scenes, and they list the sounds and scenes they hold. Anything new (a source added from Sources or Devices, set up, or pasted) is sent to the audio scene on program, so it is heard at once, as before.
The scenes epic is sized [M]. It has no children left, so quest check requires a size. Its remaining work is the end-to-end test that two published broadcasts show and play the same cut at the same moment.
Health names where a sound plays. This addresses the automated review on 5302031. health::Shown became Live. A stopped source carries a Reach that pairs where it is shown with where it is heard, so a problem caused only by a published scene's audio names that scene.

Follow-ups

None beyond the quests above.

Validation

On 91c13b8, which merges origin/main (3ff7dc9, moq-dev/studio#76):

quest gate nix develop --command just ci: passes. That covers lint, the feature matrix, and 265 tests.
quest gate nix develop --command cargo nextest run --locked --workspace --run-ignored only --no-fail-fast -- --skip bench (GPU and audio, Metal): 53/53 pass, including the_program_rests_a_stinger_clip, which checks where a sound is heard.
quest check --base origin/main --branch quest/m0/scenes/audio: 43 documents ok.

Review

Reviewer: Codex, gpt-6.1-sol, medium effort, quest review --base origin/main, session 01a1b47a-13fd-73f0-9510-3736fb30ba94.
Round 1 on 695d6fa found 2 issues, both fixed in 5302031.
Round 2 on 5302031: 0 findings. The automated GitHub review of that head (review) found one P3: a sound that only a published scene plays was reported as a program-audio failure. Fixed in 0e4ff7f.
Round 3 on 0e4ff7f (the Health fix and the quest-tree move of pan): 0 findings. The automated GitHub review of that head (review): no new findings.
Round 4 on 91c13b8 (the hand-resolved merge of origin/main, whose Health note for a CPU-painted browser source now carries a health::Live): 0 findings. This is the design checkpoint; the approach is unchanged, since the merge only threads fix(scene): browser sources under X11 paint only on the X server's GPU #76's new Health fact through this PR's types.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

Quest-Claim: 8de7cb8be4691a69

pullfrog · 2026-11-07T05:00:12Z

^{｜ Review this ➔}

A collection's [[audio_scenes]] each hold sounds and other audio scenes, flattened onto moq-audio's bus. One is on program, one may be recorded, and each published scene picks the audio scene it carries, encoded as its own audio track. The Audio panel becomes a mixer with a send per audio scene. Pan waits on moq's bus strip; the quest keeps that remainder. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A stalled or stopped sound counts when a published scene's audio scene plays it, not only the one on program, and Output names the audio scene each published broadcast carries and shows its audio error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

coderabbitai · 2026-11-07T05:37:36Z

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: CHILL
Plan: Advanced
Run ID: 7f02e03b-3baf-44b7-a22a-d4bc499a252c

Commits

Reviewing files that changed from the base of the PR and between 3ff7dc9 and 91c13b8.

Files selected for processing: 20

README.md
quest/m0/README.md
quest/m0/audio-pan.md
quest/m0/pin-audio-pan.md
quest/m0/scenes/README.md
quest/m0/scenes/audio.md
quest/m2/publish-independently.md
rs/moq-scene/src/clip.rs
rs/moq-scene/src/collection.rs
rs/moq-scene/src/compositor.rs
rs/moq-scene/src/edit.rs
rs/moq-scene/src/lib.rs
rs/moq-scene/src/source.rs
rs/moq-studio/src/engine.rs
rs/moq-studio/src/feed.rs
rs/moq-studio/src/guests.rs
rs/moq-studio/src/health.rs
rs/moq-studio/src/mix.rs
rs/moq-studio/src/setup.rs
rs/moq-studio/src/ui.rs

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

kixelated

Automated review by review (OpenAI)

Reviewed commit: 5302031
Base: 2bbdd5b

One P3 finding: the expanded health check detects published-scene-only sound failures, but its warning still attributes them to program audio. Details inline.

Direction: flattening the validated audio-scene graph into existing bus mixes keeps nesting out of the mixer, deduplicates sounds, and preserves block-boundary output picks. Own audio tracks for published scenes keep their viewer-facing track stable while switching scenes. I found no further actionable defects in model validation, rename/remove/history, paste routing, output switching, recording, or guest mix-minus. The scene-file break is explicit for this unreleased studio; pan remains pending upstream.

Verification: GitHub-only static review of the changes and relevant callers, including pinned moq bus/output semantics. I did not run quest context, builds, tests, the UI, audio playback, or performance measurements; author-reported validation is not independently verified. COMMENT only, not approval.

(Written by review (OpenAI))

kixelated · 2026-11-07T05:39:49Z

+				.into_iter()
+				.chain(published.filter_map(|scene| collection.published_audio(&scene.name)))
+				.collect();
+			let heard = |name: &str| carried.iter().any(|scene| state.sources.plays(scene, name));


[P3] Preserve which broadcast hears a stalled sound

This union now includes sounds carried only by a published scene, but the health payload still carries only picture placement. health::problems (health.rs:945–967) unconditionally describes a Sound stall as “the program plays silence for it,” and an audio-only end as “the program plays no sound for it.” For example, route a microphone only to the audio scene carried by published Tall, leave it out of program audio, and stop its capture: Health reports a program-audio failure even though only Tall is affected. Preserve the hearing destinations through Local and use them for the sound warning (or use destination-neutral wording), with a regression for sound carried solely by a published scene.

Agreed and fixed in 0e4ff7f. A stall now carries the broadcasts that play the stalled medium (health::Live), and a stopped source carries a Reach with where it is shown and where it is heard. So a microphone routed only to Tall's audio scene reads "scene tall plays silence for it". problems_list_worst_first covers the published-only sound, and the_program_rests_a_stinger_clip checks what the engine reports.

(Written by Claude Opus 5.5)

kixelated · 2026-11-07T05:40:27Z

Decisions (maintainer, /quest-complete, 2026-11-07):

Pan, blocked on moq-audio's bus Strip having no pan:
- ✅ Plan a moq-dev/moq quest for pan plus a studio pin, and move studio pan out of the scenes epic so it doesn't hold up the epic's end-to-end test.
- Plan the moq quest and pin, keeping pan in the epic.
- Drop pan.
Strip settings:
- ✅ Keep [audio.<source>] for gain, mute, and delay, dropping only exclude.
- Give each audio scene its own gain and mute per sound.
Published scenes' audio:
- ✅ Each published scene encodes its own picked audio scene.
- Point at the program's audio track.
A scene file with no [[audio_scenes]]:
- ✅ Gets one empty "Program" audio scene.
- Gets a default holding every audio source.
Guests:
- ✅ A joining guest is in no audio scene, and taking them off the air removes their sends.
- A joining guest is sent to the program's audio scene, and taking them off the air mutes them.

(Written by Claude Opus 5.5)

…n leaves the scenes epic A sound stall or end now names the program or published scenes whose audio scene plays it, instead of always the program. Pan moves to its own quest, waiting on a pin of moq-audio's strip pan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

kixelated

Automated review by review (OpenAI)

Reviewed commit: 0e4ff7f
Base: 2bbdd5b

The prior P3 finding is fixed. Engine now retains which broadcasts hear a stalled sound rather than passing picture visibility into its warning. Ended sources carry separate picture/sound reach, so a program picture with published-scene-only audio describes each destination correctly. The regression expectations cover both a tall-only microphone stall and a tall-only ended audio source, plus differing picture/audio destinations. No new actionable findings in this follow-up.

Direction: carrying reach into Health fixes attribution without changing audio routing. Moving pan into its own quest preserves the upstream dependency and explicitly requires an implemented Strip pan before bumping the pin, rather than treating the planning PR as the implementation.

Verification: GitHub-only static comparison against 5302031, including the Health fact collection/formatting, changed regressions, README and quest move. I did not run builds, quest validation, tests, the UI or audio playback, and did not independently reproduce the reported checks. COMMENT only, not approval.

(Written by review (OpenAI))

Resolves the conflict with #76: Health's uploaded note carries a health::Live, so a browser source only a published scene shows names that scene. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

kixelated · 2026-11-07T07:14:27Z

Pick (/quest-complete, maintainer away under /afk, recommended option): ✅ /quest-merge on head 91c13b8. Codex round 4 found nothing, and every decision is settled in the comment above.

(Written by Claude Opus 5.5)

kixelated

Automated review by review (OpenAI)

Reviewed commit: 91c13b8
Base: 3ff7dc9

No new actionable findings in the merge follow-up since 0e4ff7f. After accounting for #76's base changes, the meaningful adaptation carries picture reach into the new CPU-upload warning. Engine uses shown(name), filters out sources with no shown destination, and passes that same Live value to Health; the updated expectation correctly names a browser shown only in scene tall. The earlier audio-warning fix remains intact.

Direction: reusing the existing picture-reach representation keeps this diagnostic consistent with stalled-picture warnings without changing audio routing or rendering.

Verification: GitHub-only static base-aware comparison, merged Engine/Health paths, README and regression expectation. No builds, tests, quest validation, browser/GPU runs or audio checks executed. This is an incremental COMMENT, not approval or a repeat review of the entire implementation.

(Written by review (OpenAI))

chore(quest): claim quest/m0/scenes/audio.md

c2ca17c

Quest-Claim: 8de7cb8be4691a69

kixelated and others added 2 commits November 6, 2026 21:27

kixelated changed the title ~~Audio scenes~~ feat(studio)!: audio scenes replace mixes Nov 7, 2026

kixelated marked this pull request as ready for review November 7, 2026 05:37

kixelated commented Nov 7, 2026

View reviewed changes

kixelated mentioned this pull request Nov 7, 2026

quest: plan a bus strip pan for the studio's Audio panel moq-dev/moq#8133

Merged

kixelated added the quest:active label Nov 7, 2026

kixelated removed the quest:active label Nov 7, 2026

kixelated commented Nov 7, 2026

View reviewed changes

kixelated added the quest:active label Nov 7, 2026

kixelated and others added 2 commits November 6, 2026 23:06

Merge origin/main: a CPU-painted browser source names where it is shown

fa1fda1

Resolves the conflict with #76: Health's uploaded note carries a health::Live, so a browser source only a published scene shows names that scene. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

style(studio): format the merged Health test

91c13b8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

kixelated removed the quest:active label Nov 7, 2026

kixelated enabled auto-merge (squash) November 7, 2026 07:14

kixelated commented Nov 7, 2026

View reviewed changes

kixelated merged commit 167cdca into main Nov 7, 2026
3 checks passed

kixelated deleted the quest/m0/scenes/audio branch November 7, 2026 07:29

This was referenced Nov 7, 2026

fix(scene): the nightly and the browser suite pass on every path #78

Merged

test(studio): the program and a portrait scene cut together #81

Merged

Navigation Menu

Repository navigation

feat(studio)!: audio scenes replace mixes - #77

feat(studio)!: audio scenes replace mixes#77
kixelated merged 6 commits into
mainfrom
quest/m0/scenes/audio

kixelated commented Nov 7, 2026 •

edited

Loading

Uh oh!

pullfrog Bot commented Nov 7, 2026

Uh oh!

coderabbitai Bot commented Nov 7, 2026 •

edited

Loading

Review limit reached

Uh oh!

kixelated left a comment

Uh oh!

kixelated Nov 7, 2026

Uh oh!

kixelated Nov 7, 2026

Uh oh!

kixelated commented Nov 7, 2026

Uh oh!

kixelated left a comment

Uh oh!

kixelated commented Nov 7, 2026

Uh oh!

kixelated left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

kixelated commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Breaking changes

Decisions

Choices made without asking

Follow-ups

Validation

Review

Uh oh!

pullfrog Bot commented Nov 7, 2026

Uh oh!

coderabbitai Bot commented Nov 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

kixelated left a comment

Choose a reason for hiding this comment

Uh oh!

kixelated Nov 7, 2026

Choose a reason for hiding this comment

Uh oh!

kixelated Nov 7, 2026

Choose a reason for hiding this comment

Uh oh!

kixelated commented Nov 7, 2026

Uh oh!

kixelated left a comment

Choose a reason for hiding this comment

Uh oh!

kixelated commented Nov 7, 2026

Uh oh!

kixelated left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

kixelated commented Nov 7, 2026 •

edited

Loading

coderabbitai Bot commented Nov 7, 2026 •

edited

Loading