Skip to content

TT-7870 Phase 1: Audio Player and overhauled Record step - #808

Open
josephmyers wants to merge 7 commits into
developfrom
replace-workflow
Open

josephmyers wants to merge 7 commits into
developfrom
replace-workflow

Conversation

@josephmyers

@josephmyers josephmyers commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator
  • Audio Player now has an explicit, defined interface, usable through a handle. Record tracks the audio player imperatively.
  • Audio edits autosave after 3 seconds.
  • Audio Player and future audio components will use the new AudioManager, which ensures in a central location that only one audio plays at a time.
  • Recording should work for iOS Safari (it may not have been beforehand)
  • New display component for AI Processing, Convert Voice, Microphone Selection.
  • Version Count only counts 1 autosave (to avoid Versions button flickering in some cases)
  • Removed forced .5s delay and step unset on page nav, since this can be handled in a single place (PassageDetailContext) without a delay
  • useMediaUpload has its own compression ability. saving now reads mimeType property and compresses if omitted (or not if WAV is specified)
  • BOLD no longers moves to the next step for the user, but it does still make the step complete once audio is given

@josephmyers
josephmyers force-pushed the replace-workflow branch 4 times, most recently from ee9ca15 to 3f86023 Compare November 4, 2026 19:17
@josephmyers

Copy link
Copy Markdown
Collaborator Author

@gtryus, this is the first half of the Replace workflow development. Please take a look when you can. It introduces a new, leaner audio player, while attempting to preserve all user features. This work touches as little of the existing components as possible, in order to avoid breakages, but my goal for the future is to have each step move over, one at a time, at our discretion.

@josephmyers
josephmyers marked this pull request as ready for review November 4, 2026 19:33
@josephmyers
josephmyers force-pushed the replace-workflow branch 7 times, most recently from 4fff1e0 to e71e775 Compare November 11, 2026 15:13
- Audio Player now has an explicit, defined interface, usable through a handle. Record tracks the audio player imperatively.
- Audio edits autosave after 3 seconds.
- Audio Player and future audio components will use the new AudioManager, which ensures in a central location that only one audio plays at a time.
- Recording should work for iOS Safari (it may not have been beforehand)
- New display component for AI Processing, Convert Voice, Microphone Selection.
- Version Count treats only counts 1 autosave (to avoid Versions button flickering in some cases)
- Removed forced .5s delay and step unset on page nav, since this can be handled in a single place (PassageDetailContext) without a delay
- useMediaUpload has its own compression ability. saving now reads mimeType property and compresses if omitted (or not if WAV is specified)
- BOLD no longers moves to the next step for the user, but it does still make the step complete once audio is given
It would be distracting, and the point of automatic is that it should succeed or fail invisibly

@gtryus gtryus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Hard-coded English that isn't in the XLIFF (CODING_STANDARDS "Localized strings"; localization.md "Source of truth" and "Aria labels: reuse a visible label's string"):

    • In AudioPlayer.tsx: "Select Microphone..." at :1012 (reuse the existing wsAudioPlayer.selectMicrophoneMenu), "Reset Audio" at :1075, every aria-label (e.g. :923 `${label} start recording`), and the label='passage' default.
    • In MicrophoneDialog.tsx: "Select Microphone", "Cancel" and "OK" at :79, :115 and :117 (ts.cancel already exists).
    • In PassageDetailRecord.tsx: the strings at :213 and :258, and aria-label="record".
    • {t.convertVoiceAi}... and {ts.versionHistory}... add a "..." outside the localized string.
  • Modules / Single responsibility: AudioPlayer.tsx is 1,134 lines. The menu/AI/microphone code, the undo stack and the region/marker sync could each move into a hook.

  • One owner per behaviour: several new helpers repeat code that already exists.

    New code Already exists as
    utils/formatTime.ts control/formatTime.ts and pad2
    audioBufferToWav / writeString encodeWAV in utils/wav.ts
    toBase64 the one in utils/axios.tsx
    Device listing in MicrophoneDialog WSAudioPlayer's enumeration and fallbackInputDeviceId
    File-extension parsing at useMediaUpload.ts:295 extensionFromAudioContentType
    alert.main '#D32F2F' error.main and RecordButton's redColor
  • Errors are reported, not swallowed:

    • AudioPlayer :824 has a bare catch {} on cut, and :385 drops the getMic rejection.
    • usePassageAudioSave.ts:112 has .catch(() => {}).
    • MicrophoneDialog ignores enumerateDevices errors.
  • Floating promises:

    • applyAi, handleInsertSilenceClick and startRecording are passed straight to onClick.
    • memory.update and setStepComplete are not awaited in handleAudioChange.
    • useRecorder.ts:40 assigns an async function to stop: () => void, so its errors are lost.
  • Narrow types:

    • AudioPlayer :873 uses internalAudio! with no explanation.
    • err as Error is used in autosaveMedia and useMediaUpload :299, even though toError is already imported.
    • Also unexplained: as Run, as ShadowRoot, waveColor as string.
  • Derived types: {start:number;end:number} is written out inline more than 15 times even though the new TimeRange type exists. {start,end,color} is written out 4 times.

  • Effects:

    • The [audioSource] and [stickySelection] effects reset state from props, which the standard says should be done with key.
    • The MicrophoneDialog enumerateDevices effect has no guard against a stale result.
  • Flat JSX: AudioPlayer has a nested record/stop/play ternary.

  • Named constants: raw values are scattered through the code:

    • colours: '#dddddd' (twice), '#333', '#303030aa', '#ff660091', and others
    • sizes: cursorWidth: 4 (twice), '12px !important' (4 times)
    • 'audio/wav' in several files
  • Comment to remove: audioUtils.ts:1 is a commented-out import.

  • Pure helpers get unit tests: there are none for these:

    • audioUtils (clampSelectionToHighlights, mapPreviewTimeToOriginalTime, splice/clip/insert)
    • formatTime's new parsers
    • parseRecordSaveAsWav
    • clearPendingUploads and pendingUploadIds
    • waveformRenderer, useRecorder, useAudioConversion

Judgement calls (code smells)

  • Speculative Generality: these have no caller outside tests: composeReplacements, mapPreviewTimeToOriginalTime, toBase64, fromBase64, formatClock, parseTimeRange, applyRangeToText, formatShortDateTime, useIsAudioPlaying.
  • Duplicated Code:
    • The decode → OfflineAudioContext → render → WAV sequence appears 4 times in audioUtils.
    • setInternalAudio + onAudioChangeRef + once('decode') appears 5 times in AudioPlayer.
    • The loop that removes non-marker regions appears 4 times.
  • Data Clumps: (keyMap, plan, passageId) travel together to 6 call sites of clearPendingUploads and pendingUploadIds.
  • Mysterious Name: showTrash and handleTrashClick actually do "Reset Audio".
  • Hook call inside JSX: cancelled={useRef(false)} at PassageDetailRecord.tsx:431.

(a) Missing or partial

  • "AudioManager… ensures in a central location that only one audio plays at a time." Only AudioPlayer.tsx uses it (:637, :710, :790). WSAudioPlayer, MediaPlayer, LimitedMediaPlayer, HiddenPlayer, usePassagePlayback and audioPlayback.ts can still play at the same time as it. Also, the handle's play() (:288, :433) calls ws.play() directly, so playing through the handle doesn't stop other audio either. If only AudioPlayer is meant to use it for now, that's acceptable, but it isn't "central" yet.

  • "Recording should work for iOS Safari." Nothing in the code shows this:

    • The diff has no Safari-specific handling.
    • useRecorder.ts:64 falls back to new MediaRecorder(stream) without a mimeType.
    • The microphone opens when the component mounts, not on a user tap (AudioPlayer :380–386).
  • The changes to prevent deleting the only version (036b967) and to fire onVersionApplied when the active version is deleted (3e56f9b) aren't in the spec. Both are reasonable, but they aren't listed in the commit message.

  • Smaller unlisted changes: the begin(voice) signature change in SelectVoice, the new theme colours extraLight and alert, and px: 1 on the chooser.

(c) Implemented, but looks wrong

  • "Saving now reads mimeType property and compresses if omitted." I checked this myself. run() now compresses for every caller of useMediaUpload (useMediaUpload.ts:414-417), not just Record:
    • useRecordSave.ts:83 never passes mimeType, and neither does MediaRecord's upload prop. So a WAV save the user chose, and the WAV fallback after a failed compression (:178), both get re-encoded to ogg. Takes already converted to webm get converted a second time.
    • Uploader, TitleTabs and ResourceUploadStep will now re-encode files users upload (mp3, wav, possibly video into audio-only ogg). Files that aren't audio start the converter, which fails.
    • The skip check file.type === type (:292) is an exact match, so audio/ogg; codecs=opus is re-encoded too.
  • "Audio edits autosave after 3 seconds." The 3 s debounce works and is cancelled on unmount or navigation. But a manual save() (usePassageAudioSave :100) doesn't wait for an autosave that's already running. If the manual save finishes first, the autosave's removeOldAutosaves (:79) leaves the older autosave as the newest version, on top of your manual save. The test at :310 only covers the opposite order.

@gtryus gtryus changed the title Phase 1: Audio Player and overhauled Record step TT-7870 Phase 1: Audio Player and overhauled Record step Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants