Let keyboard users reach the native controls of audio and video players in an activity

[Eugene (OEvgeny)]

Fixes #5738, fixes #5709

Changelog Entry

Fixed

• Fixed the native controls of audio and video players, such as volume and "more options", not being reachable with TAB and SHIFT + TAB inside an activity: the focus now moves through every control of the player and wraps within the activity; the players of audio, video, and animation cards are now named after the card title, fixes #5738 and #5709, in PR #5891, by @OEvgeny

Description

This is a regression from #5243 (4.19.0). In 4.18.0, TAB moves through the controls of an audio player in an activity. SHIFT + TAB on the player stayed on it, even in 4.18.0. This change fixes both.

After pressing ENTER on an activity with an audio attachment or an audio card, the focus is on the <audio> element. TAB and SHIFT + TAB then kept it there. The player's own controls (play, time scrubber, volume, mute, more options) could not be reached, so mute, playback speed, and download could not be used with the keyboard. On a plain page, Edge moves through all of these controls. The activity focus trap counts the player as one element and cancelled TAB on it. #5243 replaced the focus sentinels of the trap with key handling, and the end sentinel was what used to catch TAB after the player's last control.

The trap now lets the browser move through the player's controls and wraps the focus after the last one. Video attachments (<video controls>) had the same problem and are fixed too.

The players of audio and video cards (and of animation cards without profile), which keyboard users can now reach, had no accessible name: they are now named after the card title, as the players of attachments are named after the attachment name.

#5739 and #5741 (name and role of the play/pause control) should be re-tested after this change. They were reported on the <audio> element itself (read as "BotFramework Test, audio"), which was the only element the trap let the keyboard reach. The player's play button (role button, name "play" or "pause") is now reachable.

Design

• The controls of <audio controls> and <video controls> are in a closed shadow tree. Web Chat cannot list or focus them, and document.activeElement is the media element for all of them. Their keystrokes also do not reach the page: in Edge there is no keydown on document from these controls, not even in the capture phase. So the trap cannot see when TAB leaves the last control.
• TAB on a media element with controls is not cancelled when it is the last element of the trap. A focusable guard at the end of the trap body catches the focus after the player's last control and wraps it to the first element, like the end sentinel before feat: focus trap improvements #5243. Only TAB from content that the trap cannot see reaches the guard: the controls of a media element, or an <iframe> (see follow-ups). TAB on other elements is still handled by the existing key handling.
• SHIFT + TAB on the first element, when the last element is a media element with controls: the trap moves the focus to the guard and does not cancel the key. The browser continues SHIFT + TAB from the guard into the player's last control. In all other cases, the existing wrap is used.
• The guard is not counted as a tabbable element of the trap. It is inside the trap body, so the outer trap of part grouping ignores it, as it ignores the other content of nested traps. It has no aria-hidden, because axe flags focusable content inside aria-hidden. It is an empty <div>, exposed like FocusRedirector (generic, no name).
• ESCAPE and typing on a control inside the player do nothing, because these keystrokes do not reach the page. ESCAPE still works on the <audio> or <video> element itself, one TAB or SHIFT + TAB away.
• The video tests use __tests__/assets/dark-green-video.webm, a uniform dark green video (2 seconds, no audio track), so the video stands out from the black gradient behind the player controls.
• Every test takes a snapshot after ENTER and after every TAB and SHIFT + TAB, through a full cycle in each direction, after waiting for the focused control in the accessibility tree. The ring on a control inside the player is drawn by the browser, as on a plain page; the rings of the play button and the time scrubber of a video are clipped at the edges of the <video> element by the element itself, also on a plain page.
• Possible follow-up: the players of audio and video cards still show the browser's own focus ring when the player itself has the focus (first snapshot of the audio card and video card tests). It is rectangular around the pill-shaped audio player, and black against the dark bottom of a video. Show keyboard focus indicator on all four sides of audio and video attachments #5887 replaced it with the Web Chat indicator for attachments and animation cards only. This change does not touch it.
• The player of an audio or video card is named after the card title, through the existing alt prop (aria-label) that attachments fill with the attachment name. Without a title, the player has no aria-label, as before. The profile of the media is not used for these cards, because bots send it as a hint to tell media apart (the mock bot sends audiocard), not as a label.
• The animation card already named its media after profile; this is unchanged. Without profile, its video player now falls back to the card title. Its GIF image without profile keeps alt="" (decorative), because it is not a control.
• A YouTube or Vimeo video card with a title now uses the title as the name of the <iframe>, instead of "A video clip", the same as a video attachment with a name.
• Possible follow-up: an embedded <iframe> (YouTube or Vimeo video cards) has the same invisible keystrokes. TAB out of it now wraps through the guard, but SHIFT + TAB out of an <iframe> that is the first element still leaves the activity. Not tested.

Specific Changes

• Transcript/FocusTrap.tsx:

  • TAB on a media element with controls is not cancelled.
  • A guard at the end of the body wraps the focus after the player's controls.
  • SHIFT + TAB on the first element continues from the guard when the last element is a media element with controls.

• Added testHelpers.getFocusedAccessibilityNode(), which returns the focused node of the accessibility tree via Chrome DevTools Protocol. Unlike document.activeElement, it also sees the focus inside the player's controls.

• AudioCardContent.tsx and VideoCardContent.tsx: the player is named after the card title.

• AnimationCardContent.tsx: the video player falls back to the card title when the media has no profile.

• Added __tests__/html2/transcript/navigation/focusTrap.audio.html (with a .fluent.html variant), focusTrap.audioCard.html, focusTrap.video.html, focusTrap.videoCard.html, and focusTrap.animationCard.html. They enter the activity with the keyboard, TAB and SHIFT + TAB through the player's controls, and check where the focus is and the name of the player.

  • Each test goes through a full cycle with TAB (from the player through every control, wrapping to the player) and with SHIFT + TAB (wrapping to the last control, then back through every control to the player), with a snapshot of the focus indicator after every key press.

• Added __tests__/assets/dark-green-video.webm for the video tests.

• I have added tests and executed them locally

• I have updated CHANGELOG.md

• I have updated documentation

Review Checklist

This section is for contributors to review your work.

• Accessibility reviewed (tab order, content readability, alt text, color contrast)
• Browser and platform compatibilities reviewed
• CSS styles reviewed (minimal rules, no z-index)
• Documents reviewed (docs, samples, live demo)
• Internationalization reviewed (strings, unit formatting)
• package.json and package-lock.json reviewed
• Security reviewed (no data URIs, check for nonce leak)
• Tests reviewed (coverage, legitimacy)

[Eugene (OEvgeny)]

TODO: let's add snapshots and use dark-green video.

[Eugene (OEvgeny)]

Done: the video tests now use a uniform dark green video (dark-green-video.webm, replacing the dark gray one), and the focus-trap tests snapshot the focus indicator on the players' native controls (audio in the default and Fluent themes, audio card, video, and video card; the animation card is skipped because #5887 changes its layout).

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.