From dd303b5b6546fe5b1d38511dd23270278fa36709 Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Tue, 17 Jun 2025 10:43:30 -0700 Subject: [PATCH 1/8] Apply privacy and security mitigations used by the Web Translation API to the Web Speech API --- index.bs | 110 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) diff --git a/index.bs b/index.bs index 0638cba..1a823b5 100644 --- a/index.bs +++ b/index.bs @@ -136,6 +136,116 @@ This does not preclude adding support for this as a future API enhancement, and The above recommendations are intended to reduce this risk of such attacks. +

On-Device Model Privacy Considerations

+This subsection, and the "On-Device Model Security Considerations" subsection below, unlike many "privacy considerations" sections which only summarize and restate considerations that are already normatively specified elsewhere in the document, contain some normative requirements that are not present elsewhere, and add more detail to the normative requirements present elsewhere. The novel normative requirements are called out using strong emphasis. + +

Language Pack Availability

+ +For on-device speech recognition, the exact download status of language packs can present a fingerprinting vector. How many bits this vector provides depends on the options provided to {{SpeechRecognition/available()}} or {{SpeechRecognition/install()}}, and how they influence the download (e.g., if different language packs have different availability statuses). + +

Download Masking

+ +One mitigation is for the user agent to mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} even if the actual download status is {{"available"}} or {{"downloading"}}. + +Because implementation strategies differ and other mitigations (like permission prompts for {{SpeechRecognition/install()}}) are available, a specific masking scheme is not mandated. For APIs where the user agent believes such masking is necessary, a suggested heuristic is to mask by default, subject to a masking state that is established for each (API, options, [=storage key=]) tuple. This state can be set to "unmasked" once a web page in a given [=storage key=] calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to `true` (indicating the language pack is ready). Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-time friction), this ensures that web pages only get access to the true download status after taking a more costly and less-repeatable action. + +Implementations which use such a [=storage key=]-based masking scheme must ensure that the masking state is reset when other storage for that origin is reset. + +
Installation-time friction
+ +The mitigation described in Download Masking works against attempts to silently fingerprint using {{SpeechRecognition/available()}}. The specification also contains requirements to prevent {{SpeechRecognition/install()}} from being easily used for fingerprinting, by introducing friction: + +* The {{SpeechRecognition/install()}} method both requires and consumes [=user activation=], when it would initiate a download. +* The {{SpeechRecognition/install()}} method allows the user agent to prompt the user for permission, or to implicitly reject download attempts based on previous signals (such as an observed pattern of abuse). +* Access to {{SpeechRecognition/install()}} and {{SpeechRecognition/available()}} is gated on an per-API [=policy-controlled feature=], which means that only top-level origins and their delegates can use the API. + +Additionally, initiating the download process via {{SpeechRecognition/install()}} is more or less a one-time operation for a given language. The availability status will only transition from {{"downloadable"}} to {{"downloading"}} to {{"available"}} via these guarded installation operations. That is, while {{SpeechRecognition/install()}} can be used to read some of these fingerprinting bits (by observing the resolution of its promise and subsequent calls to {{SpeechRecognition/available()}}), doing so will effectively "destroy" those bits by changing the state. + +(For details on cases where downloading might happen more than once, and how privacy and security are preserved in those cases, see Download Cancelation, Download Eviction, and Disk Space for Language Packs.) + +
Download Cancelation
+ +An important part of making the download status a less-useful fingerprinting vector is to ensure that the website cannot toggle the availability state back and forth by starting and then effectively canceling downloads. The Web Speech API's {{SpeechRecognition/install()}} method returns a promise and does not take an {{AbortSignal}} to cancel the download itself ({{SpeechRecognition/abort()}} is for an active recognition session). + +Once a download is initiated by {{SpeechRecognition/install()}}, the user agent should preserve the download progress. User agents should not cancel an ongoing language pack download in response to page-controlled actions (e.g., navigation, page unload) that could be used to manipulate the download state for fingerprinting. If a page navigates away, the download should ideally continue in the background, or at least its progress should be saved. The goal is to prevent the site from easily reverting a language pack's state from {{"downloading"}} back to {{"downloadable"}}. + +Note that canceling downloads in response to explicit, out-of-band user-controlled actions (e.g., via browser UI) is not problematic from this perspective. + +
Download Eviction
+ +Another ingredient in ensuring that websites cannot toggle the availability state back and forth is to ensure that user agents don't use a quota-based eviction system for downloaded language packs that web pages can indirectly control. For example, if a user agent evicted less-recently-used language packs when new ones are installed, a web page could trigger such evictions to toggle the state of a target language pack. + +To avoid this, user agents should not implement systems which allow web pages to control the eviction of downloaded language packs, including via indirect triggers such as further subsequent downloads. One way to fulfill this requirement is to never evict downloaded material in response to web page-initiated storage pressure, instead refusing to download new material (e.g., {{SpeechRecognition/install()}} resolving to `false`) if doing so would cause storage pressure. + +Evicting downloads in response to user-controlled actions (e.g., via a browser settings UI) is not problematic. + +
Alternate Options
+ +While some of the above requirements are specified using "must" language, most are "should." This is because implementations might use different strategies to preserve user privacy, especially for APIs with smaller models or language packs. + +The simplest is to treat language pack downloads like other stored resources, partitioning them by the downloading page's [=storage key=]. This leverages existing web origin model privacy protections. The downside is potentially redundant downloads across sites, using more user bandwidth and disk space. + +A variant is to re-download for new [=storage keys=] but re-use on-disk storage if the pack is already there, saving disk space but still using time/bandwidth. + +User agents could also attempt to fake a download for new [=storage keys=] if the language pack is already present, by waiting a similar amount of time as the real download originally took. This saves bandwidth and disk space but is less private due to network side channels (e.g., a page observing no change in network throughput). Such a scheme needs caution, as the first site initiating the download could try to inflate this time. Nevertheless, faking download times might be useful, combined with other mitigations. + +

Sensitive Language Information

+ +Even if fingerprinting risks from availability status are mitigated, knowing a user has downloaded a specific language pack (e.g., for a minority language) can be sensitive. + +For this reason, on top of installation-time friction, user agents may artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. This provides plausible deniability. If {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. + +Such fake delays are not foolproof but offer some privacy benefit, especially when combined with other mitigations like prompts. + +

Model Version

+ +The specific version or behavior of an on-device speech recognition model can also be a fingerprinting vector. These APIs do not expose model versions directly. + +The best way to prevent the model version from becoming a fingerprinting vector is to tie it to the user agent's version, such that the model's version (and behavior) only updates alongside already-exposed information (like the User-Agent string). User agents should limit the number of possible model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}}. This might involve not providing model updates to older user agent versions or ignoring already-downloaded models below a minimum version threshold after a user agent update (instead, {{SpeechRecognition/available()}} might report {{"downloadable"}} for a newer version). + +There's a tradeoff: aggressively locking new UA versions to new model versions can increase transitions between {{"available"}} and {{"downloadable"}}. This can be mitigated by allowing older models with newer UAs while a new model downloads, keeping the status {{"available"}} but briefly allowing identification of users with older-model/newer-UA combinations. + +

User Input and Speech Data

+ +Speech data is inherently sensitive. +Implementations must not train or fine-tune on-device speech recognition models on user speech input obtained through this API, or otherwise store user speech input in a way that models can consult in the future (e.g., for personalization beyond the current session or across origins). + +Using user speech input in such a way would be a significant privacy leak, potentially exposing user information or information derived from interactions with one site to another. + +This reinforces the existing requirement: "To mitigate the risk of fingerprinting, user agents MUST NOT personalize speech recognition when performing speech recognition on a {{MediaStreamTrack}}." The considerations here apply broadly to any speech processed by on-device models via this API. + +

Cloud-based vs. On-Device Implementations

+ +The Web Speech API can support both server-based (cloud) and client-based/embedded (on-device) recognition and synthesis. The {{SpeechRecognition/processLocally}} attribute allows developers to indicate a preference or requirement for on-device processing. + +When `processLocally` is `false` (the default), user speech data may be sent to a remote server for processing. Web developers should be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents should also be transparent with users about where speech processing occurs. + +When `processLocally` is `true`, the considerations in this "On-Device Model Privacy Considerations" section are paramount. + +

On-Device Model Security Considerations

+ +

Disk Space for Language Packs

+ +Downloading language packs for on-device speech recognition via {{SpeechRecognition/install()}} could use significant amounts of the user's disk space. + +In the event of storage pressure, user agents should balance the utility of these APIs with the disk space they take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents may involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. + +If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} as true) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing an {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}. + +

Runtime Shared Resources

+ +On-device speech recognition can consume significant runtime resources like CPU, memory, and potentially specialized hardware accelerators. + +User agents should ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability. For example, it should not be possible for a background tab to monopolize speech processing resources, preventing a foreground tab from using them. + +This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. If necessary to prevent resource exhaustion or instability, the user agent may cause speech recognition operations to fail (e.g., by firing an {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}). + +

OS-Provided Models

+ +One implementation strategy for on-device speech recognition is to delegate to models or capabilities provided by the underlying operating system. This can offer benefits like a consistent user experience and efficient resource usage. + +However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents must still ensure that all privacy and security requirements of this specification are met when using OS-provided models. This includes the requirements in User Input and Speech Data (preventing training on user data) and Runtime Shared Resources (ensuring fair and stable resource sharing). +

API Description

This section is normative.

From 97b6c2f08ea0a3f5b8e57e2f64f5f1a5795f347f Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Tue, 17 Jun 2025 10:43:30 -0700 Subject: [PATCH 2/8] Apply privacy and security mitigations used by the Web Translation API to the Web Speech API --- index.bs | 3 +++ 1 file changed, 3 insertions(+) diff --git a/index.bs b/index.bs index 1a823b5..31d7c64 100644 --- a/index.bs +++ b/index.bs @@ -45,6 +45,9 @@ Complain about:accidental-2119 yes, missing-example-ids yes } } +
+spec:storage; type:dfn; text:storage key
+

Introduction

From 99dd47aaae594ecb7e9b45901ca1f44acfa3ccd7 Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Tue, 17 Jun 2025 10:43:30 -0700 Subject: [PATCH 3/8] Apply privacy and security mitigations used by the Web Translation API to the Web Speech API --- index.bs | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/index.bs b/index.bs index 31d7c64..5f96d8c 100644 --- a/index.bs +++ b/index.bs @@ -148,11 +148,13 @@ For on-device speech recognition, the exact download status of language packs ca

Download Masking

-One mitigation is for the user agent to mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} even if the actual download status is {{"available"}} or {{"downloading"}}. +To prevent cross-origin fingerprinting and site collusion, user agents must partition language pack availability by [=storage key=]. User agents must mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device. -Because implementation strategies differ and other mitigations (like permission prompts for {{SpeechRecognition/install()}}) are available, a specific masking scheme is not mandated. For APIs where the user agent believes such masking is necessary, a suggested heuristic is to mask by default, subject to a masking state that is established for each (API, options, [=storage key=]) tuple. This state can be set to "unmasked" once a web page in a given [=storage key=] calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to `true` (indicating the language pack is ready). Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-time friction), this ensures that web pages only get access to the true download status after taking a more costly and less-repeatable action. +This masking state is established for each (API, options, [=storage key=]) tuple. The user agent must only set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to true (indicating the language pack is ready). -Implementations which use such a [=storage key=]-based masking scheme must ensure that the masking state is reset when other storage for that origin is reset. +Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-time friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as triggering a user permission prompt. + +Implementations must ensure that this [=storage key=]-based masking state is cleared whenever other storage for that origin is reset by the user.
Installation-time friction
@@ -168,11 +170,11 @@ Additionally, initiating the download process via {{SpeechRecognition/install()}
Download Cancelation
-An important part of making the download status a less-useful fingerprinting vector is to ensure that the website cannot toggle the availability state back and forth by starting and then effectively canceling downloads. The Web Speech API's {{SpeechRecognition/install()}} method returns a promise and does not take an {{AbortSignal}} to cancel the download itself ({{SpeechRecognition/abort()}} is for an active recognition session). +Because the availability of a language pack is strictly partitioned by [=storage key=] (see Download Masking), a web page can only observe the download state within its own partition. -Once a download is initiated by {{SpeechRecognition/install()}}, the user agent should preserve the download progress. User agents should not cancel an ongoing language pack download in response to page-controlled actions (e.g., navigation, page unload) that could be used to manipulate the download state for fingerprinting. If a page navigates away, the download should ideally continue in the background, or at least its progress should be saved. The goal is to prevent the site from easily reverting a language pack's state from {{"downloading"}} back to {{"downloadable"}}. +Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed), the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector. -Note that canceling downloads in response to explicit, out-of-band user-controlled actions (e.g., via browser UI) is not problematic from this perspective. +User agents may cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies.
Download Eviction
From 4d5f9da057892b3e466c29430e4313a36a1cdf8b Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Wed, 2 Sep 2026 18:33:54 +0000 Subject: [PATCH 4/8] Address review feedback on privacy mitigations - Fix formatting for `true` code span - Fix typo: 'an per-API' -> 'a per-API' - Remove redundant Alternate Options subsection --- index.bs | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) diff --git a/index.bs b/index.bs index 5f96d8c..6dd97bf 100644 --- a/index.bs +++ b/index.bs @@ -150,7 +150,7 @@ For on-device speech recognition, the exact download status of language packs ca To prevent cross-origin fingerprinting and site collusion, user agents must partition language pack availability by [=storage key=]. User agents must mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device. -This masking state is established for each (API, options, [=storage key=]) tuple. The user agent must only set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to true (indicating the language pack is ready). +This masking state is established for each (API, options, [=storage key=]) tuple. The user agent must only set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to `true` (indicating the language pack is ready). Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-time friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as triggering a user permission prompt. @@ -162,7 +162,7 @@ The mitigation described in user agents should not implement systems which allow web Evicting downloads in response to user-controlled actions (e.g., via a browser settings UI) is not problematic. -
Alternate Options
- -While some of the above requirements are specified using "must" language, most are "should." This is because implementations might use different strategies to preserve user privacy, especially for APIs with smaller models or language packs. - -The simplest is to treat language pack downloads like other stored resources, partitioning them by the downloading page's [=storage key=]. This leverages existing web origin model privacy protections. The downside is potentially redundant downloads across sites, using more user bandwidth and disk space. - -A variant is to re-download for new [=storage keys=] but re-use on-disk storage if the pack is already there, saving disk space but still using time/bandwidth. - -User agents could also attempt to fake a download for new [=storage keys=] if the language pack is already present, by waiting a similar amount of time as the real download originally took. This saves bandwidth and disk space but is less private due to network side channels (e.g., a page observing no change in network throughput). Such a scheme needs caution, as the first site initiating the download could try to inflate this time. Nevertheless, faking download times might be useful, combined with other mitigations. -

Sensitive Language Information

Even if fingerprinting risks from availability status are mitigated, knowing a user has downloaded a specific language pack (e.g., for a minority language) can be sensitive. From 0822a5b89317168d2f22c6e3372879f2875fceac Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Thu, 8 Oct 2026 17:52:23 +0000 Subject: [PATCH 5/8] Address review feedback from hoch@ on privacy mitigations Review comments addressed: - install(): add the transient user activation requirement to the algorithm so it matches the privacy section's claim. The check is keyed off the (masked) availability status so that whether install() requires activation cannot itself leak the true download status. Document the NotAllowedError rejection and link the privacy bullet to the new step. - Download Masking: drop the vestigial "API" component from the masking state tuple. - Installation-time friction: name the "on-device-speech-recognition" policy-controlled feature directly instead of "per-API". - Disk Space / Runtime Shared Resources: "set to `true`" and "a SpeechRecognitionErrorEvent". - OS-Provided Models: reword to match the Writing Assistance source text ("still need to ensure ... even if the user agent has less control"), making clear this restates requirements defined elsewhere rather than adding an unmarked normative one. Editorial cleanups of the same classes found while reviewing: - Remove remaining vestigial multi-API wording ("these APIs") and the "UA" abbreviation; spell out "user agent". - Mark the user-agent transparency "should" in the cloud section per this section's convention; de-2119 developer guidance. - Nest Download Masking under Language Pack Availability so that masking/friction/cancelation/eviction are siblings, as in the source. - Title-case the "Installation-Time Friction" heading; fix a subject shift in the unmasking sentence; strip trailing whitespace. Also merges upstream/main (needed because #200 restructured install()). --- index.bs | 38 +++++++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/index.bs b/index.bs index 2ea2e6b..f065888 100644 --- a/index.bs +++ b/index.bs @@ -146,13 +146,13 @@ This does not preclude adding support for this as a future API enhancement, and For on-device speech recognition, the exact download status of language packs can present a fingerprinting vector. How many bits this vector provides depends on the options provided to {{SpeechRecognition/available()}} or {{SpeechRecognition/install()}}, and how they influence the download (e.g., if different language packs have different availability statuses). -

Download Masking

+
Download Masking
To prevent cross-origin fingerprinting and site collusion, user agents must partition language pack availability by [=storage key=]. User agents must mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device. -This masking state is established for each (API, options, [=storage key=]) tuple. The user agent must only set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to `true` (indicating the language pack is ready). +This masking state is established for each (options, [=storage key=]) tuple. The user agent must only set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully starts a download or resolves to `true` (indicating the language pack is ready). -Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-time friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as triggering a user permission prompt. +Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-Time Friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as triggering a user permission prompt. Implementations must ensure that this [=storage key=]-based masking state is cleared whenever other storage for that origin is reset by the user. @@ -160,9 +160,9 @@ Implementations must ensure that this [=storage key=]-based mas The mitigation described in Download Masking works against attempts to silently fingerprint using {{SpeechRecognition/available()}}. The specification also contains requirements to prevent {{SpeechRecognition/install()}} from being easily used for fingerprinting, by introducing friction: -* The {{SpeechRecognition/install()}} method both requires and consumes [=user activation=], when it would initiate a download. +* The {{SpeechRecognition/install()}} method both requires and consumes [=transient activation=] when, as far as the calling web page can observe, it would initiate a download (see this step of the {{SpeechRecognition/install()}} algorithm). * The {{SpeechRecognition/install()}} method allows the user agent to prompt the user for permission, or to implicitly reject download attempts based on previous signals (such as an observed pattern of abuse). -* Access to {{SpeechRecognition/install()}} and {{SpeechRecognition/available()}} is gated on a per-API [=policy-controlled feature=], which means that only top-level origins and their delegates can use the API. +* Access to {{SpeechRecognition/install()}} and {{SpeechRecognition/available()}} is gated behind the [=policy-controlled feature=] "on-device-speech-recognition", which means that only top-level origins and their delegates can use these methods. Additionally, initiating the download process via {{SpeechRecognition/install()}} is more or less a one-time operation for a given language. The availability status will only transition from {{"downloadable"}} to {{"downloading"}} to {{"available"}} via these guarded installation operations. That is, while {{SpeechRecognition/install()}} can be used to read some of these fingerprinting bits (by observing the resolution of its promise and subsequent calls to {{SpeechRecognition/available()}}), doing so will effectively "destroy" those bits by changing the state. @@ -170,9 +170,9 @@ Additionally, initiating the download process via {{SpeechRecognition/install()}
Download Cancelation
-Because the availability of a language pack is strictly partitioned by [=storage key=] (see Download Masking), a web page can only observe the download state within its own partition. +Because the availability of a language pack is strictly partitioned by [=storage key=] (see Download Masking), a web page can only observe the download state within its own partition. -Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed), the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector. +Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed), the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector. User agents may cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies. @@ -194,11 +194,11 @@ Such fake delays are not foolproof but offer some privacy benefit, especially wh

Model Version

-The specific version or behavior of an on-device speech recognition model can also be a fingerprinting vector. These APIs do not expose model versions directly. +The specific version or behavior of an on-device speech recognition model can also be a fingerprinting vector. This API does not expose model versions directly. The best way to prevent the model version from becoming a fingerprinting vector is to tie it to the user agent's version, such that the model's version (and behavior) only updates alongside already-exposed information (like the User-Agent string). User agents should limit the number of possible model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}}. This might involve not providing model updates to older user agent versions or ignoring already-downloaded models below a minimum version threshold after a user agent update (instead, {{SpeechRecognition/available()}} might report {{"downloadable"}} for a newer version). -There's a tradeoff: aggressively locking new UA versions to new model versions can increase transitions between {{"available"}} and {{"downloadable"}}. This can be mitigated by allowing older models with newer UAs while a new model downloads, keeping the status {{"available"}} but briefly allowing identification of users with older-model/newer-UA combinations. +There is a tradeoff: aggressively locking new user agent versions to new model versions can increase transitions between {{"available"}} and {{"downloadable"}}. This can be mitigated by allowing older models with newer user agent versions while a new model downloads, keeping the status {{"available"}} but briefly allowing identification of users with older-model/newer-user-agent combinations.

User Input and Speech Data

@@ -213,7 +213,7 @@ This reinforces the existing requirement: "To mitigate the risk of fingerprintin The Web Speech API can support both server-based (cloud) and client-based/embedded (on-device) recognition and synthesis. The {{SpeechRecognition/processLocally}} attribute allows developers to indicate a preference or requirement for on-device processing. -When `processLocally` is `false` (the default), user speech data may be sent to a remote server for processing. Web developers should be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents should also be transparent with users about where speech processing occurs. +When `processLocally` is `false` (the default), user speech data may be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents should be transparent with users about where speech processing occurs. When `processLocally` is `true`, the considerations in this "On-Device Model Privacy Considerations" section are paramount. @@ -223,9 +223,9 @@ When `processLocally` is `true`, the considerations in this "On-Device Model Pri Downloading language packs for on-device speech recognition via {{SpeechRecognition/install()}} could use significant amounts of the user's disk space. -In the event of storage pressure, user agents should balance the utility of these APIs with the disk space they take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents may involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. +In the event of storage pressure, user agents should balance the utility of on-device speech recognition with the disk space its language packs take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents may involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. -If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} as true) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing an {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}. +If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}.

Runtime Shared Resources

@@ -233,13 +233,13 @@ On-device speech recognition can consume significant runtime resources like CPU, User agents should ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability. For example, it should not be possible for a background tab to monopolize speech processing resources, preventing a foreground tab from using them. -This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. If necessary to prevent resource exhaustion or instability, the user agent may cause speech recognition operations to fail (e.g., by firing an {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}). +This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. If necessary to prevent resource exhaustion or instability, the user agent may cause speech recognition operations to fail (e.g., by firing a {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}).

OS-Provided Models

One implementation strategy for on-device speech recognition is to delegate to models or capabilities provided by the underlying operating system. This can offer benefits like a consistent user experience and efficient resource usage. -However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents must still ensure that all privacy and security requirements of this specification are met when using OS-provided models. This includes the requirements in User Input and Speech Data (preventing training on user data) and Runtime Shared Resources (ensuring fair and stable resource sharing). +However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents still need to ensure that all privacy and security requirements of this specification are met when using OS-provided models, even if the user agent has less control over the model's behavior. Particularly notable requirements to watch out for are those in User Input and Speech Data (preventing training on user data) and Runtime Shared Resources (ensuring fair and stable resource sharing).

API Description

@@ -521,6 +521,7 @@ See [=default allowlist/'self'=]. When invoked, run these steps: @@ -529,6 +530,13 @@ See If {{SpeechRecognition/available()}} would resolve to {{"downloadable"}} for options (evaluated as if {{SpeechRecognitionOptions/processLocally}} of options were true), that is, if, as far as the calling web page can observe, this call would initiate a download, then: + 1. If the [=current settings object=]'s [=relevant global object=] does not have [=transient activation=], reject promise with a {{NotAllowedError}}, abort these steps, returning promise. + 1. [=Consume user activation=] given the [=current settings object=]'s [=relevant global object=]. + +

+ Note: Because {{SpeechRecognition/available()}} is subject to download masking, this check is based on the masked availability status rather than on the true download status. As a result, whether {{SpeechRecognition/install()}} requires [=transient activation=] does not reveal the true download status to a web page for which it is still masked. +

1. [=In parallel=], run the following steps: 1. For each lang in {{SpeechRecognitionOptions/langs}} of options, initiate the download of the on-device speech recognition language pack for lang matching the requested {{SpeechRecognitionOptions/quality}} level floor.

@@ -542,7 +550,7 @@ See strong emphasis. + +

This subsection, and the "On-Device Model Security Considerations" subsection below, unlike many "privacy considerations" sections which only summarize and restate considerations that are already normatively specified elsewhere in the document, contain some normative requirements that are not present elsewhere, and add more detail to the normative requirements present elsewhere. The novel normative requirements are called out using strong emphasis.

Language Pack Availability

@@ -148,20 +149,20 @@ For on-device speech recognition, the exact download status of language packs ca
Download Masking
-To prevent cross-origin fingerprinting and site collusion, user agents must partition language pack availability by [=storage key=]. User agents must mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device. +To prevent cross-origin fingerprinting and site collusion, user agents must partition language pack availability by [=storage key=]. User agents must mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device (see this step of the [=availability algorithm=]). -This masking state is established for each (options, [=storage key=]) tuple. The user agent must only set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully starts a download or resolves to `true` (indicating the language pack is ready). +This masking state is established for each (options, [=storage key=]) tuple (specifically, per language in {{SpeechRecognitionOptions/langs}} and {{SpeechRecognitionOptions/quality}} of `options` for a given [=storage key=], ignoring {{SpeechRecognitionOptions/processLocally}}). The user agent must only set the state to "unmasked" for a specific (options, [=storage key=]) tuple after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully starts a download or resolves to `true` (indicating the language pack is ready; see this step of the {{SpeechRecognition/install()}} algorithm). When {{SpeechRecognition/processLocally}} is `true`, {{SpeechRecognition/start()}} must also treat a language pack whose availability is masked for the caller's [=storage key=] as not available (see this step of the [=start session algorithm=]). -Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-Time Friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as triggering a user permission prompt. +Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-Time Friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as consuming a user activation or triggering a user permission prompt. -Implementations must ensure that this [=storage key=]-based masking state is cleared whenever other storage for that origin is reset by the user. +Implementations must ensure that this [=storage key=]-based masking state is cleared whenever other storage for that [=storage key=] is cleared. -
Installation-time friction
+
Installation-Time Friction
The mitigation described in Download Masking works against attempts to silently fingerprint using {{SpeechRecognition/available()}}. The specification also contains requirements to prevent {{SpeechRecognition/install()}} from being easily used for fingerprinting, by introducing friction: * The {{SpeechRecognition/install()}} method both requires and consumes [=transient activation=] when, as far as the calling web page can observe, it would initiate a download (see this step of the {{SpeechRecognition/install()}} algorithm). -* The {{SpeechRecognition/install()}} method allows the user agent to prompt the user for permission, or to implicitly reject download attempts based on previous signals (such as an observed pattern of abuse). +* The {{SpeechRecognition/install()}} method allows the user agent to prompt the user for permission, or to implicitly reject download attempts based on previous signals (such as an observed pattern of abuse; see this step of the {{SpeechRecognition/install()}} algorithm). * Access to {{SpeechRecognition/install()}} and {{SpeechRecognition/available()}} is gated behind the [=policy-controlled feature=] "on-device-speech-recognition", which means that only top-level origins and their delegates can use these methods. Additionally, initiating the download process via {{SpeechRecognition/install()}} is more or less a one-time operation for a given language. The availability status will only transition from {{"downloadable"}} to {{"downloading"}} to {{"available"}} via these guarded installation operations. That is, while {{SpeechRecognition/install()}} can be used to read some of these fingerprinting bits (by observing the resolution of its promise and subsequent calls to {{SpeechRecognition/available()}}), doing so will effectively "destroy" those bits by changing the state. @@ -174,7 +175,7 @@ Because the availability of a language pack is strictly partitioned by [=storage Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed), the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector. -User agents may cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies. +User agents may cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies, provided that doing so does not change the availability status observable by any other [=storage key=] (e.g., not canceling an underlying download that another [=storage key=] is also waiting on).
Download Eviction
@@ -188,7 +189,7 @@ Evicting downloads in response to user-controlled actions (e.g., via a browser s Even if fingerprinting risks from availability status are mitigated, knowing a user has downloaded a specific language pack (e.g., for a minority language) can be sensitive. -For this reason, on top of installation-time friction, user agents may artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. This provides plausible deniability. If {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. +For this reason, on top of installation-time friction, user agents may artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise; see this step of the {{SpeechRecognition/install()}} algorithm) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. During such a fake download, the user agent must report the availability status for that [=storage key=] as {{"downloading"}} so that polling {{SpeechRecognition/available()}} cannot distinguish a fake delay from a real download. This provides plausible deniability: if {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. Such fake delays are not foolproof but offer some privacy benefit, especially when combined with other mitigations like prompts. @@ -209,13 +210,13 @@ Using user speech input in such a way would be a significant privacy leak, poten This reinforces the existing requirement: "To mitigate the risk of fingerprinting, user agents MUST NOT personalize speech recognition when performing speech recognition on a {{MediaStreamTrack}}." The considerations here apply broadly to any speech processed by on-device models via this API. -

Cloud-based vs. On-Device Implementations

+

Cloud-Based vs. On-Device Implementations

The Web Speech API can support both server-based (cloud) and client-based/embedded (on-device) recognition and synthesis. The {{SpeechRecognition/processLocally}} attribute allows developers to indicate a preference or requirement for on-device processing. -When `processLocally` is `false` (the default), user speech data may be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents should be transparent with users about where speech processing occurs. +When {{SpeechRecognition/processLocally}} is `false` (the default), user speech data may be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents should be transparent with users about where speech processing occurs. -When `processLocally` is `true`, the considerations in this "On-Device Model Privacy Considerations" section are paramount. +When {{SpeechRecognition/processLocally}} is `true`, the considerations in this "On-Device Model Privacy Considerations" section are paramount.

On-Device Model Security Considerations

@@ -225,7 +226,7 @@ Downloading language packs for on-device speech recognition via {{SpeechRecognit In the event of storage pressure, user agents should balance the utility of on-device speech recognition with the disk space its language packs take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents may involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. -If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}. +If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}.

Runtime Shared Resources

@@ -538,11 +539,10 @@ See download masking, this check is based on the masked availability status rather than on the true download status. As a result, whether {{SpeechRecognition/install()}} requires [=transient activation=] does not reveal the true download status to a web page for which it is still masked.

1. [=In parallel=], run the following steps: - 1. For each lang in {{SpeechRecognitionOptions/langs}} of options, initiate the download of the on-device speech recognition language pack for lang matching the requested {{SpeechRecognitionOptions/quality}} level floor. -

- Note: The user agent can prompt the user for explicit permission to download the on-device speech recognition language pack. -

- 1. Wait for the download of all languages specified by {{SpeechRecognitionOptions/langs}} of options to either succeed or fail. + 1. The user agent may prompt the user for explicit permission to download the on-device speech recognition language pack(s), or may decline to initiate the download based on implicit signals or storage pressure (see Installation-Time Friction, Download Eviction, and Disk Space for Language Packs). If the download is not permitted, [=queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with false, and abort these steps. + 1. For each lang in {{SpeechRecognitionOptions/langs}} of options, initiate the download of the on-device speech recognition language pack for lang matching the requested {{SpeechRecognitionOptions/quality}} level floor (if not already installed or downloading). + 1. If the download was successfully initiated (or the language pack is already installed or downloading) for every lang in {{SpeechRecognitionOptions/langs}} of options, set the download masking state for (options, [=storage key=]) to "unmasked". + 1. Wait for the download of all languages specified by {{SpeechRecognitionOptions/langs}} of options to either succeed or fail (including any artificial delay added for privacy reasons as described in Sensitive Language Information). 1. If the download of all those languages succeeded, let result be true, otherwise let result be false. 1. [=Queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with result.

@@ -595,7 +595,7 @@ See Download Masking) is available for all language in langs, let status be {{AvailabilityStatus/available}}. 1. Else, let status be {{AvailabilityStatus/unavailable}}. 1. If {{SpeechRecognitionOptions/processLocally}} of options is `true`:

    @@ -606,10 +606,11 @@ See Sensitive Language Information), set currentLanguageStatus to {{AvailabilityStatus/downloading}}. +
  1. Else if on-device speech recognition for language matching the requested {{SpeechRecognitionOptions/quality}} of options is installed (taking into account Model Version considerations), set currentLanguageStatus to {{AvailabilityStatus/available}}.
    2. +
  2. Else if on-device speech recognition for language matching the requested {{SpeechRecognitionOptions/quality}} of options is supported by the user agent but not yet installed, set currentLanguageStatus to {{AvailabilityStatus/downloadable}}.
  3. Else (on-device speech recognition for language is not supported), set currentLanguageStatus to {{AvailabilityStatus/unavailable}}.
    4. +
  4. If currentLanguageStatus is {{AvailabilityStatus/available}} or {{AvailabilityStatus/downloading}}, and the download masking state for (options, [=storage key=]) is not "unmasked" for language, set currentLanguageStatus to {{AvailabilityStatus/downloadable}}.
  5. If currentLanguageStatus comes after finalStatus in the ordered list `[{{AvailabilityStatus/available}}, {{AvailabilityStatus/downloading}}, {{AvailabilityStatus/downloadable}}, {{AvailabilityStatus/unavailable}}]`, set finalStatus to currentLanguageStatus.
@@ -634,7 +635,7 @@ following steps: 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to `phrases-not-supported` and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. 1. Abort these steps. 1. If this.{{SpeechRecognition/[[processLocally]]}} is `true`: - 1. If the user agent determines that local speech recognition is not available for this.{{SpeechRecognition/lang}}, or if it cannot fulfill the local processing requirement for other reasons: + 1. If the user agent determines that local speech recognition is not available for this.{{SpeechRecognition/lang}} (including if its availability is masked for the caller's [=storage key=] per Download Masking), or if it cannot fulfill the local processing requirement for other reasons (such as the requirements in On-Device Model Privacy Considerations and On-Device Model Security Considerations): 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to {{SpeechRecognitionErrorCode/service-not-allowed}} and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. 1. Abort these steps. 1. Set {{[[started]]}} to `true`. @@ -643,7 +644,7 @@ following steps: 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to {{SpeechRecognitionErrorCode/not-allowed}} and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. 1. Abort these steps. 1. Once the system is successfully listening to the recognition, queue a task to - [=fire an event=] named start at [=this=]. + [=fire an event=] named start at [=this=]. The speech recognition session must conform to the requirements in On-Device Model Privacy Considerations and On-Device Model Security Considerations, notably including User Input and Speech Data and Runtime Shared Resources.

SpeechRecognition Events

From dadbc0189eec956ea286a9c76e1d8f7c46c1f191 Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Wed, 14 Oct 2026 21:40:46 +0000 Subject: [PATCH 7/8] Use standard rfc2119 markup instead of strong emphasis in privacy/security sections --- index.bs | 34 ++++++++++++++++------------------ 1 file changed, 16 insertions(+), 18 deletions(-) diff --git a/index.bs b/index.bs index 8cc938b..1335ad3 100644 --- a/index.bs +++ b/index.bs @@ -141,21 +141,19 @@ This does not preclude adding support for this as a future API enhancement, and

On-Device Model Privacy Considerations

-

This subsection, and the "On-Device Model Security Considerations" subsection below, unlike many "privacy considerations" sections which only summarize and restate considerations that are already normatively specified elsewhere in the document, contain some normative requirements that are not present elsewhere, and add more detail to the normative requirements present elsewhere. The novel normative requirements are called out using strong emphasis.

-

Language Pack Availability

For on-device speech recognition, the exact download status of language packs can present a fingerprinting vector. How many bits this vector provides depends on the options provided to {{SpeechRecognition/available()}} or {{SpeechRecognition/install()}}, and how they influence the download (e.g., if different language packs have different availability statuses).
Download Masking
-To prevent cross-origin fingerprinting and site collusion, user agents must partition language pack availability by [=storage key=]. User agents must mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device (see this step of the [=availability algorithm=]). +To prevent cross-origin fingerprinting and site collusion, user agents MUST partition language pack availability by [=storage key=]. User agents MUST mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device (see this step of the [=availability algorithm=]). -This masking state is established for each (options, [=storage key=]) tuple (specifically, per language in {{SpeechRecognitionOptions/langs}} and {{SpeechRecognitionOptions/quality}} of `options` for a given [=storage key=], ignoring {{SpeechRecognitionOptions/processLocally}}). The user agent must only set the state to "unmasked" for a specific (options, [=storage key=]) tuple after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully starts a download or resolves to `true` (indicating the language pack is ready; see this step of the {{SpeechRecognition/install()}} algorithm). When {{SpeechRecognition/processLocally}} is `true`, {{SpeechRecognition/start()}} must also treat a language pack whose availability is masked for the caller's [=storage key=] as not available (see this step of the [=start session algorithm=]). +This masking state is established for each (options, [=storage key=]) tuple (specifically, per language in {{SpeechRecognitionOptions/langs}} and {{SpeechRecognitionOptions/quality}} of `options` for a given [=storage key=], ignoring {{SpeechRecognitionOptions/processLocally}}). The user agent MUST only set the state to "unmasked" for a specific (options, [=storage key=]) tuple after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully starts a download or resolves to `true` (indicating the language pack is ready; see this step of the {{SpeechRecognition/install()}} algorithm). When {{SpeechRecognition/processLocally}} is `true`, {{SpeechRecognition/start()}} MUST also treat a language pack whose availability is masked for the caller's [=storage key=] as not available (see this step of the [=start session algorithm=]). Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-Time Friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as consuming a user activation or triggering a user permission prompt. -Implementations must ensure that this [=storage key=]-based masking state is cleared whenever other storage for that [=storage key=] is cleared. +Implementations MUST ensure that this [=storage key=]-based masking state is cleared whenever other storage for that [=storage key=] is cleared.
Installation-Time Friction
@@ -175,13 +173,13 @@ Because the availability of a language pack is strictly partitioned by [=storage Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed), the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector. -User agents may cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies, provided that doing so does not change the availability status observable by any other [=storage key=] (e.g., not canceling an underlying download that another [=storage key=] is also waiting on). +User agents MAY cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies, provided that doing so does not change the availability status observable by any other [=storage key=] (e.g., not canceling an underlying download that another [=storage key=] is also waiting on).
Download Eviction
Another ingredient in ensuring that websites cannot toggle the availability state back and forth is to ensure that user agents don't use a quota-based eviction system for downloaded language packs that web pages can indirectly control. For example, if a user agent evicted less-recently-used language packs when new ones are installed, a web page could trigger such evictions to toggle the state of a target language pack. -To avoid this, user agents should not implement systems which allow web pages to control the eviction of downloaded language packs, including via indirect triggers such as further subsequent downloads. One way to fulfill this requirement is to never evict downloaded material in response to web page-initiated storage pressure, instead refusing to download new material (e.g., {{SpeechRecognition/install()}} resolving to `false`) if doing so would cause storage pressure. +To avoid this, user agents SHOULD NOT implement systems which allow web pages to control the eviction of downloaded language packs, including via indirect triggers such as further subsequent downloads. One way to fulfill this requirement is to never evict downloaded material in response to web page-initiated storage pressure, instead refusing to download new material (e.g., {{SpeechRecognition/install()}} resolving to `false`) if doing so would cause storage pressure. Evicting downloads in response to user-controlled actions (e.g., via a browser settings UI) is not problematic. @@ -189,7 +187,7 @@ Evicting downloads in response to user-controlled actions (e.g., via a browser s Even if fingerprinting risks from availability status are mitigated, knowing a user has downloaded a specific language pack (e.g., for a minority language) can be sensitive. -For this reason, on top of installation-time friction, user agents may artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise; see this step of the {{SpeechRecognition/install()}} algorithm) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. During such a fake download, the user agent must report the availability status for that [=storage key=] as {{"downloading"}} so that polling {{SpeechRecognition/available()}} cannot distinguish a fake delay from a real download. This provides plausible deniability: if {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. +For this reason, on top of installation-time friction, user agents MAY artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise; see this step of the {{SpeechRecognition/install()}} algorithm) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. During such a fake download, the user agent MUST report the availability status for that [=storage key=] as {{"downloading"}} so that polling {{SpeechRecognition/available()}} cannot distinguish a fake delay from a real download. This provides plausible deniability: if {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. Such fake delays are not foolproof but offer some privacy benefit, especially when combined with other mitigations like prompts. @@ -197,14 +195,14 @@ Such fake delays are not foolproof but offer some privacy benefit, especially wh The specific version or behavior of an on-device speech recognition model can also be a fingerprinting vector. This API does not expose model versions directly. -The best way to prevent the model version from becoming a fingerprinting vector is to tie it to the user agent's version, such that the model's version (and behavior) only updates alongside already-exposed information (like the User-Agent string). User agents should limit the number of possible model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}}. This might involve not providing model updates to older user agent versions or ignoring already-downloaded models below a minimum version threshold after a user agent update (instead, {{SpeechRecognition/available()}} might report {{"downloadable"}} for a newer version). +The best way to prevent the model version from becoming a fingerprinting vector is to tie it to the user agent's version, such that the model's version (and behavior) only updates alongside already-exposed information (like the User-Agent string). User agents SHOULD limit the number of possible model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}}. This might involve not providing model updates to older user agent versions or ignoring already-downloaded models below a minimum version threshold after a user agent update (instead, {{SpeechRecognition/available()}} might report {{"downloadable"}} for a newer version). There is a tradeoff: aggressively locking new user agent versions to new model versions can increase transitions between {{"available"}} and {{"downloadable"}}. This can be mitigated by allowing older models with newer user agent versions while a new model downloads, keeping the status {{"available"}} but briefly allowing identification of users with older-model/newer-user-agent combinations.

User Input and Speech Data

Speech data is inherently sensitive. -Implementations must not train or fine-tune on-device speech recognition models on user speech input obtained through this API, or otherwise store user speech input in a way that models can consult in the future (e.g., for personalization beyond the current session or across origins). +Implementations MUST NOT train or fine-tune on-device speech recognition models on user speech input obtained through this API, or otherwise store user speech input in a way that models can consult in the future (e.g., for personalization beyond the current session or across origins). Using user speech input in such a way would be a significant privacy leak, potentially exposing user information or information derived from interactions with one site to another. @@ -214,7 +212,7 @@ This reinforces the existing requirement: "To mitigate the risk of fingerprintin The Web Speech API can support both server-based (cloud) and client-based/embedded (on-device) recognition and synthesis. The {{SpeechRecognition/processLocally}} attribute allows developers to indicate a preference or requirement for on-device processing. -When {{SpeechRecognition/processLocally}} is `false` (the default), user speech data may be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents should be transparent with users about where speech processing occurs. +When {{SpeechRecognition/processLocally}} is `false` (the default), user speech data may be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents SHOULD be transparent with users about where speech processing occurs. When {{SpeechRecognition/processLocally}} is `true`, the considerations in this "On-Device Model Privacy Considerations" section are paramount. @@ -224,23 +222,23 @@ When {{SpeechRecognition/processLocally}} is `true`, the considerations in this Downloading language packs for on-device speech recognition via {{SpeechRecognition/install()}} could use significant amounts of the user's disk space. -In the event of storage pressure, user agents should balance the utility of on-device speech recognition with the disk space its language packs take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents may involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. +In the event of storage pressure, user agents SHOULD balance the utility of on-device speech recognition with the disk space its language packs take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents MAY involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. -If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}. +If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) SHOULD fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}.

Runtime Shared Resources

On-device speech recognition can consume significant runtime resources like CPU, memory, and potentially specialized hardware accelerators. -User agents should ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability. For example, it should not be possible for a background tab to monopolize speech processing resources, preventing a foreground tab from using them. +User agents SHOULD ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability. For example, it should not be possible for a background tab to monopolize speech processing resources, preventing a foreground tab from using them. -This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. If necessary to prevent resource exhaustion or instability, the user agent may cause speech recognition operations to fail (e.g., by firing a {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}). +This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. If necessary to prevent resource exhaustion or instability, the user agent MAY cause speech recognition operations to fail (e.g., by firing a {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}).

OS-Provided Models

One implementation strategy for on-device speech recognition is to delegate to models or capabilities provided by the underlying operating system. This can offer benefits like a consistent user experience and efficient resource usage. -However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents still need to ensure that all privacy and security requirements of this specification are met when using OS-provided models, even if the user agent has less control over the model's behavior. Particularly notable requirements to watch out for are those in User Input and Speech Data (preventing training on user data) and Runtime Shared Resources (ensuring fair and stable resource sharing). +However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents MUST still ensure that all privacy and security requirements of this specification are met when using OS-provided models, even if the user agent has less control over the model's behavior. Particularly notable requirements to watch out for are those in User Input and Speech Data (preventing training on user data) and Runtime Shared Resources (ensuring fair and stable resource sharing).

API Description

@@ -539,7 +537,7 @@ See download masking, this check is based on the masked availability status rather than on the true download status. As a result, whether {{SpeechRecognition/install()}} requires [=transient activation=] does not reveal the true download status to a web page for which it is still masked.

1. [=In parallel=], run the following steps: - 1. The user agent may prompt the user for explicit permission to download the on-device speech recognition language pack(s), or may decline to initiate the download based on implicit signals or storage pressure (see Installation-Time Friction, Download Eviction, and Disk Space for Language Packs). If the download is not permitted, [=queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with false, and abort these steps. + 1. The user agent MAY prompt the user for explicit permission to download the on-device speech recognition language pack(s), or MAY decline to initiate the download based on implicit signals or storage pressure (see Installation-Time Friction, Download Eviction, and Disk Space for Language Packs). If the download is not permitted, [=queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with false, and abort these steps. 1. For each lang in {{SpeechRecognitionOptions/langs}} of options, initiate the download of the on-device speech recognition language pack for lang matching the requested {{SpeechRecognitionOptions/quality}} level floor (if not already installed or downloading). 1. If the download was successfully initiated (or the language pack is already installed or downloading) for every lang in {{SpeechRecognitionOptions/langs}} of options, set the download masking state for (options, [=storage key=]) to "unmasked". 1. Wait for the download of all languages specified by {{SpeechRecognitionOptions/langs}} of options to either succeed or fail (including any artificial delay added for privacy reasons as described in Sensitive Language Information). @@ -644,7 +642,7 @@ following steps: 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to {{SpeechRecognitionErrorCode/not-allowed}} and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. 1. Abort these steps. 1. Once the system is successfully listening to the recognition, queue a task to - [=fire an event=] named start at [=this=]. The speech recognition session must conform to the requirements in On-Device Model Privacy Considerations and On-Device Model Security Considerations, notably including User Input and Speech Data and Runtime Shared Resources. + [=fire an event=] named start at [=this=]. The speech recognition session MUST conform to the requirements in On-Device Model Privacy Considerations and On-Device Model Security Considerations, notably including User Input and Speech Data and Runtime Shared Resources.

SpeechRecognition Events

From 0c7a98dca4d5e14bcc8f5185a914d07ed4e04e98 Mon Sep 17 00:00:00 2001 From: Evan Liu Date: Wed, 14 Oct 2026 22:16:45 +0000 Subject: [PATCH 8/8] Move normative privacy and security requirements into API Description section --- index.bs | 101 +++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 69 insertions(+), 32 deletions(-) diff --git a/index.bs b/index.bs index 1335ad3..46f74b6 100644 --- a/index.bs +++ b/index.bs @@ -139,21 +139,23 @@ This does not preclude adding support for this as a future API enhancement, and The above recommendations are intended to reduce this risk of such attacks. +
+

On-Device Model Privacy Considerations

+

This section is non-normative.

+

Language Pack Availability

For on-device speech recognition, the exact download status of language packs can present a fingerprinting vector. How many bits this vector provides depends on the options provided to {{SpeechRecognition/available()}} or {{SpeechRecognition/install()}}, and how they influence the download (e.g., if different language packs have different availability statuses).
Download Masking
-To prevent cross-origin fingerprinting and site collusion, user agents MUST partition language pack availability by [=storage key=]. User agents MUST mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device (see this step of the [=availability algorithm=]). - -This masking state is established for each (options, [=storage key=]) tuple (specifically, per language in {{SpeechRecognitionOptions/langs}} and {{SpeechRecognitionOptions/quality}} of `options` for a given [=storage key=], ignoring {{SpeechRecognitionOptions/processLocally}}). The user agent MUST only set the state to "unmasked" for a specific (options, [=storage key=]) tuple after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully starts a download or resolves to `true` (indicating the language pack is ready; see this step of the {{SpeechRecognition/install()}} algorithm). When {{SpeechRecognition/processLocally}} is `true`, {{SpeechRecognition/start()}} MUST also treat a language pack whose availability is masked for the caller's [=storage key=] as not available (see this step of the [=start session algorithm=]). +To prevent cross-origin fingerprinting and site collusion, the specification requires user agents to partition language pack availability by [=storage key=] and mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the underlying language pack is {{"available"}} or {{"downloading"}} on the user's device (see the download masking requirements for the [=download masking state=] under {{SpeechRecognition/available()}} and this step of the [=availability algorithm=]). -Since {{SpeechRecognition/install()}} has stronger requirements (see Installation-Time Friction), this ensures that web pages only get access to the true download status after taking a more costly action, such as consuming a user activation or triggering a user permission prompt. +Because the [=download masking state=] for a given (options, [=storage key=]) tuple is only unmasked after a web page within that [=storage key=] explicitly calls {{SpeechRecognition/install()}} (see this step of {{SpeechRecognition/install()}})—and because {{SpeechRecognition/start()}} also treats masked language packs as unavailable when {{SpeechRecognition/processLocally}} is `true` (see this step of the [=start session algorithm=])—web pages only gain access to the true download status after taking a more costly action, such as consuming a user activation or triggering a user permission prompt (see Installation-Time Friction). -Implementations MUST ensure that this [=storage key=]-based masking state is cleared whenever other storage for that [=storage key=] is cleared. +The requirement to clear the download masking state whenever other storage for that [=storage key=] is cleared prevents the unmasked state from persisting across storage resets.
Installation-Time Friction
@@ -171,15 +173,13 @@ Additionally, initiating the download process via {{SpeechRecognition/install()} Because the availability of a language pack is strictly partitioned by [=storage key=] (see Download Masking), a web page can only observe the download state within its own partition. -Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed), the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector. - -User agents MAY cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies, provided that doing so does not change the availability status observable by any other [=storage key=] (e.g., not canceling an underlying download that another [=storage key=] is also waiting on). +Therefore, if a user agent cancels an ongoing download due to page-controlled actions (e.g., navigation, page unload, or an iframe being destroyed) as permitted by the cancelation rules for {{SpeechRecognition/install()}}, the resulting change in state (e.g., reverting from {{"downloading"}} to {{"downloadable"}}) is only visible to that specific [=storage key=], as long as canceling does not alter the availability status observable by other [=storage keys=]. This prevents the cancelation behavior from being used as a cross-origin fingerprinting vector.
Download Eviction
Another ingredient in ensuring that websites cannot toggle the availability state back and forth is to ensure that user agents don't use a quota-based eviction system for downloaded language packs that web pages can indirectly control. For example, if a user agent evicted less-recently-used language packs when new ones are installed, a web page could trigger such evictions to toggle the state of a target language pack. -To avoid this, user agents SHOULD NOT implement systems which allow web pages to control the eviction of downloaded language packs, including via indirect triggers such as further subsequent downloads. One way to fulfill this requirement is to never evict downloaded material in response to web page-initiated storage pressure, instead refusing to download new material (e.g., {{SpeechRecognition/install()}} resolving to `false`) if doing so would cause storage pressure. +To avoid this, the download eviction rules for {{SpeechRecognition/install()}} discourage systems that allow web pages to control the eviction of downloaded language packs, including via indirect triggers such as further subsequent downloads. One way to fulfill this is to never evict downloaded material in response to web page-initiated storage pressure, instead refusing to download new material (e.g., {{SpeechRecognition/install()}} resolving to `false`) if doing so would cause storage pressure. Evicting downloads in response to user-controlled actions (e.g., via a browser settings UI) is not problematic. @@ -187,7 +187,7 @@ Evicting downloads in response to user-controlled actions (e.g., via a browser s Even if fingerprinting risks from availability status are mitigated, knowing a user has downloaded a specific language pack (e.g., for a minority language) can be sensitive. -For this reason, on top of installation-time friction, user agents MAY artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise; see this step of the {{SpeechRecognition/install()}} algorithm) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. During such a fake download, the user agent MUST report the availability status for that [=storage key=] as {{"downloading"}} so that polling {{SpeechRecognition/available()}} cannot distinguish a fake delay from a real download. This provides plausible deniability: if {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. +For this reason, on top of installation-time friction, this step of the {{SpeechRecognition/install()}} algorithm allows user agents to artificially fake a download (e.g., by adding a delay to the resolution of the {{SpeechRecognition/install()}} promise while reporting {{"downloading"}} for that [=storage key=]) if they believe it would be helpful for privacy reasons, instead of having {{SpeechRecognition/install()}} resolve instantly if the language pack is already present. This provides plausible deniability: if {{SpeechRecognition/install()}} takes a few seconds to resolve `true`, it could be a fake delay or a quick real download. Such fake delays are not foolproof but offer some privacy benefit, especially when combined with other mitigations like prompts. @@ -195,50 +195,53 @@ Such fake delays are not foolproof but offer some privacy benefit, especially wh The specific version or behavior of an on-device speech recognition model can also be a fingerprinting vector. This API does not expose model versions directly. -The best way to prevent the model version from becoming a fingerprinting vector is to tie it to the user agent's version, such that the model's version (and behavior) only updates alongside already-exposed information (like the User-Agent string). User agents SHOULD limit the number of possible model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}}. This might involve not providing model updates to older user agent versions or ignoring already-downloaded models below a minimum version threshold after a user agent update (instead, {{SpeechRecognition/available()}} might report {{"downloadable"}} for a newer version). +The best way to prevent the model version from becoming a fingerprinting vector is to tie it to the user agent's version, such that the model's version (and behavior) only updates alongside already-exposed information (like the User-Agent string). Accordingly, the model version rules under {{SpeechRecognition/available()}} encourage limiting the number of possible model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}}. This might involve not providing model updates to older user agent versions or ignoring already-downloaded models below a minimum version threshold after a user agent update (instead, {{SpeechRecognition/available()}} might report {{"downloadable"}} for a newer version). There is a tradeoff: aggressively locking new user agent versions to new model versions can increase transitions between {{"available"}} and {{"downloadable"}}. This can be mitigated by allowing older models with newer user agent versions while a new model downloads, keeping the status {{"available"}} but briefly allowing identification of users with older-model/newer-user-agent combinations.

User Input and Speech Data

-Speech data is inherently sensitive. -Implementations MUST NOT train or fine-tune on-device speech recognition models on user speech input obtained through this API, or otherwise store user speech input in a way that models can consult in the future (e.g., for personalization beyond the current session or across origins). +Speech data is inherently sensitive. The user input retention rules under {{SpeechRecognition/processLocally}} prohibit implementations from training or fine-tuning on-device speech recognition models on user speech input obtained through this API, or otherwise storing user speech input in a way that models can consult in the future (e.g., for personalization beyond the current session or across origins). Using user speech input in such a way would be a significant privacy leak, potentially exposing user information or information derived from interactions with one site to another. -This reinforces the existing requirement: "To mitigate the risk of fingerprinting, user agents MUST NOT personalize speech recognition when performing speech recognition on a {{MediaStreamTrack}}." The considerations here apply broadly to any speech processed by on-device models via this API. +This reinforces the existing requirement in Security and privacy considerations against personalizing speech recognition when performing speech recognition on a {{MediaStreamTrack}}, and applies broadly to any speech processed by on-device models via this API.

Cloud-Based vs. On-Device Implementations

-The Web Speech API can support both server-based (cloud) and client-based/embedded (on-device) recognition and synthesis. The {{SpeechRecognition/processLocally}} attribute allows developers to indicate a preference or requirement for on-device processing. +The Web Speech API can support both server-based (cloud) and client-based/embedded (on-device) recognition and synthesis. The {{SpeechRecognition/processLocally}} attribute allows developers to indicate a requirement for on-device processing. -When {{SpeechRecognition/processLocally}} is `false` (the default), user speech data may be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing. User agents SHOULD be transparent with users about where speech processing occurs. +When {{SpeechRecognition/processLocally}} is `false` (the default), user speech data might be sent to a remote server for processing. Web developers need to be aware of this possibility and the associated privacy implications if they do not explicitly request local processing, and user agents are encouraged to be transparent with users about where speech processing occurs. When {{SpeechRecognition/processLocally}} is `true`, the considerations in this "On-Device Model Privacy Considerations" section are paramount.

On-Device Model Security Considerations

+

This section is non-normative.

+

Disk Space for Language Packs

Downloading language packs for on-device speech recognition via {{SpeechRecognition/install()}} could use significant amounts of the user's disk space. -In the event of storage pressure, user agents SHOULD balance the utility of on-device speech recognition with the disk space its language packs take up, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in Download Eviction when considering freeing up disk space by evicting language packs. User agents MAY involve the user in these decisions, e.g., via download-time prompts or a browser UI for managing downloaded language packs. +As specified in the storage pressure rules for {{SpeechRecognition/install()}}, in the event of storage pressure, user agents are expected to balance the utility of on-device speech recognition with the disk space its language packs take up—possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways, while remaining mindful of the privacy impacts discussed in Download Eviction and potentially involving the user via download-time prompts or a browser UI for managing downloaded language packs. -If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) SHOULD fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}. +If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) are expected to fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} (or {{"unavailable"}} if no longer supported), and {{SpeechRecognition/start()}} firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/service-not-allowed}} or {{SpeechRecognitionErrorCode/language-not-supported}}.

Runtime Shared Resources

On-device speech recognition can consume significant runtime resources like CPU, memory, and potentially specialized hardware accelerators. -User agents SHOULD ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability. For example, it should not be possible for a background tab to monopolize speech processing resources, preventing a foreground tab from using them. +As described in the runtime resource requirements of the [=start session algorithm=], user agents are expected to ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, another web page's general operation, or overall system stability. For example, a background tab ought not to be able to monopolize speech processing resources and prevent a foreground tab from using them. -This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. If necessary to prevent resource exhaustion or instability, the user agent MAY cause speech recognition operations to fail (e.g., by firing a {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}). +This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior, and failing speech recognition operations (e.g., by firing a {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}) when necessary to prevent resource exhaustion or instability.

OS-Provided Models

One implementation strategy for on-device speech recognition is to delegate to models or capabilities provided by the underlying operating system. This can offer benefits like a consistent user experience and efficient resource usage. -However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents MUST still ensure that all privacy and security requirements of this specification are met when using OS-provided models, even if the user agent has less control over the model's behavior. Particularly notable requirements to watch out for are those in User Input and Speech Data (preventing training on user data) and Runtime Shared Resources (ensuring fair and stable resource sharing). +However, this approach comes with the usual considerations of exposing OS capabilities to the web. As specified in the requirements for OS-provided models under {{SpeechRecognition/processLocally}}, user agents still need to ensure that all privacy and security requirements of this specification are met when using OS-provided models, even if the user agent has less control over the model's behavior. Particularly notable requirements to watch out for are not training on user speech input (see User Input and Speech Data) and fair and stable resource sharing (see Runtime Shared Resources). + +

API Description

@@ -449,6 +452,15 @@ interface SpeechRecognitionPhrase {
This attribute, when set to true, indicates a requirement that the speech recognition process MUST be performed locally on the user's device. If set to false, the user agent can choose between local and remote processing. The default value is false. +

+ When {{SpeechRecognition/processLocally}} is `false`, user speech data might be sent to a remote server for processing; user agents SHOULD be transparent with users about where speech processing occurs (see Cloud-Based vs. On-Device Implementations). +

+

+ Regardless of the value of {{SpeechRecognition/processLocally}}, implementations MUST NOT train or fine-tune on-device speech recognition models on user speech input obtained through this API, or otherwise store user speech input in a way that models can consult in the future (e.g., for personalization beyond the current session or across origins; see User Input and Speech Data). +

+

+ When delegating on-device speech recognition to models or capabilities provided by the underlying operating system, user agents MUST ensure that all privacy and security requirements of this specification are met, even if the user agent has less control over the model's behavior (see OS-Provided Models). +

phrases attribute
@@ -508,6 +520,16 @@ See [=default allowlist/'self'=]. +

+ To prevent cross-origin fingerprinting and site collusion, user agents MUST partition on-device language pack availability by [=storage key=]. User agents MUST mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default when {{SpeechRecognitionOptions/processLocally}} is `true`, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device (see this step of the [=availability algorithm=] and Download Masking). + This download masking state is established for each (options, [=storage key=]) tuple (specifically, per language in {{SpeechRecognitionOptions/langs}} and {{SpeechRecognitionOptions/quality}} of options for a given [=storage key=], ignoring {{SpeechRecognitionOptions/processLocally}}). The user agent MUST only set the state to "unmasked" for a specific (options, [=storage key=]) tuple after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and that call either successfully initiates (or joins an ongoing) download or resolves to `true` (indicating the language pack is ready; see this step of {{SpeechRecognition/install()}}). + Implementations MUST ensure that this [=storage key=]-based [=download masking state=] is cleared whenever other storage for that [=storage key=] is cleared. +

+ +

+ User agents SHOULD limit the number of possible on-device model versions that a single user agent version can be paired with when determining if a language pack is {{"available"}} via {{SpeechRecognition/available()}} (for example, by tying model updates to user agent updates or ignoring already-downloaded models below a minimum version threshold after a user agent update and reporting {{"downloadable"}} for a newer version instead; see Model Version). +

+ When invoked, run these steps: 1. Let promise be a new promise. 1. Run the availability algorithm with options and promise. @@ -534,13 +556,13 @@ See download masking, this check is based on the masked availability status rather than on the true download status. As a result, whether {{SpeechRecognition/install()}} requires [=transient activation=] does not reveal the true download status to a web page for which it is still masked. + Note: Because {{SpeechRecognition/available()}} is subject to [=download masking state=] rules, this check is based on the masked availability status rather than on the true download status. As a result, whether {{SpeechRecognition/install()}} requires [=transient activation=] does not reveal the true download status to a web page for which it is still masked.

1. [=In parallel=], run the following steps: - 1. The user agent MAY prompt the user for explicit permission to download the on-device speech recognition language pack(s), or MAY decline to initiate the download based on implicit signals or storage pressure (see Installation-Time Friction, Download Eviction, and Disk Space for Language Packs). If the download is not permitted, [=queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with false, and abort these steps. + 1. The user agent MAY prompt the user for explicit permission to download the on-device speech recognition language pack(s), or MAY decline to initiate the download based on implicit signals (such as an observed pattern of abuse; see Installation-Time Friction) or storage pressure. In the event of storage pressure, user agents SHOULD balance the utility of on-device speech recognition with the disk space its language packs take up (possibly by declining new downloads or freeing up disk space in other ways, subject to the eviction requirements below; see Disk Space for Language Packs), and MAY involve the user in these decisions (e.g., via the download-time permission prompt or a browser UI for managing downloaded language packs). If the download is not permitted, [=queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with false, and abort these steps. 1. For each lang in {{SpeechRecognitionOptions/langs}} of options, initiate the download of the on-device speech recognition language pack for lang matching the requested {{SpeechRecognitionOptions/quality}} level floor (if not already installed or downloading). - 1. If the download was successfully initiated (or the language pack is already installed or downloading) for every lang in {{SpeechRecognitionOptions/langs}} of options, set the download masking state for (options, [=storage key=]) to "unmasked". - 1. Wait for the download of all languages specified by {{SpeechRecognitionOptions/langs}} of options to either succeed or fail (including any artificial delay added for privacy reasons as described in Sensitive Language Information). + 1. If the download was successfully initiated (or the language pack is already installed or downloading) for every lang in {{SpeechRecognitionOptions/langs}} of options, set the [=download masking state=] for (options, [=storage key=]) to "unmasked". + 1. Wait for the download of all languages specified by {{SpeechRecognitionOptions/langs}} of options to either succeed or fail. User agents MAY artificially fake a download (e.g., by adding a delay before resolving promise) if they believe it would be helpful for privacy reasons, instead of resolving instantly if the language pack is already present (see Sensitive Language Information). During such a fake download, the user agent MUST report the availability status for that [=storage key=] as {{"downloading"}} (and treat the language pack as not yet installed for other purposes observable by that [=storage key=], including in the [=start session algorithm=] and the {{SpeechRecognitionOptions/processLocally}} `false` branch of the [=availability algorithm=]) so that the web page cannot distinguish a fake delay from a real download. 1. If the download of all those languages succeeded, let result be true, otherwise let result be false. 1. [=Queue a task=] on the [=relevant global object=]'s [=task queue=] to resolve promise with result.

@@ -550,6 +572,14 @@ See + User agents MAY cancel ongoing language pack downloads when the requesting document is destroyed or navigated away from, in accordance with their standard resource management policies, provided that doing so does not change the availability status observable by any other [=storage key=] (e.g., not canceling an underlying download that another [=storage key=] is also waiting on; see Download Cancelation). +

+ +

+ User agents SHOULD NOT implement systems which allow web pages to control the eviction of downloaded language packs, including via indirect triggers such as further subsequent downloads (for example, quota-based or least-recently-used eviction in response to web page-initiated storage pressure; see Download Eviction). Evicting downloads in response to user-controlled actions (e.g., via a browser settings UI) is permitted. +

@@ -593,7 +623,7 @@ See Download Masking) is available for all language in langs, let status be {{AvailabilityStatus/available}}. + 1. Else if speech recognition (which may be remote, or on-device if its [=download masking state=] is "unmasked" for the caller's [=storage key=]) is available for all language in langs, let status be {{AvailabilityStatus/available}}. 1. Else, let status be {{AvailabilityStatus/unavailable}}. 1. If {{SpeechRecognitionOptions/processLocally}} of options is `true`:
    @@ -604,11 +634,11 @@ See Sensitive Language Information), set currentLanguageStatus to {{AvailabilityStatus/downloading}}. -
  1. Else if on-device speech recognition for language matching the requested {{SpeechRecognitionOptions/quality}} of options is installed (taking into account Model Version considerations), set currentLanguageStatus to {{AvailabilityStatus/available}}.
    2. +
  2. If on-device speech recognition for language matching the requested {{SpeechRecognitionOptions/quality}} of options is installed (taking into account model version limits) and no fake download is in progress for the caller's [=storage key=] per the `install()` algorithm, set currentLanguageStatus to {{AvailabilityStatus/available}}.
    3. +
  3. Else if on-device speech recognition for language matching the requested {{SpeechRecognitionOptions/quality}} of options is currently being downloaded (including a fake download for the caller's [=storage key=] per the `install()` algorithm), set currentLanguageStatus to {{AvailabilityStatus/downloading}}.
  4. Else if on-device speech recognition for language matching the requested {{SpeechRecognitionOptions/quality}} of options is supported by the user agent but not yet installed, set currentLanguageStatus to {{AvailabilityStatus/downloadable}}.
  5. Else (on-device speech recognition for language is not supported), set currentLanguageStatus to {{AvailabilityStatus/unavailable}}.
    6. -
  6. If currentLanguageStatus is {{AvailabilityStatus/available}} or {{AvailabilityStatus/downloading}}, and the download masking state for (options, [=storage key=]) is not "unmasked" for language, set currentLanguageStatus to {{AvailabilityStatus/downloadable}}.
    7. +
  7. If currentLanguageStatus is {{AvailabilityStatus/available}} or {{AvailabilityStatus/downloading}}, and the [=download masking state=] for (options, [=storage key=]) is not "unmasked" for language, set currentLanguageStatus to {{AvailabilityStatus/downloadable}}.
  8. If currentLanguageStatus comes after finalStatus in the ordered list `[{{AvailabilityStatus/available}}, {{AvailabilityStatus/downloading}}, {{AvailabilityStatus/downloadable}}, {{AvailabilityStatus/unavailable}}]`, set finalStatus to currentLanguageStatus.
@@ -633,8 +663,8 @@ following steps: 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to `phrases-not-supported` and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. 1. Abort these steps. 1. If this.{{SpeechRecognition/[[processLocally]]}} is `true`: - 1. If the user agent determines that local speech recognition is not available for this.{{SpeechRecognition/lang}} (including if its availability is masked for the caller's [=storage key=] per Download Masking), or if it cannot fulfill the local processing requirement for other reasons (such as the requirements in On-Device Model Privacy Considerations and On-Device Model Security Considerations): - 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to {{SpeechRecognitionErrorCode/service-not-allowed}} and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. + 1. The user agent MUST treat a language pack whose [=download masking state=] is not "unmasked" for the caller's [=storage key=] as not available for local speech recognition. If the user agent determines that local speech recognition is not available for this.{{SpeechRecognition/lang}} (including when its [=download masking state=] is not "unmasked" for the caller's [=storage key=] or when a previously installed language pack was evicted; see the download masking requirements and the eviction failure requirements below), or if it cannot fulfill the local processing requirement for other reasons (such as preventing resource exhaustion): + 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to {{SpeechRecognitionErrorCode/service-not-allowed}} and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason, which MUST NOT reveal whether a language pack whose [=download masking state=] is not "unmasked" for the caller's [=storage key=] is installed or being downloaded. 1. Abort these steps. 1. Set {{[[started]]}} to `true`. 1. If |requestMicrophonePermission| is `true` and [=request @@ -642,7 +672,14 @@ following steps: 1. [=Queue a task=] to [=fire an event=] named error at [=this=] using {{SpeechRecognitionErrorEvent}} with its {{SpeechRecognitionErrorEvent/error}} attribute initialized to {{SpeechRecognitionErrorCode/not-allowed}} and its {{SpeechRecognitionErrorEvent/message}} attribute set to an implementation-defined string detailing the reason. 1. Abort these steps. 1. Once the system is successfully listening to the recognition, queue a task to - [=fire an event=] named start at [=this=]. The speech recognition session MUST conform to the requirements in On-Device Model Privacy Considerations and On-Device Model Security Considerations, notably including User Input and Speech Data and Runtime Shared Resources. + [=fire an event=] named start at [=this=]. + +

+ If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} set to `true`) SHOULD fail gracefully (such as {{SpeechRecognition/available()}} returning {{"downloadable"}} (or {{"unavailable"}} if no longer supported), and {{SpeechRecognition/start()}} firing a {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/service-not-allowed}} (or {{SpeechRecognitionErrorCode/language-not-supported}} if no longer supported); see Disk Space for Language Packs). +

+

+ User agents SHOULD ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability (for example, by queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior; see Runtime Shared Resources). If necessary to prevent resource exhaustion or instability, the user agent MAY cause speech recognition operations to fail (e.g., by firing a {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}). On-device speech recognition sessions are also subject to the user input retention rules and the OS-provided model rules under {{SpeechRecognition/processLocally}}. +

SpeechRecognition Events