Repository navigation
Conversation
lgirdwood
changed the title
Member
Author
|
@naveen-manohar @harajend @bardliao @ujfalusi this is just for testing on the CI, it has a lot of patches not for upstream, just for debug. Will allow us to roll out WOV test alongside other WOV PR in flow. |
Contributor
|
Thanks @lgirdwood |
…overy Support building microWakeWord with external or dynamic toolchain configurations where libc.a / libm.a archive layouts differ between Xtensa toolchain versions. Specifically: - Check for XTENSA_SYSTEM libc archive path when not found in standard Zephyr paths. - Check libc archive member names to extract either Newlib (lib_a-*.o) or standard libc/libm members into the shim archive. - Run mww_shim_flatten.py to flatten .text.* and .literal.* sections into .text and .literal to prevent CALL8 out-of-range relocation overflows. - Enable -mlongcalls and -mtext-section-literals on Xtensa architecture builds. - Conditionally include ibuffer_allocator.cc when present in the TFLM tree. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
tlv_value_set_uaol_caps() set link_count to the devicetree device count unconditionally and indexed link_caps[] by device index, so a link whose uaol_get_capabilities() call failed was still advertised to the host as a present link with all-zero capabilities. That matters now that the driver rejects the query when the host has not set UAOLCTL.OFLEN: without this, a firmware that cannot read any link still reports link_count = 1 with zeroed stream counts and FIFO sizes. Count only the links that answered, pack them from index 0, and size the TLV to match. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
… on IPC4 kpb.c drives both a sel_sink (buf_id 0, the real-time feed to detectors) and a host_sink (buf_id 1, the drain that bursts pre-roll to the host). Declare two sink pins in kpb.toml so the drain path can be bound from topology. Additionally, under IPC4 pipeline state transitions and resets are synchronous; returning -EBUSY on reset breaks the host teardown state machine and causes widget free failures. Guard the deferred -EBUSY drain check under #if !CONFIG_IPC_MAJOR_4. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add NOTIFIER_ID_D0IX_STATE to notify registered DSP components when the host changes the DSP D0 substate via SOF_IPC4_MOD_SET_D0IX. Components such as microWakeWord that keep running while the host sleeps in S0iX / D0i3 can use this event to adjust telemetry reporting or power characteristics. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
When source or sink component is not a processing_module (such as a host copier or non-module comp_dev), comp_mod() returns NULL. Safely guard src_module_data and dst_module_data before accessing mpd.in_buff_size and mpd.out_buff_size to prevent NULL pointer dereferences when configuring DP ring buffers. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Hostless internal pipelines (such as background keyword detection or internal audio processing chains that feed an arbiter or buffer without a host copier) do not have a host dev. Allow ipc4_pipeline_prepare() and ipc4_pipeline_trigger() to operate on pipelines where host is NULL by falling back to source_comp or sink_comp. Furthermore, allow stopping pipelines that are currently in COMP_STATE_PREPARE. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…drain streaming Improve the WoV Arbiter component: - Host Notification: send IPC4 notification to host upon keyword detection and periodically during pre-roll buffer drain. - Active Slot Reset: reset active_slot to WOV_ARB_NO_ACTIVE and notify host/kcontrols on stream stop, reset, and free. Also emit WOV_ARB_CMD_RESUME to unpause detectors. - Drain Streaming: ensure the arbiter only produces data when an active slot is triggered, leaving the host PCM idle/blocked during low-power listening so the platform can enter D0i3. - Update wov_arbiter README documentation. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Adapt the ECNS module to support the dual-rate DMIC architecture: - Input pin 0: 16 kHz stream from dmic16k (up to 320 frames per 20ms period). Extracts left channel to mono output pin 0 feeding KPB. - Input pin 1: 48 kHz stream from dmic01 (up to 960 frames per 20ms period). Copies stereo 48 kHz output to pin 1 feeding Host Copier PCM 11. - Operates in 20ms DP mode with independent buffer tracking per pin. - Update ECNS header and README documentation. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…ter to fix heap leak
TFLite Micro defines TF_LITE_REMOVE_VIRTUAL_DELETE on MicroMutableOpResolver,
which expands to a no-op 'void operator delete(void* p) {}'. Consequently,
calling 'delete inst->op_resolver' in MWW_Free() never returned the memory
to the heap, leaking ~600 bytes per slot on every stream open/close cycle.
Across multi-slot configurations (3 detector slots), repeated stream cycles
exhausted the DSP system heap, failing on run 10 with -ENOMEM in
MWW_InitOps() and returning -22 (IPC4_INVALID_REQUEST) to snd_pcm_start().
Eliminate heap allocations from MWW model initialization by allocating
both MwwOpResolver and MicroInterpreter in aligned static storage using
placement new and explicit destructor invocation.
Verified with 10 consecutive D0i3 sleep-and-wake test cycles on Panther Lake
hardware (Aphid), achieving 10/10 passes with zero errors.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…er controls Add power and testing enhancements to microWakeWord: - D0i3 Power State: listen for NOTIFIER_ID_D0IX_STATE and suppress score telemetry notifications while the host is in D0i3 to prevent spurious wakeups. - Test Trigger Controls: implement SOF_IPC4_SWITCH_CONTROL_PARAM_ID to expose a volatile 'wovdebug' switch kcontrol per slot. Setting this control arms a fake wake timer (MWW_FAKE_WAKE_DELAY_MS) that triggers a keyword detect event and automatically resets the control upon firing. - Add CONFIG_MICROWAKEWORD_FAKE_WAKE and app/wov-d0i3-fake-wake.conf test overlay. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Disable UAOL on Intel ADSP targets in boards/intel_adsp/Kconfig.defconfig (default n). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…odules Functional and feature topologies (such as SoundWire RT721/RT722 and DMIC pipelines) require open audio processing components like DRC, TDFB, Multiband DRC, Aria, and MUX. Ensure these modules are enabled in both PTL and WCL board configurations and module overlays so they are built into sof-ptl-openmodules.ri and sof-wcl-openmodules.ri. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…slot WoV pipelines Implement the dual-rate DMIC topology architecture with multi-slot WoV: - Provide dual PDM DAI endpoints: - dmic01 on PDM0: 48 kHz stereo 16-bit - dmic16k on PDM1: 16 kHz stereo 16-bit - Connect dmic16k into ECNS (pin 0) and dmic01 into ECNS (pin 1). - Connect ECNS output pin 0 (16k mono clean) to KPB and multi-slot WoV detectors (slots 0-2: OK Google, Alexa, Hey Jarvis), feeding WoV Arbiter and Host Copier PCM 12 (16 kHz mono WoV capture). - Connect ECNS output pin 1 (48k stereo clean) to Host Copier PCM 11 (48 kHz stereo capture). - Remove deprecated raw PCM 10 host copier. - Add volatile wovdebug switch controls and score enum controls per slot. - Set capture_compatible_d0i3 to true on WoV PCMs. - Update 4ch and multi manifests for PTL, WCL, and generic Intel platforms. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…topologies Integrate the multi-slot WoV pipelines and feature flags into SoundWire and HDA topologies: - Wire multi-slot WoV feature inclusion into cavs-sdw.conf, cavs-rt5682.conf, and sof-hda-generic.conf. - Ensure Speaker and other primary PCMs are preserved when WoV features are instantiated. - Add CMake topology build targets for Panther Lake (PTL) ACE 3.0, Wildcat Lake (WCL), and Tiger Lake (TGL) with multi-slot WoV SoundWire and HDA targets in production and development CMake files. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…PTL, TGL, and WCL Provide prebuilt topology binary files for quick testing and hardware bring-up: - build/sof-ptl-dmic-wov-multi-4ch.tplg - build/sof-tgl-dmic-wov-multi-4ch.tplg - build/sof-wcl-dmic-wov-multi-4ch.tplg Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Add test utilities for verifying Wake-on-Voice capture pipelines: - wov_blocking_read.c: alsa-lib utility that opens the WoV capture PCM in blocking read mode with SND_PCM_NO_IRQ_WAITEVENT or mmap to ensure the driver does not generate periodic IRQs during listening. - wov_blocking_read_tinyalsa.c: TinyALSA version of the blocking reader. - test_wov_no_irq.py: automated test runner validating WoV capture and wake across all slots in S0 and D0i3 (idle suspend) states. - Makefile: build targets for wov_blocking_read and wov_blocking_read_tinyalsa. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…i3 verification Comprehensive documentation updates for multi-slot Wake-on-Voice and dual-rate DMIC: - tools/topology/topology2/README.md: - Document complete pipeline architecture and DMIC routing diagrams. - Document dual-rate DMIC PDM DAI structure (dmic01 @ 48k, dmic16k @ 16k). - Document ECNS 20ms DP processing and pin mappings. - Document KPB circular buffering, pre-roll drain, and WoV Arbiter slot routing. - Document multi-slot WoV detector pipeline (OK Google, Alexa, Hey Jarvis). - Document PCM endpoints: PCM 11 (48k stereo clean), PCM 12 (16k mono WoV). - Add build steps with alsatplg and verification runbooks. - doc/developer_guides/wov_ecns_integration_guide.md: - Update integration guide with dual-rate DMIC architecture and D0i3 verification. - README.md: - Update top-level README with microWakeWord and WoV documentation references. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…ECNS, KPB, and host copier Remove redundant mixin and mixout components between ECNS output pins and downstream consumers: - ECNS Output Pin 0 (16 kHz mono clean) connects directly to KPB (kpb.116.1 / kpb.106.1). - ECNS Output Pin 1 (48 kHz stereo clean) connects directly to the Host Copier capture endpoint (host-copier.11.capture / host-copier.10.capture). - Remove mixin.115.1, mixin.115.2, mixout.116.1, and mixout.117.1 from dmic-wov-multi.conf. - Remove mixin.105.1, mixin.105.2, mixout.106.1, and mixout.107.1 from standalone 4ch manifests (PTL, WCL, TGL). - Update prebuilt topology binaries and README topology diagrams and PCM tables. - Validated on Aphid hardware across 10x S0 and 10x D0i3 WoV capture tests (20/20 PASS). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…DMEs Synchronize Mermaid architecture diagrams to reflect direct widget routing: - Add dedicated Dual-Rate DMIC & Multi-Slot WoV flowchart to tools/topology/topology2/README.md. - Update src/audio/ecns/README.md, src/audio/wov_arbiter/README.md, and doc/developer_guides/wov_ecns_integration_guide.md to eliminate intermediate mixin.115.1/mixin.115.2 and mixout.116.1/mixout.117.1 widgets. - Update src/audio/microwakeword/README.md to eliminate intermediate mixin.105.1/mixin.105.2 and mixout.106.1/mixout.107.1 widgets. - Show direct ECNS Pin 0 connection to KPB and ECNS Pin 1 connection to Host Copier (PCM 11 / PCM 10). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
… 2-endpoint table Align standalone DMIC multi-slot Wake-on-Voice topology manifests (PTL, WCL, and TGL) to embed the full 2-endpoint NHLT table matching production SoundWire and platform topologies: - Define Object.Dai.DMIC with both Endpoint 0 (dmic01 @ 48 kHz, id 6) and Endpoint 1 (dmic16k @ 16 kHz, id 7) with 4-channel active PDM0/PDM1 mics. - Set Pipeline 100 DMIC_DAI_INDEX=1 and DMIC_NAME="dmic16k" so the 16 kHz capture copier matches Endpoint 1 in the topology-embedded NHLT table. - Generates the exact 6,211-byte, 2-endpoint NHLT binary table on PTL and WCL matching production nhlt-sof-ptl-rt721-4ch.bin bit-for-bit (md5: e16dcb62d2b3bacc939b24a8a0a9d0f0). - Generates matching 6,211-byte NHLT binary table on TGL with CAVS 2.5 driver version 1. - Update prebuilt topology binaries for PTL, WCL, and TGL in build/. - Update tools/wov_capture/run_wov_10x.py to dynamically auto-detect DMIC Multi-WOV capture PCM device index and active slot debug kcontrol names. - Update documentation in tools/topology/topology2/README.md. - Validated on Aphid hardware across 10x S0 and 10x D0i3 WoV capture tests (20/20 PASS). Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
…cation
In IPC4, the Linux kernel assigns module instance IDs sequentially as
it traverses DAPM widgets. Because widgets are stored in a LIFO linked list
(list_add), the widgets in pipelines 101, 102, 103 are instantiated in
reverse order:
- mww.103.1 -> instance 0
- mww.102.1 -> instance 1
- mww.101.1 -> instance 2
Previously, mww.c derived slot_id as (IPC4_INST_ID(dev->ipc_config.id) % 3),
which assigned slot 0 to mww.103.1 and slot 2 to mww.101.1. When the host
triggered wovdebug_101, the arbiter received slot 2 and set wov_active_slot
to 3 ('Slot 3'), inverting slots 1 and 3.
Align with detect_test.c by computing:
cd->wov_slot_id = (uint8_t)(2 - (IPC4_INST_ID(dev->ipc_config.id) % 3));
This ensures:
- wovdebug_101 (Pipeline 101) -> slot 0 -> wov_active_slot = 1 ('Slot 1')
- wovdebug_102 (Pipeline 102) -> slot 1 -> wov_active_slot = 2 ('Slot 2')
- wovdebug_103 (Pipeline 103) -> slot 2 -> wov_active_slot = 3 ('Slot 3')
Also update wov_blocking_read.c and run_wov_10x.py:
- wov_blocking_read queries wov_active_slot upon receiving audio frames
and asserts that triggered active_slot matches the expected slot.
- run_wov_10x.py logs and validates trig_slot == slot for each iteration.
- Validated on Aphid hardware across 10x S0 and 10x D0i3 runs (20/20 PASS).
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Immediate logging runs every log call synchronously in the caller's context, including the LL scheduler tick and DAI start. On ADL this stalls the DMIC capture pipeline start: SET_PIPELINE_STATE(RUNNING) times out and the DMIC GP-DMA overruns afterwards. Drop CONFIG_LOG_MODE_IMMEDIATE and go back to the default deferred mode. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
CONFIG_ZEPHYR_DP_SCHEDULER can be enabled on cAVS, but only the ACE platform_init() registered the DP scheduler. Creating a DP module on cAVS then failed with "unable to bind scheduler to task (type 3)" and module init returned -ENODEV. Call scheduler_dp_init() after the LL scheduler is set up, as ACE does. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The DP lock is taken by the LL thread on every tick in scheduler_dp_ll_tick() and by each DP thread after every run. It was a k_sem, which has no priority inheritance: when a DP thread holding it was preempted by other DP threads, the cooperative high priority LL thread blocked until the holder got CPU time again. With several busy DP modules (3 MFCC + 3 MWW on cAVS 2.5) single LL ticks took up to ~40 ms, overrunning the DMIC DMAs of unrelated streams. With a k_mutex the holder inherits the LL priority: LL max drops from ~1.5M to <30k timer cycles, with no LL overruns or DMA xruns. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
In IPC4 a buffer gets its stream params from the consuming module at bind time. Legacy comp_driver consumers don't do that, so a DP module feeding one (e.g. MWW to the WoV arbiter) saw rate and frame size 0 on its sink and module_adapter_calculate_dp_period() divided by zero. Before calculating the DP period, apply the module's own stream params to any sink buffer which has no rate yet. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The dual-rate multi-slot WoV pipelines declare 2ch DMIC DAI copiers for both dmic01 (48 kHz) and dmic16k (16 kHz). This matches the dmic-wov-feature topology, which splits PDM0/PDM1 between the two DAIs, but not dmic-generic.conf, where both DAIs enable all NUM_DMICS mics. On 4-mic platforms (e.g. sof-adl-max98357a-rt5682-wov-multi) the NHLT only carries 4ch DMIC blobs, so the kernel fails to start the ECNS capture with: no matching blob for sample rate: 16000 sample width: 16 channels: 2 Add DMIC_WOV_DAI_CHANNELS (default 2) and set it to 4 in dmic-generic when NUM_DMICS is 4. With 4 channels the DAI copiers take 4ch input and keep the 2ch output, letting the copier remap the first two mics to stereo for the ECNS, so the rest of the WoV graph is unchanged. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
ecns_prepare() forced the Pin 1 host output buffer to 4 channels whenever the 48 kHz dmic01 input was connected. With a 2-channel DMIC topology the host copier would then receive data labeled as 4-channel. Take the Pin 1 channel count from the dmic01 input buffer instead, so both 2 and 4 channel DMIC configurations work. Also replace the magic channel map and channel config values with named channel map macros and the IPC4 channel config enums. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The dmic01 DAI copier formats and the ECNS Pin 1 input buffer size were hardcoded for 4 channels. Use the WOVHOST channel definitions and a new ECNS_PIN1_IN_IBS that follow DMIC_WOV_DAI_CHANNELS, so the topology also builds correctly for a 2-channel DMIC. The channel config and map in the DET16K and WOVHOST defines are now literal values. The references to the CHANNEL_* defines resolved to 0, so the DAI copiers got channel map 0 and remapped every output channel from channel 0. This also fixes the dmic16k detection path, which had been delivering channel 0 on all channels. Add the sof-adl-max98357a-rt5682-2ch-wov-multi development target that uses only the PDM0 stereo microphones on Google Brya. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add 2-channel PCH DMIC variants of the PTL and WCL rt721 and rt722 multi-wov topologies for devices with only the PDM0 microphone pair. All of them embed the NHLT so the dmic16k DAI finds its blob. cavs-sdw.conf enables the PDM1 microphones by default, so the 2ch targets set PDM1_MIC_A_ENABLE=0, PDM1_MIC_B_ENABLE=0 and DMIC_WOV_DAI_CHANNELS=2 explicitly. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The dmic16k DAI copier advertised both 32-bit and 16-bit input formats. With the 16-bit format available the DMIC gateway on PTL was set up so that the DMA write position advanced erratically and the detector path received only about 87% of the expected 16 kHz frame rate, starving the MFCC and MWW slots. Advertise only the 32-bit format, as the other DMIC copiers do. The dmic16k path now delivers exactly 16000 frames per second end to end through ECNS and KPB. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
sof_init() runs at POST_KERNEL and calls idc_init() on core 0, which starts the core 0 IDC p4wq worker thread. The p4wq static init runs later at APPLICATION level by default, clears the queue and creates the thread in delayed-start state, so the core 0 IDC worker is never started. Secondary cores run idc_init() after power-up and are not affected. The problem shows up as soon as a secondary core notifies core 0, e.g. MWW on core 1 notifying KPB on detection: "Out of worker threads", the IDC times out and the next submit corrupts the p4wq queue. Select CONFIG_P4WQ_INIT_STAGE_EARLY for PTL and WCL. Verified on PTL with WoV detector slots on secondary cores. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
notifier_event() sends events to other cores as non-blocking IDC messages. Each target core has a single notify_data slot and the IDC work item is reused, so two events sent back to back to the same core, such as the MWW detection followed by the KPB drain request, can overwrite the first event's data or resubmit a work item that is still queued. Send remote notifications as blocking IDC so the next event to the same core is not issued before the previous one has been handled. This adds latency on the notifying core, as it now waits for the remote callback to complete. Verified on PTL with WoV detection on core 1 notifying the arbiter and KPB on core 0. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
When MWW runs on a different core than KPB, the keyword trigger passes a kpb_event_data whose client_data member points to a kpb_client written by the MWW core. notifier_event() only writes back the kpb_event_data itself, so KPB read a stale client and started draining with a zero length request, which never moved to host copy. Write back the client data in MWW before sending the notification and invalidate it in the KPB event handler before use. KPB now drains the requested 2000 ms of history after a keyword trigger. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The WoV arbiter consumes and discards everything arriving on its KPB input while no slot is active. MWW notified KPB to begin draining before notifying the arbiter of the detection, so the whole 2 s history was drained into the arbiter and dropped before it switched to the detecting slot. The captured stream then started with live audio after the keyword. Send the detection notification to the arbiter first and request KPB draining after it, both for real detections and the fake wake test path. The keyword is now present at the start of the captured stream. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The MWW TFLM inference takes most of a core per detector slot and it is not clear from the total alone which operators dominate. Add an optional MicroProfilerInterface implementation that accumulates the CCOUNT cycles of every operator in invoke order. When CONFIG_COMP_MWW_PROFILE is enabled, the per-operator averages and their total are logged with the stream shutdown summary. The first inference after a reset also runs the CALL_ONCE init subgraph and is excluded from the averages. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Profiling showed the MWW inference spending over 9M cycles per run in the portable TFLM reference kernels, most of it in CONV_2D and DEPTHWISE_CONV_2D, and in the QUANTIZE and DEQUANTIZE of the model's float streaming state. When building with xt-clang, compile these operators from the TFLM Xtensa kernels backed by the Cadence HiFi4 nnlib and link a private nnlib archive. nnlib exports a global gcd() that clashes with SOF's, so it is renamed at build time, and the float quantize path needs __fpclassifyf in the libc shim. Other toolchains keep the reference kernels. The new CONFIG_COMP_MWW_HIFI_NNLIB allows opting out. On PTL this reduces an inference from about 9.4M to 1.7M cycles. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The model output fluctuates strongly between consecutive inferences during a keyword, for example 97% followed by 23%, so requiring two consecutive inferences above 85% often missed clearly spoken keywords. Trigger instead when the average probability of the last four inferences reaches 50%. In a test with three repeated keywords the best four-inference averages were 56% to 67%, while other speech stayed below 25%. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
======================================================================
microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50)
======================================================================
Class Role Files Detected Rate Mean Peak
------------------------------------------------------------------
ambient Negative 1900 3 0.2% 0.003
unknown Negative 5000 17 0.3% 0.006
strawberry Positive 3000 3000 100.0% 0.996
------------------------------------------------------------------
>>> Confusion Matrix:
Actual Positive Actual Negative
Predicted Positive: TP = 3000 FP = 20
Predicted Negative: FN = 0 TN = 6880
>>> Performance Summary Metrics:
- Accuracy : 99.80%
- Precision : 99.34%
- Recall / True Positive Rate : 100.00% (3000/3000)
- False Positive / Alarm Rate : 0.29% (20/6900)
- F1-Score : 0.9967
======================================================================
======================================================================
microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50)
======================================================================
Class Role Files Detected Rate Mean Peak
------------------------------------------------------------------
ambient Negative 1900 3 0.2% 0.008
unknown Negative 5000 27 0.5% 0.011
banana Positive 3000 2981 99.4% 0.992
------------------------------------------------------------------
>>> Confusion Matrix:
Actual Positive Actual Negative
Predicted Positive: TP = 2981 FP = 30
Predicted Negative: FN = 19 TN = 6870
>>> Performance Summary Metrics:
- Accuracy : 99.51%
- Precision : 99.00%
- Recall / True Positive Rate : 99.37% (2981/3000)
- False Positive / Alarm Rate : 0.43% (30/6900)
- F1-Score : 0.9918
======================================================================
======================================================================
microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50)
======================================================================
Class Role Files Detected Rate Mean Peak
------------------------------------------------------------------
ambient Negative 1900 0 0.0% 0.001
unknown Negative 5000 36 0.7% 0.010
orange Positive 3000 2995 99.8% 0.995
------------------------------------------------------------------
>>> Confusion Matrix:
Actual Positive Actual Negative
Predicted Positive: TP = 2995 FP = 36
Predicted Negative: FN = 5 TN = 6864
>>> Performance Summary Metrics:
- Accuracy : 99.59%
- Precision : 98.81%
- Recall / True Positive Rate : 99.83% (2995/3000)
- False Positive / Alarm Rate : 0.52% (36/6900)
- F1-Score : 0.9932
======================================================================
Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
MFCC configured the PCAN gain control with zero input correction bits while the log scale stage used three. Google's microfrontend uses log2(fft_size) - 6 for both, which compensates the 1/N scale of its fixed-point FFT. The mismatch made the SOF PCAN features about 15 int8 steps lower on speech than the TFLM microfrontend features the microWakeWord models are trained with, and the models failed to detect keywords on the DSP. SOF compensates the FFT length in the Mel filter bank and scales the linear Mel magnitude to Google's 512-point range, so the correction is 3 for every FFT size. Add PCAN_INPUT_CORRECTION_BITS and use it for both stages. Also limit the linear Mel magnitude to 32 bits instead of 16, as the microfrontend filter bank output is not clamped to 16 bits. With the change the testbench features of recorded keywords differ from the microfrontend features by about 1.5 int8 steps on average. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The 4ch wov-multi targets did not set DMIC_WOV_DAI_CHANNELS, so dmic-default.conf left it at 2. Only the first two microphones were fed to ECNS, and its mono downmix for KPB and the detectors averaged just those two, giving a noisier detection input than the 4ch platforms allow. Set DMIC_WOV_DAI_CHANNELS=4 for the PTL and WCL DMIC, rt721, rt722 and HDA generic 4ch wov-multi targets and the generic DMIC one. ECNS pin 0 and the 48 kHz ECNS host capture then carry all four channels. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The detector input on the KPB path is quiet, with keyword peaks around -36 dBFS and room background around -57 dBFS. Replace the built-in 100 Hz high-pass coefficients for the pin 0 mono output with the +30 dB version from highpass_100hz_30db_16khz.conf to lift the detector and KPB history level by another 10 dB. The biquad coefficients are the same as in the +20 dB set, only the output shift (-5) and gain (0x7e33) differ. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Add ECNS + Wake-on-Voice instance on sdw headset microphone similar to existing DMIC WoV graph. Jack-In module copier(module-copier.11.0, 48 kHz 2ch) feeds ECNS pin 1 directly, ECNS pin0 through a 48k->16k SRC. ECNS pin 0 drives KPB -> 3 x (MFCC + MWW) -> WoV arbiter, as in the DMIC graph. The two ECNS outputs PCMs are: PCM 13 "Headset ECNS Capture" (48 kHz 2ch) PCM 14 "Headset Multi-WOV" (16 kHz 1ch) Pipeline reuses classes & UUIDs from dmic-wov-multi.conf Included by INCLUDE_SDW_JACK_WOV=true. Signed-off-by: Naveen Manohar <naveen.m@intel.com>
Enable headset ECNS & WoV for WCL RT722 4ch & 2ch multi-WoV topologies. Signed-off-by: Naveen Manohar <naveen.m@intel.com>
DMIC & headset WoV graphs share NOTIFIER_ID_WOV_DETECT, DSP memory only fits one set of detectors. The instance_count in the module manifest is not enforced for built-in modules, so opening a second WoV PCM while one is active creates 2nd arbiter and its detectors fail to allocate. Reject wov_arb_new() with an error when an arbiter instance already exists. Signed-off-by: Naveen Manohar <naveen.m@intel.com>
Poll for capture data before reading a keyword-gated PCM endpoint. Propagate poll & read failures instead of reporting empty captures as successful and handle interruption during requested shutdown. Test: Verified on WCL with a 3sec 16kHz 48kHz capture. ./wov_capture_app \ -D 0 -d 12 -c 1 -r 16000 -f S16_LE \ -t 3 -n 1 -F /data/local/tmp/wov_pcm12.wav -v Signed-off-by: Naveen Manohar <naveen.m@intel.com>
======================================================================
microWakeWord PCAN Streaming Evaluation Report (avg of 4 >= 0.50)
======================================================================
Class Role Files Detected Rate Mean Peak
------------------------------------------------------------------
ambient Negative 1900 5 0.3% 0.008
unknown Negative 5000 27 0.5% 0.010
hi_intel Positive 4980 4973 99.9% 0.995
------------------------------------------------------------------
>>> Confusion Matrix:
Actual Positive Actual Negative
Predicted Positive: TP = 4973 FP = 32
Predicted Negative: FN = 7 TN = 6868
>>> Performance Summary Metrics:
- Accuracy : 99.67%
- Precision : 99.36%
- Recall / True Positive Rate : 99.86% (4973/4980)
- False Positive / Alarm Rate : 0.46% (32/6900)
- F1-Score : 0.9961
======================================================================
Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The banana model trained on synthetic Piper speech does not detect real spoken "banana" reliably, because many synthetic samples are slurred towards "panana". The hi intel model is trained on real recordings from about 160 speakers and stays reliable in offline tests with room background, level changes and reverberation. Use the built-in hi intel model for slot 1 instead of banana and update the slot descriptions in the documentation. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
The ECNS high-pass filters have fixed gain, +30 dB on the Pin 0 KPB detection output and +20 dB on the Pin 1 host capture output. Add a capture boost switch control to select between these boosted responses and the same filters with 0 dB gain. The 0 dB coefficient sets are added as separate blobs, converted from the highpass_100hz_0db_16khz, highpass_40hz_0db_48khz and highpass_40hz_0db_16khz topology blobs. A change only moves the IIR coefficient pointers, keeping the filter delay lines, and it is applied in copy() and prepare() so it also takes effect during streaming. This keeps the freedom to select other responses later. Switch controls are dispatched by control index. The capture boost is the first control (index 0) and the test signal the second (index 1), since the test signal may be removed later. The firmware starts boosted, so topologies without the control keep the current behavior; with the control, the kernel sends its value (default off) when the widget is set up. In the wov-multi topologies the controls are named per ECNS instance, "DMIC ECNS boost" and "Jack In ECNS boost", and the test signal switches "DMIC ecns_kpb_test_signal" and "Jack In ecns_kpb_test_signal", so a topology with both the DMIC and the jack input ECNS has unique control names. Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com>
Some topologies are byte-for-byte identical to an existing one and only need to be reachable under a different name. Add a SYMLINKS list, processed alongside TPLGS in production/CMakeLists.txt, that creates a real symlink instead of building a duplicate topology. cmake -E copy_if_different dereferences symlinks when deploying the build output, turning them back into regular file copies. Switch to "cp -a" so the deploy step preserves symlinks as symlinks. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> (cherry picked from commit c22e996)
WCL DMIC function topologies are byte-for-byte identical to the PTL ones, only reachable under a different name. Drop the duplicated TPLGS entries and list them in SYMLINKS instead, so they are built as symlinks to the PTL topologies rather than separately compiled duplicates. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> (cherry picked from commit dd2eeae)
The WCL DMIC and RT721 multi-slot WoV topologies are byte-for-byte identical to the PTL ones, only reachable under a different name. Drop the duplicated TPLGS entries and list them in SYMLINKS instead, so they are built as symlinks to the PTL topologies rather than separately compiled duplicates. The WCL RT722 targets stay as separate TPLGS entries since they enable INCLUDE_SDW_JACK_WOV, which the PTL ones do not. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
The hda-generic naming convention declares in case NHLT blob is included: hda-generoc-[DSP plaform]- PTL and WCL are ACE3, they use the same blob for NHLT, the name of the topology must be: sof-hda-generic-ace3-4ch-wov-multi.tplg Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
platform/intel/wcl.conf was a verbatim copy of ptl.conf and the WCL
manifests only differed from the PTL ones by PLATFORM="wcl". WCL is an
ACE3 platform exactly like PTL; the SOF topology layer does not need a
separate platform key for it, as the existing sof-wcl-dmic-*-id* symlinks
already show.
Remove wcl.conf and the "wcl" IncludeByKey.PLATFORM entries, delete the
duplicated WCL DMIC multi-WoV manifests and generate sof-wcl-dmic-wov-multi
and sof-wcl-dmic-wov-multi-4ch as symlinks to the PTL topologies instead.
The WCL RT722 targets were the only ones enabling INCLUDE_SDW_JACK_WOV;
PTL and WCL are identical from the topology's point of view, so enable
it on the PTL RT722 targets and turn the WCL ones into symlinks as well.
Update the README to match, including the stale
sof-{ptl,wcl}-hda-generic-4ch-wov-multi target names which were renamed
to sof-hda-generic-ace3-4ch-wov-multi earlier.
Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
wov_capture_app used -c for the card and -C for the channel count, while tinycap (and arecord) use -D for the card/device and -c for the channel count. The subtle difference made it easy to pass the card number as the channel count and vice versa when switching between the tools. Switch to -D CARD and -c CHANS to match tinycap, and update the usage examples and the wov_arbiter README accordingly. Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
…rase Detect ID When both DMIC and SoundWire headset WoV paths are instantiated in the same topology (e.g. rt722 wov-multi), both wov-arbiter widgets previously generated an ALSA mixer enum control with the identical name 'wov_active_slot', causing a naming collision in the mixer. Disambiguate the active slot controls per input path: - Change default wov-arbiter enum name to 'DMIC Phrase Detect ID'. - Override wov-arbiter enum name in sdw-jack-wov-multi.conf to 'Jack In Phrase Detect ID'. - Update integration guides and documentation accordingly. Co-developed-by: Naveen Manohar <naveen.m@intel.com> Signed-off-by: Naveen Manohar <naveen.m@intel.com> Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Support unique phrase detect control names across the automated test harnesses and validation utilities: - In run_wov_10x.py, test_wov_no_irq.py, and wov_multi_slot_test.py: add find_active_slot_control() to auto-detect 'DMIC Phrase Detect ID', 'Jack In Phrase Detect ID', or fallback to 'wov_active_slot'. - In wov_multi_slot_test.py, add --control parameter to allow selecting a specific phrase detect control when multiple interfaces are present. - In wov_blocking_read.c, probe 'DMIC Phrase Detect ID', 'Jack In Phrase Detect ID', and 'wov_active_slot'. Co-developed-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com> Co-developed-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Seppo Ingalsuo <seppo.ingalsuo@linux.intel.com> Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces multi-slot Wake-on-Voice (WoV) with low-power D0i3 sleep/wake capability, hardware PDM DMIC input, and modular topology feature integration across Panther Lake (PTL), Wildcat Lake (WCL), and Tiger Lake (TGL).
Highlights
Hardware PDM DMIC Wake-on-Voice:
dmic16k, DAI index 1) directly from the SoC PDM decimator, avoiding software resampling.dmic01) and external SoundWire microphones (Capture-SmartMic).Multi-Slot WoV Pipeline Architecture:
DMIC Multi-WOV).D0i3 Low-Power Sleep & Wake:
capture_compatible_d0i3 1to maintain low-power DSP SRAM retention across host S0ix sleep.COMP_STATE_PREPAREand suppress telemetry during D0i3.Topology Integration & Modularity:
dmic-wov-feature.confallowing multi-slot WoV to be included as a feature module into functional topologies (INCLUDE_WOV=multi).sof-ptl-rt721-4ch-wov-multi,sof-ptl-rt722-4ch-wov-multi, and WCL equivalents.Toolchain & Compiler Fixes:
CALL8out-of-range relocation in the libc shim by unifying.textsection placement.ibuffer_allocator.ccto support both upstream TFLM and trees carrying Xtensa ICF vtable fixes.Validation
test_startandwov_blocking_read.alsatplg.