Repository navigation
audio: modules & math: add single-precision floating-point acceleration for processing modules - #11329
audio: modules & math: add single-precision floating-point acceleration for processing modules#11329lgirdwood wants to merge 15 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved build failures, memory-safety defects, and incorrect filter processing and format dispatch block approval.
40 open issues
Convert impulse-response coefficients numerically to float · New Missing shared float types and helpers break compilation · New Prevent out-of-range full-scale float conversion · New Keep DRC UUID definition visible to non-Zephyr trace code · New Validate EQ blob structure before parsing · New IPC4 EQ dispatch ignores FLOAT format · New Portable float FIR kernels missing on HiFi builds · New Reject nonpositive sizes before allocating response storage · New IPC4 EQ setup and dispatch ignore FLOAT · New Use size_t for acquisition sample counts · New Use size_t for source and sink sample counts · New Calculate chunks from current float cursors · New Validate selector channel index before storing · New IPC4 float selector is incorrectly guarded by S32 · New IPC4 SRC dispatch ignores FLOAT format · New Portable float SRC implementation missing on HiFi · New Unequal-channel TDFB pass overruns output buffer · New Reject unequal channels without blob routing · New Portable float TDFB kernel missing on HiFi builds · New Saturate positive full-scale before integer conversion · New
And 20 more that still need to be addressed.
What changed in this PR
Adds single-precision floating-point processing to SOF’s audio modules and filter primitives, targeting hardware-FPU pipelines.
Changes:
- Introduces float FIR/IIR kernels and PCM conversions.
- Extends filtering, mixing, gain control, routing, and resampling.
- Updates format dispatch, state allocation, and static-pipeline integration.
| File | Description |
|---|---|
| src/math/iir_df1_float.c | Float IIR initialization and coefficient conversion. |
| src/math/fir_float.c | Float FIR processing primitives. |
| src/math/CMakeLists.txt | Registers float math sources. |
| src/include/sof/math/iir_df1_float.h | Float IIR state and kernel. |
| src/include/sof/math/fir_float.h | Float FIR interface. |
| src/include/sof/audio/selector.h | Shared channel limits. |
| src/audio/volume/volume.c | Float zero-crossing detection and static initialization. |
| src/audio/volume/volume_generic.c | Float gain and passthrough kernels. |
| src/audio/tone/tone.h | Declares float tone processing. |
| src/audio/tone/tone.c | Float tone output. |
| src/audio/tone/tone-ipc4.c | Format negotiation and IPC compatibility changes. |
| src/audio/tone/tone-ipc3.c | Float format selection. |
| src/audio/tone/Kconfig | Relaxes tone dependencies. |
| src/audio/tone/CMakeLists.txt | Changes tone implementation selection. |
| src/audio/tdfb/tdfb.c | Float beamformer setup and dispatch. |
| src/audio/tdfb/tdfb_generic.c | Float beamforming kernel. |
| src/audio/tdfb/tdfb_comp.h | Float beamformer state. |
| src/audio/src/src_ipc3.c | Float SRC dispatch. |
| src/audio/src/src_generic.c | Float polyphase resampling. |
| src/audio/src/src_common.h | Declares float SRC processing. |
| src/audio/src/src_common.c | Float passthrough support. |
| src/audio/selector/selector.c | Enum controls and static registration. |
| src/audio/selector/selector_generic.c | Float channel routing. |
| src/audio/pcm_converter/pcm_converter_generic.c | Hardware-FPU PCM conversions. |
| src/audio/mux/mux_generic.c | Float routing entries. |
| src/audio/multiband_drc/multiband_drc.h | Float compressor state. |
| src/audio/multiband_drc/multiband_drc.c | Float filter allocation and setup. |
| src/audio/multiband_drc/multiband_drc_generic.c | Float multiband processing. |
| src/audio/mixin_mixout/mixin_mixout.c | Accepts float streams. |
| src/audio/mixin_mixout/mixin_mixout_generic.c | Float mixing and integer-loop unrolling. |
| src/audio/mixer/mixer_generic.c | Float summation and two-source optimization. |
| src/audio/level_multiplier/level_multiplier.h | Cached float gain. |
| src/audio/level_multiplier/level_multiplier.c | Initializes float gain. |
| src/audio/level_multiplier/level_multiplier-ipc4.c | Updates float gain from controls. |
| src/audio/level_multiplier/level_multiplier-generic.c | Float scaling kernel. |
| src/audio/eq_iir/eq_iir.h | Float EQ state and interfaces. |
| src/audio/eq_iir/eq_iir.c | Float EQ setup and processing. |
| src/audio/eq_iir/eq_iir_ipc3.c | Float EQ dispatch. |
| src/audio/eq_fir/eq_fir.h | Float FIR state and dispatch interfaces. |
| src/audio/eq_fir/eq_fir.c | Float coefficients and delay allocation. |
| src/audio/eq_fir/eq_fir_ipc3.c | Float FIR selection. |
| src/audio/eq_fir/eq_fir_generic.c | Float FIR stream processing. |
| src/audio/drc/drc.c | Alignment, UUID ownership, and diagnostics. |
| src/audio/drc/drc_log.c | Adjusts UUID declarations. |
| src/audio/drc/drc_generic.c | Float detector and compression paths. |
| src/audio/dcblock/dcblock.h | Float filter state and coefficients. |
| src/audio/dcblock/dcblock.c | Converts DC-block coefficients. |
| src/audio/dcblock/dcblock_generic.c | Float DC-block processing. |
| src/audio/crossover/crossover.h | Float crossover interfaces. |
| src/audio/crossover/crossover.c | Float crossover initialization. |
| src/audio/crossover/crossover_generic.c | Float band splitting and routing. |
| src/audio/asrc/asrc_farrow.h | Declares float ASRC filtering. |
| src/audio/asrc/asrc_farrow_generic.c | Adds float convolution kernel. |
Review effort: Balanced
4c5135d to
d2bfe5b
Compare
Upstream Float Processing Modules - CI & Review Fixes AppliedThank you for the thorough review! We have rebased and updated the branch, squashing targeted fixes directly into their respective commits and addressing all reported review feedback:
|
- Implement inline iir_df1_float() evaluating cascaded biquads via single-cycle FMA operations without register spills. - Provide helper calculators for delay and coefficient buffer sizing (iir_delay_size_df1_float, iir_coef_size_df1_float). - Convert and normalize biquad coefficients from SOF Q30/Q14 EQ blobs to IEEE 754 floats in iir_init_coef_df1_float(). Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
75347f4 to
bc316b8
Compare
CI Status Update & Fixes AppliedAll CI checks on head commit Fix Summary:
All 15 commits remain bisectable, cleanly structured, and signed off. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved build failures, state corruption, unsafe control handling, and incorrect processing dispatch block approval.
32 open issues
HiFi3 DRC backend mishandles float samples · New IPC4 EQ dispatch selects S32 for float streams · New Float DF1 helper lacks format guard · New FIR delay clearing corrupts alternate format metadata · New No-blob setup allows controls to dereference null config · New Fixed DF1 processing ignores grouped series behavior Restore required module adapter dependency IPC3 adapter lacks tone component initialization Portable float SRC implementation missing on HiFi IPC4 SRC dispatch ignores FLOAT format Calculate chunks from current float cursors Use size_t for source and sink sample counts Use size_t for acquisition sample counts IPC4 EQ setup and dispatch ignore FLOAT IPC4 EQ dispatch ignores FLOAT format Keep DRC UUID definition visible to non-Zephyr trace code Missing shared float types and helpers break compilation Convert impulse-response coefficients numerically to float IPC3 dispatch accepts float when disabled · New Q15 conversion loses quiet negative signal sign · New
And 12 more that still need to be addressed.
16 resolved since last review
Use size_t for S32 sink sample count Saturate positive full-scale before integer conversion Portable float TDFB kernel missing on HiFi builds Reject unequal channels without blob routing Unequal-channel TDFB pass overruns output buffer IPC4 float selector is incorrectly guarded by S32 Validate selector channel index before storing Reject nonpositive sizes before allocating response storage Portable float FIR kernels missing on HiFi builds Validate EQ blob structure before parsing Prevent out-of-range full-scale float conversion Include float builds in mux and demux guards Release source buffer when sink acquisition fails Static gain callbacks fail to update float gain Handle FLOAT explicitly in EQ processor selection Float DRC builds lack guarded delay helper
Review effort: Balanced
- Add single-precision floating-point FIR filter state structure (struct fir_state_float) and circular delay line tracking. - Implement fir_float() single-sample processing kernel. - Implement 2-way unrolled fir_float_2x() processing kernel to maximize instruction-level parallelism (ILP) on pipelined FPU architectures. - Add coefficient initialization and scaling from struct sof_fir_coef_data. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
…rsions - Under CONFIG_FPU, accelerate S16, S24, and S32 integer <-> float conversions using linear scaling and single-cycle hardware floating-point operations. - On RISC-V targets with FPU, utilize fcvt.w.s with round-to-nearest-even (rne) for optimal cycle efficiency and bit-exact rounding. - Fallback gracefully to integer soft-float emulation when CONFIG_FPU is disabled. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
…ssing detection - Implement vol_zc_get_float() to detect zero crossings in single-precision floating-point circular stream buffers, preventing clicks during ramping. - Add vol_float_to_float() generic gain scaling routine across channels. - Add vol_passthrough_float_to_float() zero-overhead buffer copy path. - Register SOF_IPC_FRAME_FLOAT in volume_func_map and zc_func_map. - Add static component synthesis for IPC3 volume configurations. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement eq_iir_setup_float() allocating delay and coefficient memory and initializing per-channel biquad states using iir_df1_float. - Implement eq_iir_float_default() evaluating cascaded biquads over circular buffers with single-cycle FMA operations. - Update struct comp_data to store float delay/coef states under CONFIG_FORMAT_FLOAT. - Add SOF_IPC_FRAME_FLOAT to function map tables (fm_configured, fm_passthrough) and handle float format routing in eq_iir_new_blob(). Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement drc_float_default() processing function for SOF_IPC_FRAME_FLOAT. - Add float detector average calculation in drc_update_detector_average(). - Apply normalized float gain scaling in drc_compress_output() delay lines. - Add SOF_IPC_FRAME_FLOAT to drc_proc_fnmap table. - Support static UUID linkage when building without loadable modules. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement dcblock_generic_float() evaluating 1st-order DC rejection filter y[n] = x[n] - x[n-1] + R * y[n-1] with float coefficients and delay state. - Add dcblock_float_default() stream processing routine over circular buffers. - Add float state union and float coefficient storage in comp_data. - Register SOF_IPC_FRAME_FLOAT in dcblock_fnmap. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement eq_fir_setup_float() allocating float coefficient/delay buffers and parsing struct sof_fir_coef_data into float FIR filter state. - Add eq_fir_float() and unrolled eq_fir_2x_float() circular buffer processing routines. - Integrate set_float_fir() and wire SOF_IPC_FRAME_FLOAT in set_fir_func(). - Free allocated float delay and coefficient memory in eq_fir_free_delaylines(). Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement crossover_init_coef_lr4_float() to calculate normalized float biquad pairs for Linkwitz-Riley 4th order (LR4) filters. - Add generic float splitting routines (2-way, 3-way, 4-way). - Add crossover_float_default() processing loop for multi-sink buffer distribution. - Register SOF_IPC_FRAME_FLOAT in crossover_proc_fnmap. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement multiband_drc_float_default() processing pipeline integrating float crossover splitting, emphasis/deemphasis IIR filters, and per-band DRCs. - Add multiband_drc_eq_init_coef_ch_float() and float state tracking in comp_data. - Add multiband_drc_float_process_drc() handling delay lines and envelope tracking. - Register SOF_IPC_FRAME_FLOAT in multiband_drc_proc_fnmap. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement mix_n_float() summing multiple float sources into one sink with 4-sample loop unrolling for instruction-level parallelism. - Add mix_float() and mix_float_gain() in mixin_mixout. - Add SOF_IPC_FRAME_FLOAT mappings across mixer_func_map, mix_func_map, and mux_func_map. - Optimize 2-source fixed-point mixing with 4-sample vector unrolling. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement tdfb_core_float() and tdfb_fir_float() using fir_float_2x() to evaluate 2-sample unrolled multi-channel FIR filter banks. - Add float FIR state union (struct fir_state_float fir_f) and in_f/out_f buffers. - Allocate and initialize float filter bank coefficients in tdfb_init_coef(). - Wire SOF_IPC_FRAME_FLOAT in set_func() and set_pass_func(). Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement fir_filter_generic_float() evaluating polyphase subfilters with float coefficients and normalized scaling for mono and stereo streams. - Implement src_polyphase_stage_cir_float() circular buffer polyphase stage. - Wire SOF_IPC_FRAME_FLOAT in src_prepare_general() and src_copy_sxx(). Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement tone_float_default() generating normalized single-precision floating-point sine audio samples. - Support SOF_IPC_FRAME_FLOAT across tone_params() and tone-ipc3/ipc4. - Allow tone compilation across IPC3 and IPC4 configurations. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
- Implement level_multiplier_float() with 4-sample unrolled gain multiplication. - Add float gain state (gain_f) and automatic Q-format to float translation. - Implement process_frame_float() and sel_float() matrix routing for selector. - Add ALSA enum control handling (COMP_CMD_SET_VALUE/COMP_CMD_GET_VALUE) in selector. - Wire SOF_IPC_FRAME_FLOAT in level_multiplier_proc_fnmap and selector func_table. Signed-off-by: Liam Girdwood <lgirdwood@gmail.com>
bc316b8 to
4edee5e
Compare
Update: Additional Review Comments Addressed & SquashedAll items raised in the latest review feedback have been addressed and squashed cleanly into their respective commits in the series (HEAD:
|


Overview
This pull request introduces single-precision IEEE 754 floating-point (
SOF_IPC_FRAME_FLOAT/CONFIG_FORMAT_FLOAT) acceleration across SOF core audio processing modules and mathematical filter primitives.SOF firmware has historically focused on fixed-point arithmetic (
SOF_IPC_FRAME_S16_LE,SOF_IPC_FRAME_S24_4LE,SOF_IPC_FRAME_S32_LE) and Xtensa HiFi SIMD intrinsics. With the introduction of declarative static pipeline architectures (CONFIG_STATIC_PIPELINE) on modern microcontroller and DSP SoCs featuring single-precision hardware FPUs—such as the Espressif ESP32-P4 (RV32IMAFC/ilp32f@ 400 MHz) and the PJRC Teensy 4.1 (NXP i.MX RT1062 Cortex-M7 with FPv5-D16 @ 600 MHz)—native floating-point execution delivers dramatic cycle efficiency and audio quality benefits:fmadd.son RISC-V,VFMA.F32on Cortex-M7) in a single cycle without register spilling or 64-bit multi-cycle software division/emulation.Hardware Benchmark Profile (ESP32-P4 HP Core @ 400 MHz, 48 kHz Stereo, 1 ms Period)
eq_iirdrcvolumeeq_firtdfbasrccrossovermultiband_drctonelevel_multiplierselectordcblockmixerSeries Commits
math: add single-precision float Direct Form 1 (DF1) IIR biquad routinesmath: add single-precision float FIR filtering routinesaudio: pcm_converter: add hardware FPU accelerated float format conversionsaudio: volume: add single-precision float volume scaling and zero-crossing detectionaudio: eq_iir: add single-precision float 4-band parametric IIR EQaudio: drc: add single-precision float Dynamic Range Compressoraudio: dcblock: add single-precision float filter accelerationaudio: eq_fir: add single-precision float FIR filtering supportaudio: crossover: add float Linkwitz-Riley 4th order processingaudio: multiband_drc: add single-precision float multiband compressoraudio: mixer: add float stream mixing, muxing, and mixin/mixout supportaudio: tdfb: add single-precision float beamformer accelerationaudio: src: add single-precision float polyphase sample rate conversionaudio: asrc: implement float FIR filtering and 4-way unrolled ILPaudio: tone: add single-precision float synthesis and format negotiationaudio: level_multiplier & selector: add float scaling and matrix routingCompatibility & Safety
#if CONFIG_FORMAT_FLOATand conditionally activated whenSOF_IPC_FRAME_FLOATis negotiated.CONFIG_FORMAT_FLOATor targets configured with fixed-point streams (S16_LE,S24_4LE,S32_LE) continue executing unchanged.checkpatch.pl.