Skip to content

[SDK] Fix BatchLogRecordProcessor batching and flush behavior - #4743

Merged
marcalff merged 4 commits into
open-telemetry:mainfrom
yswdqz:fix/blrp-strict-batch
Oct 30, 2026
Merged

marcalff merged 4 commits into
open-telemetry:mainfrom
yswdqz:fix/blrp-strict-batch

Conversation

@yswdqz

@yswdqz yswdqz commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Fixes #4498

Changes

Align BatchLogRecordProcessor batching behavior with the
BatchSpanProcessor fix in #4466.

  • Export at most one batch per normal worker iteration instead of
    draining the queue in a tight loop.
  • Wait for a full batch unless the scheduled delay expires or an
    explicit wakeup occurs. Existing queue-pressure wakeups are preserved.
  • Snapshot the buffered record count for ForceFlush and shutdown,
    exporting records in batches capped by max_export_batch_size.
  • Avoid treating subsequent exports as force-flush operations after
    a previous ForceFlush has completed.
  • Add regression tests for full-batch waiting, half-full queue wakeups,
    flush/shutdown batch limits, and behavior after ForceFlush.

No public API changes.

Testing

Validated locally on macOS with AppleClang, C++17, and a Debug build:

  • All 18 BatchLogRecordProcessor tests passed.
  • clang-format 18 checks passed for the modified C++ files.
  • markdownlint-cli 0.46.0 passed for CHANGELOG.md.

The full CI matrix has not been run locally.

Checklist

  • CHANGELOG.md updated for non-trivial changes
  • Unit tests have been added
  • Changes in public API reviewed — no public API changes

…ing partial batches

Align the logs batch processor with BatchSpanProcessor after open-telemetry#4466:

* The worker waits until max_export_batch_size records are buffered before
  exporting on a regular wakeup.
* ForceFlush()/Shutdown() drain everything buffered, split into batches of at
  most max_export_batch_size.
* ForceFlush() returns only after all records buffered before the call have
  been exported.

Fixes open-telemetry#4498
@yswdqz
yswdqz requested a review from a team as a code owner October 28, 2026 07:08

@owent owent left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.51%. Comparing base (3ee38aa) to head (f3f3dbb).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #4743      +/-   ##
==========================================
- Coverage   86.51%   86.51%   -0.00%     
==========================================
  Files         530      530              
  Lines       21138    21136       -2     
==========================================
- Hits        18286    18284       -2     
  Misses       2852     2852              
Files with missing lines Coverage Δ
sdk/src/logs/batch_log_record_processor.cc 91.01% <100.00%> (-0.09%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mateenali66

Copy link
Copy Markdown
Member

checked whether this needs the locked notify from #4741, since my note there said logs would once the full-batch wait landed. it doesn't. OnEmit sets is_force_wakeup_background_worker before it notifies, so a half-full notify ends the wait instead of re-parking on a partial batch the way the span worker does.

stress probe with queue 2, batch 2 and a 10 min delay, one emit, a swept spin, then two more emits, 50k rounds per run. the worker sleeps through a full queue in 15 and 19 rounds on main and in 15 and 17 at d23cb0c, so that stall is older than this PR and unchanged by it. the #4741 change mirrored into OnEmit takes it to 0 of 50k. that fits a follow-up once both merge, not a blocker here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[SDK] BatchLogRecordProcessor still drains the queue in a tight loop, inconsistent with BatchSpanProcessor fix

4 participants