Repository navigation
Reserve vector capacity when array lengths are known - #739
Merged
Merged
Conversation
Pass array lengths through an optional reserve hook before parsing elements. Reserve storage for growable sequence containers and return reservation failures as parse errors. Fixed-capacity and associative containers keep their existing insertion behavior. Enable the hint in 12 format readers and Generic. Add per-reader move-count regression coverage and all-format benchmarks for large and many short vectors, including serialized size counters and CI summary entries. Validated 933 tests and 42 benchmark cases across all 14 formats in the all-format suite. Serialized sizes match the baseline. One existing UBJSON test needed a local -Wno-error=deprecated-declarations workaround with the Apple SDK; that build-only workaround is not part of this change.
Contributor
Author
|
CI failure is not related to my changes (some vcpkg issue) linux / (headers-C++20-gcc-14) (pull_request) |
Collaborator
|
@Altren no problem |
This was referenced Oct 15, 2026
Collaborator
|
@Altren thanks for the contribution. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Array deserialization currently grows vectors incrementally even when the reader knows the number of elements. Pass that count to an optional shared reserve hook before parsing. Reservation failures become parse errors; fixed-capacity and associative containers retain their insertion behavior.
This applies to Avro, Boost.Serialization, Cap'n Proto, CBOR, Cereal, FlexBuffers, JSON, MessagePack, TOML, UBJSON, YAML, YAS, and Generic. BSON and XML do not expose a direct element count, so their readers are unchanged and remain in the benchmark as controls.
The vectors_read benchmark covers all serialized formats. It compares a single vector of 1,000 or 1,000,000 floats with 10,000 vectors of 64 floats. Setup, schema creation, and full round-trip validation are outside the timed loop. Serialized byte counts are reported.
Validation:
cause 127 extra moves. These reallocations disappear with the change.
Performance for 10,000 vectors of 64 floats (CPU time; lower is better):
The largest measured gains are in FlexBuffers (1.29x), YAS (1.28x),
Boost.Serialization (1.22x), and Cereal (1.21x). Single million-element
vectors show little change (0.98–1.03x across formats); TOML and YAML do
not show a gain in these measurements. Small differences may be noise.
Serialized sizes are unchanged for every workload.
The benchmark is included in reflect-cpp-all-format-benchmarks and runs in
the Linux, macOS, and Windows CI benchmark summaries. Timing results are
informational rather than pass/fail thresholds. The move-count regression
tests deterministically check that vector growth does not add relocations.
Run:
./build/benchmarks/all/reflect-cpp-all-format-benchmarks --benchmark_filter=vectors_read