From 80b120b44f134626d690d7143207416c8274678d Mon Sep 17 00:00:00 2001 From: Daniel Lemire Date: Thu, 19 Nov 2026 21:45:38 -0500 Subject: [PATCH] [opt](load) Read JSON documents straight into the padded buffer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### Background The v2 JSON reader (`be/src/format_v2/json/json_reader.cpp`) parses each document with the simdjson On-Demand parser, which needs the input followed by `SIMDJSON_PADDING` (64) readable bytes. The reader keeps a reusable `_padding_buffer` for that. In routine load and stream load (`FILE_STREAM`), a document is one pipe message, or several chunks of a chunked HTTP transfer. When `read_json_by_line` is false, a document is the whole file split. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Performance Problem Summary: Pipe messages and whole-file documents were first copied into `_document_buffer`, then copied again into `_padding_buffer` before parsing. Whole-file reads also zero-filled a freshly allocated `_document_buffer` the size of the file. In addition, `open()` zero-filled an 8 MB `_padding_buffer` in every reader, even for small documents. This PR reads and copies documents straight into `_padding_buffer`, and grows that buffer to the largest document instead of allocating 8 MB up front. Results and error handling are unchanged. ### Details - **Why it was slow:** The cost follows from the code. For each pipe message, `_read_one_document_from_pipe` does `_document_buffer.assign(...)`, then `_parse_next_json` does `memcpy(_padding_buffer, _document_view, size)`. For a whole-file document, `_document_buffer.resize(n)` allocates and zeroes `n` bytes, `read_at` fills them, and the same `memcpy` follows. `open()` runs `_padding_buffer.resize(8 MB + 64)` every time. - **Optimization:** `_read_one_document_from_pipe` copies messages, and appends chunks, directly into `_padding_buffer`. Whole-file mode calls `read_at` directly into `_padding_buffer`. `_parse_next_json` copies only when the document is not already at the start of the buffer. That is line mode, where the line reader's span is still borrowed, and a skipped UTF-8 BOM, where it uses `memmove` because the ranges overlap. `_document_buffer` and `_padded_size` are removed. The buffer grows with `std::string::resize`, which keeps the contents and grows capacity geometrically. - **Results:** A standalone program reproduces the old and new buffer handling with simdjson On-Demand and a full traversal of every value. These are not Doris end-to-end numbers. Apple M4, clang, `-O3`, best of 30 iterations, two runs (range shown): | Workload | Before | After | |---|---:|---:| | Pipe, 20,000 messages of 89 B | 1.05–1.07 GB/s | 1.15–1.16 GB/s | | Pipe, 2,000 messages of 6.3 KB (twitter statuses) | 3.42 GB/s | 3.56–3.58 GB/s | | Whole file, twitter.json (632 KB) | 2.93–3.04 GB/s | 3.28–3.30 GB/s | | Whole file, citm_catalog.json (1.7 MB) | 3.28–3.36 GB/s | 3.75–3.96 GB/s | | Whole file, 40 MB array | 1.93–1.95 GB/s | 2.16–2.19 GB/s | A new reader's 8 MB zero-fill costs about 49 µs on the same machine, while a 1 KB document parses in about 1 µs. - **Applicability and cost:** Applies to `FILE_STREAM` (routine load, stream load) and to whole-file JSON. Line mode (`read_json_by_line`) keeps its single copy and only benefits from the smaller initial allocation. The padding buffer is now sized to the largest document seen by the reader, plus 64 bytes. Before, it was at least 8 MB. A reader that sees a large document grows the buffer once, as before. No new config. ### Testing - `be/test/format_v2/json/json_reader_test.cpp`: the line-mode test now checks that the document is still borrowed from the line reader and that the padding buffer fits the longest line plus padding (not 8 MB). The whole-file outer-array test checks that the document is read in place and that the buffer is the file size plus padding. - The buffer logic (chunked pipe appends across growth, BOM in whole-file and line mode, a long line after short ones) was checked in a standalone program under ASan and UBSan. - NOT RUN: the Doris BE build and `run-be-ut.sh`. I could not build the BE locally. ### Release note Faster JSON routine load, stream load, and whole-file JSON load, with less memory per JSON reader. ### Check List (For Author) - Behavior changed: No - Does this need documentation: No - Pre-submission status: Partially verified: not built in the Doris tree and BE unit tests not run; numbers come from a standalone emulation, not a Doris benchmark. --- be/src/format_v2/json/json_reader.cpp | 43 ++++++++++++--------- be/src/format_v2/json/json_reader.h | 9 +++-- be/test/format_v2/json/json_reader_test.cpp | 16 ++++++-- 3 files changed, 44 insertions(+), 24 deletions(-) diff --git a/be/src/format_v2/json/json_reader.cpp b/be/src/format_v2/json/json_reader.cpp index 3a3e2fe361570f..4c226f6e5b355b 100644 --- a/be/src/format_v2/json/json_reader.cpp +++ b/be/src/format_v2/json/json_reader.cpp @@ -359,7 +359,6 @@ Status JsonReader::open(std::shared_ptr request) { } RETURN_IF_ERROR(_parse_jsonpath_and_json_root()); _json_parser = std::make_unique(); - _padding_buffer.resize(_padded_size); _reader_eof = false; _single_document_read = false; _eof = false; @@ -653,11 +652,11 @@ Status JsonReader::_read_one_document(size_t* size, bool* eof) { *eof = true; return Status::OK(); } - _document_buffer.resize(cast_set(read_size)); - Slice result(_document_buffer.data(), _document_buffer.size()); + const auto document_size = cast_set(read_size); + _grow_padding_buffer(document_size); + Slice result(_padding_buffer.data(), document_size); RETURN_IF_ERROR(_physical_file_reader->read_at(_current_offset, result, size, _io_ctx.get())); - _document_buffer.resize(*size); - _document_view = _document_buffer; + _document_view = std::string_view(_padding_buffer.data(), *size); if (*size == 0) { *eof = true; } @@ -672,12 +671,12 @@ Status JsonReader::_read_one_document_from_pipe(size_t* read_size) { DorisUniqueBufferPtr file_buf; RETURN_IF_ERROR(stream_load_pipe->read_one_message(&file_buf, read_size)); if (*read_size == 0) { - _document_buffer.clear(); _document_view = {}; return Status::OK(); } - _document_buffer.assign(reinterpret_cast(file_buf.get()), *read_size); - _document_view = _document_buffer; + _grow_padding_buffer(*read_size); + std::memcpy(_padding_buffer.data(), file_buf.get(), *read_size); + _document_view = std::string_view(_padding_buffer.data(), *read_size); if (!stream_load_pipe->is_chunked_transfer()) { return Status::OK(); } @@ -689,13 +688,21 @@ Status JsonReader::_read_one_document_from_pipe(size_t* read_size) { if (next_size == 0) { break; } - _document_buffer.append(reinterpret_cast(next_buf.get()), next_size); + _grow_padding_buffer(*read_size + next_size); + std::memcpy(_padding_buffer.data() + *read_size, next_buf.get(), next_size); *read_size += next_size; } - _document_view = _document_buffer; + _document_view = std::string_view(_padding_buffer.data(), *read_size); return Status::OK(); } +void JsonReader::_grow_padding_buffer(size_t size) { + const size_t needed = size + simdjson::SIMDJSON_PADDING; + if (_padding_buffer.size() < needed) { + _padding_buffer.resize(needed); + } +} + Status JsonReader::_parse_next_json(size_t* size, bool* eof) { RETURN_IF_ERROR(_read_one_document(size, eof)); if (*eof || *size == 0) { @@ -707,17 +714,17 @@ Status JsonReader::_parse_next_json(size_t* size, bool* eof) { _document_view.remove_prefix(3); *size -= 3; } - if (*size + simdjson::SIMDJSON_PADDING > _padded_size) { - _padded_size = *size + simdjson::SIMDJSON_PADDING; - _padding_buffer.resize(_padded_size); - } // Ondemand values reference the input buffer. Keep the padded bytes in a member buffer until the - // current document is fully materialized. - std::memcpy(_padding_buffer.data(), _document_view.data(), *size); + // current document is fully materialized. File and pipe documents are already there, at offset + // 3 when a BOM was skipped, so the ranges may overlap. + if (_document_view.data() != _padding_buffer.data()) { + _grow_padding_buffer(*size); + std::memmove(_padding_buffer.data(), _document_view.data(), *size); + } _original_doc_size = *size; + const std::string_view document(_padding_buffer.data(), *size); const auto error = - _json_parser->iterate(std::string_view(_padding_buffer.data(), *size), _padded_size) - .get(_original_json_doc); + _json_parser->iterate(document, _padding_buffer.size()).get(_original_json_doc); if (error != simdjson::error_code::SUCCESS) { return Status::DataQualityError( "Parse json data for JsonDoc failed. code: {}, error info: {}", error, diff --git a/be/src/format_v2/json/json_reader.h b/be/src/format_v2/json/json_reader.h index 2ac77c2242c2c1..13baa533c9d8e9 100644 --- a/be/src/format_v2/json/json_reader.h +++ b/be/src/format_v2/json/json_reader.h @@ -111,7 +111,10 @@ class JsonReader final : public FileReader { Status close() override; #ifdef BE_TEST - size_t TEST_document_buffer_size() const { return _document_buffer.size(); } + bool TEST_document_in_padding_buffer() const { + return !_document_view.empty() && _document_view.data() == _padding_buffer.data(); + } + size_t TEST_padding_buffer_size() const { return _padding_buffer.size(); } #endif private: @@ -139,6 +142,8 @@ class JsonReader final : public FileReader { // Chunked HTTP transfers concatenate their pipe buffers into one document. Status _read_one_document(size_t* size, bool* eof); Status _read_one_document_from_pipe(size_t* read_size); + // Grows `_padding_buffer` to hold `size` bytes plus simdjson padding, keeping its contents. + void _grow_padding_buffer(size_t size); // Moves the logical document into a simdjson-padded buffer and creates an ondemand document. Status _parse_next_json(size_t* size, bool* eof); // Applies json_root and validates the object/array shape required by strip_outer_array. @@ -223,11 +228,9 @@ class JsonReader final : public FileReader { simdjson::ondemand::value _json_value; simdjson::ondemand::array _array; simdjson::ondemand::array_iterator _array_iter; - std::string _document_buffer; std::string_view _document_view; std::string _padding_buffer; size_t _original_doc_size = 0; - size_t _padded_size = 1024 * 1024 * 8 + simdjson::SIMDJSON_PADDING; SimdJsonReadContext _simdjson_context; }; diff --git a/be/test/format_v2/json/json_reader_test.cpp b/be/test/format_v2/json/json_reader_test.cpp index 8d33c0b13cab63..e072f690118432 100644 --- a/be/test/format_v2/json/json_reader_test.cpp +++ b/be/test/format_v2/json/json_reader_test.cpp @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -262,7 +263,8 @@ struct ReadResult { bool eof = false; size_t second_rows = 0; bool second_eof = false; - size_t document_buffer_size = 0; + bool document_in_padding_buffer = false; + size_t padding_buffer_size = 0; int64_t rejected_rows = 0; std::map> missing_fields; std::vector schema; @@ -310,7 +312,8 @@ ReadResult read_once(const std::string& file_name, const std::string& content, result.block = make_block(result.schema, requested_local_ids); result.status = reader.get_block(&result.block, &result.rows, &result.eof); - result.document_buffer_size = reader.TEST_document_buffer_size(); + result.document_in_padding_buffer = reader.TEST_document_in_padding_buffer(); + result.padding_buffer_size = reader.TEST_padding_buffer_size(); result.rejected_rows = io_ctx->load_filtered_rows; result.missing_fields = reader.missing_fields(); if (result.status.ok() && read_twice) { @@ -687,7 +690,10 @@ TEST(JsonReaderTest, ReadsRequestedColumnsInFileScanRequestOrder) { ASSERT_TRUE(result.second_status.ok()) << result.second_status.to_string(); EXPECT_EQ(result.second_rows, 0); EXPECT_TRUE(result.second_eof); - EXPECT_EQ(result.document_buffer_size, 0); + EXPECT_FALSE(result.document_in_padding_buffer); + // The buffer fits the longest line instead of a fixed 8 MB. + EXPECT_EQ(result.padding_buffer_size, + std::string_view(R"({"id":1,"name":"alice"})").size() + simdjson::SIMDJSON_PADDING); } TEST(JsonReaderTest, HiveColumnLookupIsCaseInsensitiveWithoutNormalizedKeys) { @@ -716,6 +722,10 @@ TEST(JsonReaderTest, ReadsSingleDocumentOuterArray) { EXPECT_EQ(nullable_string_at(*result.block.get_by_position(1).column, 0), "carol"); EXPECT_EQ(nullable_int_at(*result.block.get_by_position(0).column, 1), 4); EXPECT_EQ(nullable_string_at(*result.block.get_by_position(1).column, 1), "dave"); + EXPECT_TRUE(result.document_in_padding_buffer); + EXPECT_EQ(result.padding_buffer_size, + std::string_view(R"([{"id":3,"name":"carol"},{"id":4,"name":"dave"}])").size() + + simdjson::SIMDJSON_PADDING); } TEST(JsonReaderTest, ReadsJsonRootByLine) {