Inflate OASIS CBLOCKs in one go with zlib

[nikosavola]

• OASIS CBLOCK records were decompressed by the built-in DEFLATE decoder one bit at a time, with a modulo and an assert per output byte; reading a CBLOCK-compressed file took twice as long as reading the same file uncompressed
• the reader now passes the CBLOCK byte counts to a new tl::InputStream::inflate_block, which decompresses the whole block with zlib (already linked for writing) and serves reads from memory; the streaming decoder stays for other inflate users and for comp-byte-count = 0
• the counts are treated as leniently as before: comp-byte-count decides where the next record starts (padding after the DEFLATE stream is skipped), uncomp-byte-count is only a hint for the initial buffer size, which grows as needed up to 1 GiB
• InputStream::get has an inline fast path; the OASIS varint readers are one template per signedness and decode from the buffer directly when enough bytes are available (C++11, with [[likely]] under C++20)
• pos () now stays at the end of a CBLOCK while its content is read, so error messages for records inside a CBLOCK report the block end
• new tests: block inflate round trips, padding and wrong size hints, truncated input, peek/unget at the block end, and an OASIS round trip with CBLOCKs through a gzipped file

benchmark (1 thread)	before (ms)	after (ms)	speedup
oasis_read_vexriscv_compressed	138.69 ± 1.42	75.71 ± 1.01	1.83x
oasis_read_vexriscv (no CBLOCKs)	67.77 ± 1.30	68.59 ± 1.18	0.99x
gds2_read_vexriscv	134.85 ± 0.48	128.46 ± 0.79	1.05x

After the change the CBLOCK file reads about as fast as the uncompressed one; the rest is parsing. The table was measured on the first version of this change, before the lenient count handling and the varint templates; those changed error handling and code structure but not the decoding path.

The benchmarks use the Google Benchmark suite from parallelize-rebased on my fork, which is not part of this PR. Measured on the parallelize branch, where the changed code is the same as on master. Numbers: 2x Intel Xeon Gold 6148 (40 cores, exclusive node), g++ 13 -O2, 20 runs each, mean ± standard deviation.

Benchmark code

//  Reads the vexriscv layout back from a file written once in the given format;
//  the CBLOCK variant exercises the built-in inflate decoder of the OASIS reader
void layout_read_benchmark (benchmark::State &state, const char *format, bool compressed_oasis)
{
  if (! prepare_benchmark (state)) {
    return;
  }

  const std::string layout_path = locate_testdata_file ("vexriscv.oas.gz");
  if (layout_path.empty ()) {
    state.SkipWithError ("vexriscv test data not found - set TESTSRC to the repository root or run from within the repository");
    return;
  }

  std::string error;
  db::Layout *layout = cached_layout (layout_path, error);
  if (! layout) {
    state.SkipWithError (("cannot read the layout: " + error).c_str ());
    return;
  }

  db::SaveLayoutOptions options;
  options.set_format (format);
  if (std::string (format) == "OASIS") {
    //  CBLOCKs are on by default, so the plain variant has to switch them off
    db::OASISWriterOptions &oasis_options = options.get_options<db::OASISWriterOptions> ();
    oasis_options.write_cblocks = compressed_oasis;
    if (compressed_oasis) {
      oasis_options.compression_level = int (db::max_oasis_compression_level);
    }
  }

  try {

    tl::TemporaryFile tmp ((std::string ("vexriscv_read_") + format));
    {
      tl::OutputStream stream (tmp.path ());
      db::Writer writer (options);
      writer.write (*layout, stream);
    }
    state.counters ["input_kb"] = file_size_kb (tmp.path ());

    size_t shapes = 0;
    for (auto _ : state) {
      db::Layout ly;
      tl::InputStream stream (tmp.path ());
      db::Reader reader (stream);
      reader.read (ly);
      shapes = 0;
      for (db::Layout::const_iterator c = ly.begin (); c != ly.end (); ++c) {
        for (db::Layout::layer_iterator l = ly.begin_layers (); l != ly.end_layers (); ++l) {
          shapes += c->shapes ((*l).first).size ();
        }
      }
    }
    state.counters ["shapes"] = double (shapes);

  } catch (tl::Exception &ex) {
    state.SkipWithError (("stream reading failed: " + ex.msg ()).c_str ());
  } catch (std::exception &ex) {
    state.SkipWithError (("stream reading failed: " + std::string (ex.what ())).c_str ());
  }
}

void gds2_read_vexriscv (benchmark::State &state)
{
  layout_read_benchmark (state, "GDS2", false);
}

void oasis_read_vexriscv (benchmark::State &state)
{
  layout_read_benchmark (state, "OASIS", false);
}

void oasis_read_vexriscv_compressed (benchmark::State &state)
{
  layout_read_benchmark (state, "OASIS", true);
}

BENCHMARK (gds2_read_vexriscv)->UseRealTime ()->Unit (benchmark::kMillisecond);
BENCHMARK (oasis_read_vexriscv)->UseRealTime ()->Unit (benchmark::kMillisecond);
BENCHMARK (oasis_read_vexriscv_compressed)->UseRealTime ()->Unit (benchmark::kMillisecond);

This PR was carried out with AI assistance (open-weight GLM 5.3) and reviewed before submission.

[klayoutmatthias]

Does it work with double .gz OASIS? CBLOCK inside and .gz outside? Such files exist.

And the patch is too big for my taste. I don't want to review it.

[nikosavola]

Double-compressed files work: the .gz layer is a delegate in front (InputZLibFile), and inflate_block pulls the CBLOCK bytes through it with its raw reads. TEST(CBlockGzip) in this PR covers exactly that case: the same CBLOCK layout read plain, then again through a .oas.gz wrapper, both compared against the original layout.

Sorry about the size. I can split it into independent pieces instead: (1) the inline fast path on tl::InputStream::get, (2) the varint readers decoding from the buffered bytes, (3) inflate_block with the CBLOCK byte counts and the reader-side handling. (1) and (2) are small on their own and useful for every streamer, not just OASIS. Would that work for you?

[nikosavola]

Closing this one as it is too big to review, as you said. I will split it into smaller PRs. #2505 is the first one, the inline fast path on InputStream::get. The varint readers and the CBLOCK inflate will follow as separate PRs.

[nikosavola]

#2506 is the CBLOCK inflate with zlib. I dropped the third piece, the varint readers decoding straight from the buffer: with #2505 in place it measured no gain (1.00x on raw and CBLOCK reads), so it is not worth the extra code.