Skip to content

Improve Arrow Java ownership and add streaming named parameters - #39

Open
ShawnChen-Sirius wants to merge 2 commits into
mainfrom
codex/arrow-java-enhancements
Open

ShawnChen-Sirius wants to merge 2 commits into
mainfrom
codex/arrow-java-enhancements

Conversation

@ShawnChen-Sirius

@ShawnChen-Sirius ShawnChen-Sirius commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Close accepted Arrow readers on insert validation and native failures; preserve caller-owned batch roots.
  • Add immutable named Arrow output choices while retaining the existing constructor.
  • Bind typed named parameters in incremental Arrow queries through the core C API, reusing JDBC escaped-text encoding. Bump the JNI ABI to 4 to reject mismatched native packages at load time.
  • Document ownership, parameter semantics, and the remaining batch-size and JDBC ResultSet integration limits.

Verification

  • macOS arm64: rebuilt JNI shim against chDB core 26.11.0-rc.3 and ran mvn -pl chdb-arrow -am test with chdb.arrow.it=true.
  • ArrowSessionTest: 29 tests, 0 failures, 1 skipped for the known core invalid String-to-Int64 insert regression. ArrowOptionsTest: 1 passed.

Follow-up to #38, which is merged into main. The local-engine build was only for testing; no local-engine package is intended for production.

Note

Add streaming named parameters to Arrow Java and fix reader ownership

  • ArrowSession.queryArrow gains overloads that accept a named string parameter map; bindings are query-scoped and values are escaped via the JDBC text encoding (ArrowSession.java, ChdbConnection.java)
  • ArrowOptions gets immutable named choices withUnsupportedAsBinary, withLowCardinalityAsDictionary, and withStringAsBinary that preserve other settings (ArrowOptions.java)
  • New JNI entrypoint ChdbNative.arrowStreamOpenWithParams routes nonempty parameter maps to the core named-parameter API; empty maps use the existing path (chdb_jni.cpp)
  • insertOwnedReader now takes ownership inside a try-with-resources after the open-session and no-active-query checks, so the reader closes even when table-name or schema validation fails (ArrowSession.java)
  • Behavioral Change: JNI ABI version bumps from 3 to 4 in ChdbNative.JNI_ABI_VERSION, kJniAbiVersion, and the chdb.jni.abi.version Maven property; native shims and Java must be rebuilt together

Macroscope summarized d28b9f7.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant