Skip to content

Improve Arrow empty sources and cancellation in Node - #109

Merged
wudidapaopao merged 2 commits into
mainfrom
codex/arrow-node-ux-enhancements
Nov 13, 2026
Merged

wudidapaopao merged 2 commits into
mainfrom
codex/arrow-node-ux-enhancements

Conversation

@ShawnChen-Sirius

@ShawnChen-Sirius ShawnChen-Sirius commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Accept schema-bearing Arrow Tables with zero batches as zero-row query sources and inserts.
  • Add AbortSignal support to direct Arrow queries, output readers, input readers, and reader inserts; preserve native teardown ordering and document early-settle behavior.
  • Add regression coverage for empty sources and cancellation before the first batch, during fetch, and during insert.

Validation

  • npm run build:ts
  • 75 Arrow-focused tests passed.
  • Full test:v3 run: 559 passed, 11 skipped, one localhost watchdog test failed because the sandbox denied listening on 127.0.0.1 (EPERM). That test passed when rerun outside the sandbox (26/26 in its file).

Arrow IPC byte-stream input is intentionally outside this PR; direct input still accepts Arrow objects and batch iterables.

Note

Add AbortSignal support and zero-batch Table sources to Node Arrow APIs

  • Session.queryArrow, queryArrowStream, registerArrowReader, insertArrow, and insertArrowReader accept an optional AbortSignal. Abort is checked before native work starts, while awaiting batches, and after query resolution; aborts settle the caller early with ChdbAbortError while pending native work drains first.
  • On abort, readers are closed, the native reader is unregistered, and upstream iterator cleanup is requested via return() without waiting for a blocked next().
  • registerArrowTable and insertArrow accept a Table with a schema but zero batches as a zero-row source; emptyArrowBatch builds the structural zero-row batch and rejects an empty schema.
  • DirectArrowOptions in index.d.ts gains the signal property, and arrow.md documents the new behaviors.
  • Risk: aborting a buffered query ends the caller's wait but native computation continues until teardown; there is no rollback guarantee for inserts. Check withArrowAbort and ChdbArrowReader in index.js.

Macroscope summarized 14ca7fa.

Comment thread index.js Resolved
@ShawnChen-Sirius

Copy link
Copy Markdown
Contributor Author

@wudidapaopao, please review this PR

@wudidapaopao wudidapaopao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wudidapaopao
wudidapaopao merged commit c5f1a7f into main Nov 13, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants