Skip to content

Keep import duckdb from opening a database and fix the default connection race - #680

Merged
evertlammerts merged 2 commits into
v2.0-cyanopterafrom
fix-import-opens-database
Oct 17, 2026
Merged

evertlammerts merged 2 commits into
v2.0-cyanopterafrom
fix-import-opens-database

Conversation

@evertlammerts

Copy link
Copy Markdown
Member

Fixes #612.

Importing duckdb opened the default in-memory database and started its thread pools, because the DB API NUMBER type object parsed the strings "BIGNUM" and "DECIMAL" at module scope, and parsing a type string without a connection runs on the default connection. Logical types belong to a connection's catalog, so the DB API categories no longer hold type instances at all: STRING, NUMBER, DATETIME and BINARY are sets of type ids, and column_type == duckdb.NUMBER tests the id of the type a connection's result handed back. Import now starts no thread.

Consequences:

  • Every DECIMAL is a NUMBER. Before, only DECIMAL(18,3) matched, because "DECIMAL" parsed to that one width and scale.
  • A JSON column is a STRING (its id is varchar), and an aggregate state matches by the id of its value.
  • DBAPITypeObject takes a collection of type ids; its types list is gone.

Comparing a column type with a DB API type object, column type on the left, printed a nanobind implicit conversion failure to stderr on every call. A third equality overload handles that operand; every equality result is unchanged.

Removing the import-time connection exposed a pre-existing deadlock: the default connection holder kept its mutex while connecting, and connecting releases the GIL, so two threads whose first duckdb.sql() created the default connection hung the process (before, only duckdb.close() followed by such a race reached it). set_default_connection racing a reader deadlocked the same way, on 1.5.6 too. The holder now keeps its mutex only around pointer reads and swaps; connecting and destroying happen outside it. A finalizer that fetches the default connection no longer deadlocks when the default is replaced, and once the module's cleanup has run while the interpreter finalizes, the holder neither opens nor keeps a connection, so nothing is left open at exit. A first-use race may briefly open one database per racing thread.

@evertlammerts
evertlammerts force-pushed the fix-import-opens-database branch 2 times, most recently from 7524d9d to 8dc7a27 Compare October 16, 2026 13:41
@evertlammerts
evertlammerts force-pushed the fix-import-opens-database branch from 8dc7a27 to 0436912 Compare October 16, 2026 13:43
@evertlammerts
evertlammerts added this pull request to the merge queue Oct 16, 2026
Merged via the queue into v2.0-cyanoptera with commit f199d53 Oct 17, 2026
32 checks passed
@evertlammerts
evertlammerts deleted the fix-import-opens-database branch October 17, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DuckDB spawns worker threads on module import rather than connection creation

1 participant