Skip to content

Keep a view's file-like source alive through its relation - #689

Draft
evertlammerts wants to merge 1 commit into
duckdb:v2.0-cyanopterafrom
evertlammerts:fix-view-keeps-file-dependency
Draft

evertlammerts wants to merge 1 commit into
duckdb:v2.0-cyanopterafrom
evertlammerts:fix-view-keeps-file-dependency

Conversation

@evertlammerts

Copy link
Copy Markdown
Member

Fixes #477. A view created with to_view() or create_view() over a relation that reads a Python file-like object (read_csv(io.StringIO(...)), read_json, read_parquet on a buffer) stopped working once the relation was garbage collected: the next query against the view raised an IOException for a missing object-store file. The relation owned the registered file object; the view did not, because CreateViewRelation copies the query and the copy drops the relation's external dependency.

The view now gets the relation's own query node, with one table reference in it carrying a dependency on the relation. The search for that reference walks the query node, its expressions and subqueries, and picks the first reference whose copy keeps the dependency (base tables, subqueries, table functions, expression lists, column data and SHOW references). A relation that only appears inside a scalar subquery or an ANY comparison is found too. The same holds for a replacement scan that resolves to a relation. Column names, including duplicates such as a, a, pass through unchanged.

The new test file registers a file-like source, drops every Python reference to it, forces a collection, and reads the view, and it checks the source is released after DROP VIEW. It covers CTEs that shadow the source name, scalar and ANY subqueries, window expressions, JSON operators, pivots, views over views, and Arrow and pandas sources.

One pre-existing problem is pinned with a strict xfail rather than fixed: when the view's query fails during binding, 2.0 raises SystemError instead of the engine's error. That is a separate regression in the error path.

@evertlammerts
evertlammerts marked this pull request as draft October 16, 2026 17:04

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

read_csv from filehandle, when turned into a view, stops working with No files found that match the pattern "DUCKDB_INTERNAL_OBJECTSTORE://...

1 participant