Repository navigation
Release DataFusion 55.1.0 (minor/patch) Release (Sep 2026) #24462
Description
Is your feature request related to a problem or challenge?
- Part of [EPIC] DataFusion Releases #19783
- Follow on to Release DataFusion
55.0.0(Jul / Aug 2026) #22393
Plan a 55.0.1 or 55.1.0 patch release to fix any bugs / issues found in 55.0.0.
Steps:
- Update version and changelog
- Test with DataFusion Python
- Test with DataFusion Comet
- Test with delta.rs:
- Test vortex:
- Test with iceberg-rust:
- Test with LakeSail:
- Test with parquet viewer
- Test with datafusion-materialized-views
- Voting Thread:
- Publish to crates.io
Potential Content:
Bug Fixes
-
fix: preserve MERGE target qualifier bindings #24429 (@wirybeaver)-- breaking API change, so can't include it - Issue Parquet filter pushdown silently drops a
get_fieldpredicate when the file needs schema adaptation (wrong results) #24109 (@adriangb); Backport: [branch-55] fix: apply struct field filters when the file schema needs adaptation (#24125) #24530 - Issue
GroupedHashAggregateStream::emitthrows ArrowError: column types must match schema types #24069 (@hsiang-c); Backport: [branch-55] fix: adapt input batches with stricter nested nullability to planned schema in aggregation (#24394) #24699 - Issue UnnestExec ignores datafusion.execution.batch_size #24383 (@andygrove); Backport: [branch-55] fix: make UnnestExec respect datafusion.execution.batch_size (#24384) #24529
- Issue fix(ffi): use the session's LogicalExtensionCodec in create_physical_plan #24690 (@timsaucer); Backport: [branch-55] fix(ffi): use the session's LogicalExtensionCodec in create_physical_plan (#24690) #24694
- Issue Projection field metadata is lost during physical plan optimization #24721 (@gene-bordegaray); Backport: [branch-55] fix: preserve projection metadata during optimization (#24670) #24992
-
Issue Physical-plan serialization drops ProjectionExec metadata #24695 (@gene-bordegaray)Not a regression from 54 and will require breaking API change - Issue DF 55: physical-optimizer
enforce_sortingpanics onRightMarkjoins #24717 (@Tpt); Backport:[branch-55] Fix panic on RightMark hash joins when propagating ordering (#24718) #24759 - Issue FFI constructors silently discard arguments when the input is already foreign #24722 (@timsaucer) Backport: [branch-55] fix: preserve provided arguments during FFI object construction (#24723) #24752
- [branch-55] Align metadata propagation through Physical and Logical casts (#23169) #24875 dependency for Projection field metadata is lost during physical plan optimization #24721
- Issue panic: 'StructArray::new' on zero-field (empty) structs in ScalarValue::compact and ScalarValue::new_default #24581 (@unikdahal); Backport: [branch-55] fix(common): support empty struct in ScalarValue::compact and new_default (#24582) #24876
Performance
- TBD
Other
- Issue Error publishing datafusion-spark in datafusion 55.0.0 release "cannot find module or crate
datafusionin this scope" #24474 (@alamb); Backport: [branch-55] Reduce datafusion-spark dependency footprint (#24351) #25277
Minor Features:
Metadata
Metadata
Assignees
Labels
No labels
Activity
[-]Release DataFusion `55.1.0` (Sep 2026) (minor/patch) Release[/-][+]Release DataFusion `55.1.0` minor/patch) Release (Sep 2026) ([/+][-]Release DataFusion `55.1.0` minor/patch) Release (Sep 2026) ([/-][+]Release DataFusion `55.1.0` (minor/patch) Release (Sep 2026) [/+][branch-55] fix: make UnnestExec respect datafusion.execution.batch_s…
[branch-55] fix: apply struct field filters when the file schema need…
alamb commentedon Aug 24, 2026
Is there anyone waiting on this release ?
Is there any PMC member to help run it?
timsaucer commentedon Aug 26, 2026
I added an issue (#24688) and fix (#24690) that
datafusion-pythonwill need before we release 55.0.0. I can run this release since I'm the one interested in getting it.alamb commentedon Aug 26, 2026
Thanks ! Just let me know what I can do to help
timsaucer commentedon Aug 26, 2026
I mentioned on the PR but I think #24429 may not be eligible for back port. #24195 looks like it has the same issue (or was built on top of #24429). Other than that I've put up two PRs that are cherry picks targeting
branch-55linked abovegene-bordegaray commentedon Aug 27, 2026
I believe I found some bugs while upgrading around losing metadata on projection collapsing and on serialization which I have a PR for one in progress and issue tracking the other:
alamb commentedon Aug 27, 2026
@Tpt found another bug:
alamb commentedon Aug 27, 2026
I have updated the description of this issue with the requested tickets from @gene-bordegaray and @Tpt
comphead commentedon Aug 28, 2026
hi folks, wondering 55.1.0 is expected for what part of Sep?
alamb commentedon Aug 29, 2026
From my perspective, we could release it as soon as possoble (and we can always make a 55.2.0 as well if we find more things()
timsaucer commentedon Aug 29, 2026
I agree. As soon as @gene-bordegaray has their two fixes ready I would like to do the release.
1 remaining item
unikdahal commentedon Sep 1, 2026
Would it make sense to include #24582 in the 55.1.0 backports?
branch-55still has the affected code paths, and the fix is narrow with no API change just prevents the empty-struct panic. Happy to prepare the backport PR if appropriate.comphead commentedon Sep 1, 2026
A join correctness issue, would be great to include it in the hotfix #24854. Addressing the solution.
UPD: Solution was addressed in #21585 and was merged couple hours ago but the size of the fix is not appropriate for the backport
geoffreyclaude commentedon Sep 6, 2026
I've got another bug fix I'd like to include, but it's been present since DF47 so it can probably wait for DF55.2! Given the amount of new backport requests, a DF55.2 makes more sense to me than trying to cram everything in DF55.1.
#24932 prevents a TopK pushdown past DISTINCT, which meant pruning happened before deduplicating.
gene-bordegaray commentedon Sep 6, 2026
Backport PR for fixing metadata preservation on projection collapses: #24992
timsaucer commentedon Sep 8, 2026
Starting the release process. Thanks for the help everyone!
timsaucer commentedon Sep 8, 2026
Request for review on the changelog/version: #25070
timsaucer commentedon Sep 8, 2026
Release vote started: https://lists.apache.org/thread/yzkdnsh1qkxqgz1w8cypj4h73v7s7xjj
timsaucer commentedon Sep 11, 2026
During publish I had to follow the same local change as Andrew listed in #24474
This is fixed on
mainbut I am noting here in case we do a55.2.0at some point.timsaucer commentedon Sep 11, 2026
One last thing to do is update
main: #25200alamb commentedon Sep 13, 2026
I made a PR here to fix it in case
chore: update version 55.1.0 (apache#25070) (apache#25200)
[branch-55] Reduce datafusion-spark dependency footprint (#24351) (#2…
alamb commentedon Sep 25, 2026
I filed a potential follow on ticket for a 55.2 release if anyone is interetsed
55.2.0(patch) Release (Oct 2026) #25758chore: update version 55.1.0 (apache#25070) (apache#25200)