Repository navigation
Release DataFusion 50.0.0 (Aug/Sep 2025) #16799
Description
Activity
Marked the reduce Expr size task here:
Reacted by Andrew LambWe are discussing a 49.0.1 patch here:
- test release branch 50 with protection rules and extended test support chore: Enforce checks for RC branches #17132
Reacted by Andrew LambThe DataFusion 50 upgrade may be challenging for many people. The update in Sail currently involves more than 100 file changes, and I am still not finished.
lakehq/sail#799dbt uses DataFusion 45. Attempted upgrade revealed:
Without solving this, we can't upgrade. The planning times for certain queries skyrocketed.
Should I add this to the "Bugs that need to be fixed" category, @alamb ?dbt uses DataFusion 45. Attempted upgrade revealed:
Without solving this, we can't upgrade. The planning times for certain queries skyrocketed. Should I add this to the "Bugs that need to be fixed" category, @alamb ?
If we think we can fix it let's do it
Reacted by Piotr FindeisenThe DataFusion 50 upgrade may be challenging for many people. The update in Sail currently involves more than 100 file changes, and I am still not finished. lakehq/sail#799
Thanks @shehabgamin -- it looks like many of those changes are related to UDF changes. Are there any others we should highlight?
I think we should start preparing to make the release -- maybe we can try and test/finalize anything needed this week and shoot for making the release candidate late next week.
Is anyone else interested interested/willing to coordinate / make the release candidate for this release (ideally a PMC member, @xudong963 perhaps?) If not, I can do so
I will do it later, thanks!
Reacted by Andrew LambTwo things I would like to get in to the 50.0.0 release:
35 remaining items
Hey @shehabgamin and @rkrishn7 , the voting process is nearly done.
I think we can prepare for the 50.0.1 release. I'll open an issue for that later, cc @alamb .
Reacted by Rohan KrishnaswamyI hoped for a proper solution of
- Exponential planning time when window function is partitioned by multiple columns #17401.
Since it didn't happen, here is a stop-gap solution - Prevent exponential planning time for Window functions #17563
we should also backport this to 50.0.1
- Exponential planning time when window function is partitioned by multiple columns #17401.
I know this is a bit late, but this super tiny PR seemed to slip through the cracks. Would be great to get it into 50 if possible.
I hoped for a proper solution of
I think we can prepare for the 50.0.1 release. I'll open an issue for that later, cc @alamb .
I filed a ticket to discuss 50.0.1: and tried to collect what people have requested there. #17594
@shehabgamin and @rkrishn7 can you prepare backports of your requests to the
branch-50branch?Reacted by Rohan Krishnaswamy and Shehab Amin50.0.0 has been officially approved, and released to crates.io so I am closing this ticket. Thanks to everyoe who helped and to @xudong963 for running the release 🎉
Reacted by Adam Gutglick, ayemjay, Rohan Krishnaswamy, Shehab Amin, Xin Sun and Qi ZhuHi @ddupg -- I think @timsaucer is always looking for more people to help review PRs and help with datafusion-python in general so perhaps there are some things you can to do to accelerate the 50.0.0 release
BTW it would be sweet if anyone has time to make a blog for this release
I will be happy to help with it
- added a commit that references this issue
on Sep 19, 2025 BTW it would be sweet if anyone has time to make a blog for this release
* [Blog post for the DataFusion `50.0.0` release #16931](https://github.com/apache/datafusion/issues/16931)I will be happy to help with it
I could help if possible. Are all the topics in the "Features to mention in the blog" of this issue or are there more items to mention?
Reacted by Andrew LambI could help if possible. Are all the topics in the "Features to mention in the blog" of this issue or are there more items to mention?
THANK YOU @nuno-faria
I think the list on this ticket is a good place to start; You can also troll through the changelog to see if there is anything else that is worth pointing out
Perhaps we can follow the model of the last post: apache/datafusion-site#91
Thanks to @nuno-faria we have a draft DataFusion 50 blog in case anyone is interested in reviewing:
- added a commit that references this issue
on Oct 3, 2025
Is your feature request related to a problem or challenge?
Tracking ticket for next release, also a place to track desired inclusions
Previous release will be #16235 (July, 2025) so next major release would be late Aug / early Sep 2025
Steps:
branch-50branch:50.0.0release #1693151.0.0(Nov 2025) #17558TODOs
Prior release tickets:
49.0.0: Release DataFusion49.0.0(July 2025) #16235Related
Changes to add to upgrade guide
Features to mention in the blog (if they make it)
SessionConfigreference toScalarFunctionArgs#13519 @Omega359WHERE,ORDER BY,LIMIT,SELECT,EXTENDpipe operators #17278 from @simonvandelPerformance
Bugs that need to be fixed
DynamicFilterPhysicalExprcauses recursive queries to fail #16998CooperativeExecincorrectly implementsmaintains_input_order#16994Bugs that would be good to fix / investigate
Community Wishlist
Date32to string given timestamp specifiers #15361DataFrameWriteOptions::with_single_file_outputproduces a directory #13323