Repository navigation
Conversation
|
Make sure to bump version in DESCRIPTION when posterior gets updated |
|
Updated snaps just to see if tests pass, but there are a lot of (repeated) warnings in the new version. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #290 +/- ##
==========================================
- Coverage 92.88% 92.68% -0.20%
==========================================
Files 31 31
Lines 3289 3063 -226
==========================================
- Hits 3055 2839 -216
+ Misses 234 224 -10 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Unfortunately posterior has many warnings and the progress on having better control of the warnings with warning levels has been slow stan-dev/posterior#309 |
Yeah and warnings add some overhead (even if suppressed), so I expect this to be a bit slower than the current implementation in the case that there are many warnings. I wish there was just a |
|
Just in case if you had not noticed |
|
No I missed that PR, I hadn't seen that |
|
@avehtari it would also be good if |
|
I think this looks good now, the only question I have is whether we care about the extra |
|
Seems like its probably worth opening a PR to modify ps_tail then to add two flags, sorted and silent? |
|
Before adding silent flag to any function, I think we should decide whether we would in general prefer to use per function flags or global package options. See also stan-dev/posterior#309, stan-dev/cmdstanr#1175, and #353 |
PerformanceUsing the 4000x1359 I did not test with this PR. As that (I'll add later comments about desired behavior which helps to clarify which warnings we want to see) |
|
Thanks! I forgot about that example from that PR, good idea to add it to #352. |
|
This comment was marked as outdated.
This comment was marked as outdated.
So here we're seeing a non-trivial slowdown. Although we should probably increase the number of repetitions before inferring too much. |
|
Touchstone now is using 60 reps per branch. Will see if there's still a slowdown. |
|
This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4fccdef is merged into master:
Further explanation regarding interpretation and methodology can be found in the documentation. |
|
Seems like posterior's implementation is still slow. Should we get these fixes into posterior before merging this? Or is a ~10% slowdown okay, and we'll get a speedup once posterior is updated and ps_tail() is faster. It's a bit of a pain but I'm reluctantly in favor of waiting. |
|
~10% slowdown matches speed comparison on my laptop. posterior package has at least more conditionals as the function supports more options. The question is how much we want to specialize e.g. by making a separate function that is only for log ratios (no check whether non-log vs log, and no check for which tail). |
|
Separate function makes sense to me. Are there any real downsides to that approach? |
|
Downside is having repeated code and if behavior of one function is changed, do we remember to change the behavior of the other function. This has been especially problem when pareto related code has been replicated in different packages. It should be more likely that we remember fix two versions if they are in the same package. |
|
Since a new posterior release is imminent (stan-dev/posterior#473), this PR is finally almost ready to go again. Once the new version number is decided I will recreate this PR on latest loo main and bump minimum posterior version to the new release, and we can merge once posterior 1.7.2/1.8.0 is on CRAN. |
1593dea to
bf0ec33
Compare
Rely on
posteriorfor pareto smoothening tails.