Address remaining panic-on-persistence failures #381

Activity

1. tnull

   added this to the 0.5 milestone on Oct 17, 2024
2. tnull

   modified the milestones: 0.5, 0.6 on Jan 24, 2025
3. tnull

   added

   blocked on upstream

   on Mar 21, 2025
4. tnull

   modified the milestones: 0.6, 0.7 on Jun 9, 2025
5. tnull

   modified the milestones: 0.7, 0.8 on Nov 17, 2025
6. tnull commented on Nov 17, 2025

   

   tnull

   on Nov 17, 2025

   CollaboratorAuthor

   More actions

   In particular, this will require fully switching over to KVStore (i.e., including ChannelMonitor persistence) which we intend to do for v0.8.
7. tnull

   removed

   blocked on upstream

   on Jun 11, 2026
8. tnull commented on Jun 11, 2026

   

   tnull

   on Jun 11, 2026

   Last edited by tnull

   CollaboratorAuthor

   More actions

   This was mostly addressed by #919, but there are still remaining cases we'd need to address to fully resolve this. For the local ones we'd need some kind of wrapper for the node that cleanly shuts down and attempts restart whenever we hit a persistence failure (after retries for VSS, ofc). We also still have UnrecoverableError variants upstream that we'd need to drop to fully resolve this.

   • P0, easy: make the background processor task return Result<(), io::Error> and make Node::stop() return Error::PersistenceFailed instead of panicking when final persistence fails.
   • P1, not easy: running-node persistence failures should remain tracked for a restart/refactor design.
   • P1, upstream-blocked panics:
     • LDK documents UnrecoverableError as an immediate panic: https://github.com/lightningdevkit/rust-lightning/blob/3dfcc4cca1866c5e5d4d4eaf3b82e09584e2ce5c/lightning/src/chain/chainmonitor.rs#L102-L104
     • Chain-sync monitor persistence failure panics: https://github.com/lightningdevkit/rust-lightning/blob/3dfcc4cca1866c5e5d4d4eaf3b82e09584e2ce5c/lightning/src/chain/chainmonitor.rs#L481-L503
     • New monitor persistence UnrecoverableError panics: https://github.com/lightningdevkit/rust-lightning/blob/3dfcc4cca1866c5e5d4d4eaf3b82e09584e2ce5c/lightning/src/chain/chainmonitor.rs#L1124-L1137
     • Monitor update persistence UnrecoverableError panics: https://github.com/lightningdevkit/rust-lightning/blob/3dfcc4cca1866c5e5d4d4eaf3b82e09584e2ce5c/lightning/src/chain/chainmonitor.rs#L1223-L1231
     • Sync persisters still convert write failures into UnrecoverableError: https://github.com/lightningdevkit/rust-lightning/blob/3dfcc4cca1866c5e5d4d4eaf3b82e09584e2ce5c/lightning/src/util/persist.rs#L727-L744

   With #919, #782, #667 we made some good progress here, but for now I'll remove this from the 0.8 milestone.
9. tnull

   added

   blocked on upstream

   on Jun 11, 2026
10. tnull

    removed this from the 0.8 milestone on Jun 11, 2026
11. TheBlueMatt commented on Jul 24, 2026

    

    TheBlueMatt

    on Jul 24, 2026

    Contributor

    More actions

    P1, upstream-blocked panics:

    The only thing we could realistically do to "fix" these would be to simply remove the UnrecoverableError return value. If this is still somehow an "issue" we should just do that - this is not blocked on upstream, there is no way to handle this in lightning, the only appropriate way to handle this is to set a flag in ldk-node, do not "complete" the async monitor update persistence, and read that flag to shut down.