Integrity fixes on main after 2.4.0: impact on encrypted databases and release plans

[robertomartana]

Hello, and thank you for SQLite3 Multiple Ciphers.

We use it through better-sqlite3-multiple-ciphers 13.0.3, which bundles SQLite3MC 2.4.0, for per-tenant encrypted databases. Looking at the commits on main since 2.4.0, we noticed several fixes around error propagation on encrypted writes and memory handling during rekey.

1. Can any of these issues in 2.4.0 cause a write to be reported as successful when it was not, or leave an encrypted database corrupted or unreadable?
2. Is there a recommended procedure for rekeying a database with a new salt, and does it need any of the unreleased fixes to be safe?
3. Do you have an expected date for the next release (2.6.2 / 2.7.0)?

Thanks,

Roberto Martana

[utelle]

We use it through better-sqlite3-multiple-ciphers 13.0.3, which bundles SQLite3MC 2.4.0, for per-tenant encrypted databases. Looking at the commits on main since 2.4.0, we noticed several fixes around error propagation on encrypted writes and memory handling during rekey.

In fact, I'm currently in the process of making some fixes to the error-handling code to make SQLite3MC more robust.

1. Can any of these issues in 2.4.0 cause a write to be reported as successful when it was not, or leave an encrypted database corrupted or unreadable?

In practice, most of the errors addressed by these fixes occurred very rarely, if at all, but of course it wasn't entirely impossible for them to occur.

2. Is there a recommended procedure for rekeying a database with a new salt, and does it need any of the unreleased fixes to be safe?

All versions up to 2.6.1 use a new salt on rekeying a database, unless the database is in WAL-journal mode. Rekeying in WAL-journal mode is supported since version 2.4.0 keeping the previous salt.

It was detected lately that even in Rollback-journal mode a failure during the rekey operation could lead to database corruption, if new salt was used. Therefore the default behaviour was changed on trunk to keep the previous salt.

However, it will be possible to specify a new salt. This will require users to take a few precautions to avoid data loss in the event of an error. A description of the exact procedure will be added to the documentation.

3. Do you have an expected date for the next release (2.6.2 / 2.7.0)?

The next release, version 2.7.0, is expected to take place sometime next week. I can't give an exact date yet, as there are still some tests to be completed to ensure that all known potential sources of errors have been fixed.

[utelle]

@robertomartana In the meantime I was confronted with an AI generated review listing several more potential bugs. Many of the listed issues concern corner cases that are likely to occur very rarely in practice. However, there are some reports that I do need to take seriously. I want to set aside enough time to address them. Therefore, I will postpone the planned release of version 2.7.0 by 1 to 2 weeks.