Repository navigation
Conversation
d49a707 to
beec2f9
Compare
413d69f to
4d8d095
Compare
4d8d095 to
0a8cc0e
Compare
| @Override | ||
| protected void dropInNamespace(String database, String table, boolean purge) { | ||
| if (purge) { | ||
| namespace.dropTable(new DropTableRequest().id(tableId(database, table))); |
There was a problem hiding this comment.
[P1] Preserve storage options and surface failures when purging table data
When credentials or a custom endpoint are configured only in the Amoro catalog (for example, a private S3/OSS bucket or MinIO), reads work through storageOptions(catalogProperties, location), but this purge path does not use those options. In the pinned lance-namespace-hive3:0.4.1 implementation, doDropTable() removes the HMS entry first, then safeDropDataset() calls Dataset.drop(location, Collections.emptyMap()) and catches deletion exceptions, logging only a warning. Consequently, dropTable(..., true) can report success after removing the metadata while leaving the underlying data intact; retrying by table name then cannot clean it up because the HMS entry is gone.
Please use the same resolved storage options as the read path for data deletion and propagate deletion failures instead of reporting a successful purge. A regression test should cover catalog-only credentials/custom endpoints and a failed data deletion.
Why are the changes needed?
Close #4273 .
Brief change log
How was this patch tested?
Add some test cases that check the changes thoroughly including negative and positive cases if possible
Add screenshots for manual tests if appropriate
lance-hive metastore-local storage
lance-hive metastore-s3 storage



Documentation