Skip to content

test(persisters): clean up after the Redis and Postgres integration tests - #1035

Open
YongGoose wants to merge 1 commit into
apache:mainfrom
YongGoose:fix/persister-test-cleanup
Open

YongGoose wants to merge 1 commit into
apache:mainfrom
YongGoose:fix/persister-test-cleanup

Conversation

@YongGoose

Copy link
Copy Markdown
Member

The Redis and PostgreSQL persister integration tests left their data behind, so re-running them against the same databases failed 22 tests with duplicate-key errors. Each test now removes what it wrote.

Closes #1034

Changes

  • test_postgresql.py: the sync and async fixtures delete the rows in their test table on teardown. I used DELETE rather than DROP TABLE because DROP needs an exclusive lock and waits indefinitely behind any connection still idle in a read transaction (for example the unpickled persister in test_serialization_with_pickle, until it is garbage collected). DELETE doesn't conflict with readers.
  • test_b_redis.py: an autouse fixture records the keys in db 0 and db 1 before each test and deletes only the keys that appeared during it. This covers the fixture-based tests and test_redis_persister_class_backwards_compatible, and leaves any existing data in db 0 (Redis' default db) alone, so there's no FLUSHDB.

The persisters themselves are unchanged.

How I tested this

With postgres:15, mongo:7.0 and redis:7 (the CI images):

  • Ran BURR_CI_INTEGRATION_TESTS=true pytest tests/integrations/persisters three times in a row against the same databases: 68 passed each time. Before this change: 68 passed on the first run, then 22 failed on every later run.
  • Afterwards no test keys are left in Redis and both Postgres test tables are empty. A key I set in db 0 and db 1 before the runs was still there.
  • Got the same result running the suite twice inside a Linux container (Python 3.12).
  • Checked the DELETE vs DROP TABLE reasoning: with another connection idle in transaction after a SELECT on testtable, DELETE FROM testtable returned immediately, while DROP TABLE testtable was still waiting after 5 minutes.
  • pre-commit run passes on the changed files.

Notes

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments (n/a)
  • Project documentation has been updated if adding/changing functionality. (n/a, test-only change)

🤖 Generated with Claude Code

…ests

The persisters reject saving a step that already exists, and the Redis and
Postgres fixtures only closed their connections on teardown, so a second run
against the same databases failed 22 tests with duplicate-key errors.

- Postgres: the sync and async fixtures delete the rows in their test table.
  DELETE rather than DROP TABLE, which waits indefinitely behind any
  connection left idle in a read transaction.
- Redis: an autouse fixture deletes only the keys created during each test,
  so data that already lives in db 0 (Redis' default) is left alone.

Fixes apache#1034

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Redis and Postgres persister integration tests fail when re-run against the same databases

1 participant