Provenance capture defaults to off - #1568
Open
dimitri-yatsenko wants to merge 2 commits into
Open
dimitri-yatsenko wants to merge 2 commits into
dimitri-yatsenko wants to merge 2 commits into
Conversation
`provenance.capture` shipped defaulting to True, so upgrading to 2.3.4 would have changed the DDL of every Entry table declared afterwards. That is a deployment's decision, not a library default, and it left 2.3.4 as the only release on the 2.3 line that alters what an unchanged schema declares. It also makes the pair consistent: `jobs.add_job_metadata` already defaults to False for the same reason, and both add a hidden column at declaration. The platform turns capture on per project through the usual channels -- `DJ_PROVENANCE_CAPTURE`, the config file, or the secrets directory. Nothing else changes: a table declared with capture on still records on insert, and `deploy.add_prov_column` still retrofits one declared without it.
dimitri-yatsenko
requested review from
MilagrosMarin,
lum-agilyti and
ttngu207
October 2, 2026 22:29
It said tables declared from 2.3.4 onward already carry the slot. With capture off by default the reverse holds: a table has the column only if it was declared while a deployment had capture on, which is what makes this function the normal path rather than a migration.
dimitri-yatsenko
added a commit
to datajoint/datajoint-docs
that referenced
this pull request
Oct 2, 2026
Follows datajoint/datajoint-python#1568. Capture changes the DDL of every Manual table declared after it is enabled, so it is a deployment's decision rather than a library default -- `jobs.add_job_metadata` defaults off for the same reason. - The spec's "Capture defaults on" section argued the opposite case. It now states the one the default rests on, and keeps the property a deployment gets once it enables capture. - Both settings tables and the config example read `False`. - The how-to told readers to configure a source first and turn capture off later, which meant following it from the top recorded nothing. "Turn capture on" is now the first step. - The tutorial said DataJoint records the origin of every Manual insert. It does where a deployment has enabled it.
dimitri-yatsenko
added a commit
to datajoint/datajoint-docs
that referenced
this pull request
Oct 2, 2026
Follows datajoint/datajoint-python#1568. The notes promised a column on every Manual table and argued the case for defaulting on, both of which are now wrong. Capture is a deployment's decision, and the upgrade note is the stronger line: an unchanged schema declares under 2.3.4 exactly what it declared under 2.3.3.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
provenance.captureshipped defaulting toTrue. That makes 2.3.4 the only release on the 2.3 line where upgrading changes what an unchanged schema declares: everydj.Manualtable declared after the upgrade gains a hidden nullable column.Adding a column at declaration is a deployment's decision.
jobs.add_job_metadataalready defaults toFalsefor exactly this reason, and the two settings do the same kind of thing, so defaultingcaptureon was inconsistent as well as disruptive.The platform enables it per project through the usual channels —
DJ_PROVENANCE_CAPTURE, the config file, or the secrets directory.Nothing else changes. A table declared with capture on still records on insert;
deploy.add_prov_columnstill retrofits a table declared without it; the insert path is unchanged.A side effect worth noting: with capture off, a default install no longer reaches
build_payloadon insert, so the unmemoized_get_job_versioncall there costs nothing unless a deployment has turned both capture andversion_method="git"on. Memoizing it is still worth doing — tracked from @ttngu207's review of #1555 — but it is no longer on a default path.Verification
452 unit and 688 integration tests pass on MySQL and PostgreSQL. The provenance integration tests set
capture = Truein their fixture, so they still exercise the feature; the unit test that pinned the default now pinsFalse.