Skip to content

DL2: declared files between steps (part A; WIP) - #1241

Draft
trissim wants to merge 5 commits into
mainfrom
refactor/DL2-declared-files-between-steps
Draft

trissim wants to merge 5 commits into
mainfrom
refactor/DL2-declared-files-between-steps

Conversation

@trissim

@trissim trissim commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Part A (links, one writer, roots, identity) is committed; part B (plate-scoped producer feeding later steps) not started. Checkpoint pushed when work stopped at the usage limit.

🤖 Generated with Claude Code

trissim and others added 5 commits October 12, 2026 14:37
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…en steps, have one writer

Roots are behaviour on the DatasetPathDeclaration family; global_output_folder
gets its own declaration. OutputFileOf names an earlier step's output file as an
input-file argument. The compiler builds the table of produced files once per
pipeline, rejects a declared output file with several writers, and records
declared_files on every compiled invocation. The executor checks a linked input
exists and refuses a second call writing one declared file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…classes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…references

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant