Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ This repository has five packages.

| Package | Contents | Reference |
|---|---|---|
| `codeupipe` | The eight primitives. An export command that writes a pipeline as plain Python with no import of codeupipe. A config file reader. No dependencies | [conformance/API.md](conformance/API.md), [docs/static-export.md](docs/static-export.md), [docs/config.md](docs/config.md) |
| `codeupipe` | The eight primitives. A compile command that writes a pipeline as plain Python with no import of codeupipe. A config file reader. No dependencies | [conformance/API.md](conformance/API.md), [docs/compile.md](docs/compile.md), [docs/config.md](docs/config.md) |
| `codeupipe-lint` | Two linters for pipeline code. Depends on `codeupipe` only | [packages/codeupipe-lint/README.md](packages/codeupipe-lint/README.md) |
| `codeupipe` for TypeScript | The eight primitives. No dependencies | [packages/codeupipe-ts/README.md](packages/codeupipe-ts/README.md) |
| `codeupipe-lint` for TypeScript | A linter for pipeline code in TypeScript | [packages/codeupipe-lint-ts/README.md](packages/codeupipe-lint-ts/README.md) |
Expand Down Expand Up @@ -154,27 +154,27 @@ An editable install needs pip 21.3 or later.
| Command | What it does |
|---|---|
| `python -m pytest tests -q` | Runs the tests |
| `python conformance/run.py` | Runs the 27 conformance cases against the Python dynamic run and the three exports, and against the TypeScript targets that the machine can run |
| `python conformance/run.py` | Runs the 27 conformance cases against the Python dynamic run and the three compiled forms, and against the TypeScript targets that the machine can run |
| `python -m codeupipe_lint.anonymous_steps` | Runs the anonymous-step linter. Paths come from `cup.config.json` |
| `python -m codeupipe_lint.invariants` | Runs the invariant linter. Paths come from `cup.config.json` |
| `python -m codeupipe.lower examples.pricing.pricing_pipeline:PricingPipeline --out build/pricing` | Exports the example to `build/pricing/src/steps/` |
| `python -m codeupipe.compile examples.pricing.pricing_pipeline:PricingPipeline --out build/pricing` | Compiles the example to `build/pricing/src/steps/` |
| `python conformance/bench.py --check` | Compares benchmark numbers with the recorded baseline. Fails outside 25% |
| `python -m scorecard.differential` | Compares the export with the dynamic run on random pipelines |
| `python -m scorecard.differential` | Compares the compiled code with the dynamic run on random pipelines |

## Layout

| Path | Contents |
|---|---|
| `packages/codeupipe/` | The `codeupipe` package: `core/`, `lower/` (the export), `config.py` |
| `packages/codeupipe/` | The `codeupipe` package: `core/`, `compile/` (the compiler), `config.py` |
| `packages/codeupipe-lint/` | The `codeupipe-lint` package: `anonymous_steps/`, `invariants/` |
| `packages/codeupipe-ts/` | The TypeScript runtime, with its tests and its example |
| `packages/codeupipe-lint-ts/` | The TypeScript linter, with its tests |
| `packages/codeupipe-compile-ts/` | The TypeScript compiler, with its tests |
| `conformance/` | The contract ([API.md](conformance/API.md)), the cases, the runners, the benchmark |
| `tests/` | The tests for the two Python packages |
| `examples/pricing/` | The example, with its test in `tests/examples/` |
| `scorecard/` | The comparison of the export with the dynamic run on random pipelines |
| `docs/` | Reference for the export and the config file |
| `scorecard/` | The comparison of the compiled code with the dynamic run on random pipelines |
| `docs/` | Reference for the compiler and the config file |

The root `pyproject.toml` is not a package. It holds the test settings.

Expand Down
4 changes: 2 additions & 2 deletions conformance/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,7 @@ and limits:

**Conformance.** The shared cases name their steps; the runner builds every
step, so it maps each reported label back to the case's step name. A label
that is wrong, or differs between the dynamic run and an export, does not
that is wrong, or differs between the dynamic run and compiled code, does not
map and fails the comparison.

## Not in the contract yet
Expand All @@ -213,7 +213,7 @@ map and fails the comparison.
| Run | `run` is sync. `run_async` is opt-in | One `run`. It returns a Promise. A step can be sync or async |
| Absent | `None` | `null` |
| Label of a step | `type(step).__name__` | `step.constructor.name` |
| Export | `codeupipe.lower` | `packages/codeupipe-compile-ts` |
| Compile | `codeupipe.compile` | `packages/codeupipe-compile-ts` |
| Linter | `packages/codeupipe-lint` | `packages/codeupipe-lint-ts` |

Notes on TypeScript:
Expand Down
8 changes: 4 additions & 4 deletions conformance/BENCH.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ compares are in `bench/baseline.json` (`python conformance/bench.py --check`,
[README.md](README.md)).

`python:run` is the dynamic run. The three `static-*` targets are the
exports: `static-observe` records State and calls Hooks as `run` does; the
two fast exports do neither, and refuse the `hook` scenario rather than
compiles: `static-observe` records State and calls Hooks as `run` does; the
two fast modes do neither, and refuse the `hook` scenario rather than
silently dropping the Hook.

µs per run (best of 5 × 20,000, after 20,000 warmup runs); lower is better.
Expand All @@ -25,5 +25,5 @@ silently dropping the Hook.
| `heavy-allocation` | 50 | 17.14 | 2.80 | 1.49 | 7.22 |

Not measured (the target refuses the scenario):
- python:static-steps hook: UnsupportedFilterShape: <pipeline>.call at line 0:0 — cannot lower Hook. Hint: Hooks do not run in the fast static export. Export with observe=True (top-level Hooks only), or remove them.
- python:static-inline hook: UnsupportedFilterShape: <pipeline>.call at line 0:0 — cannot lower Hook. Hint: Hooks do not run in the fast static export. Export with observe=True (top-level Hooks only), or remove them.
- python:static-steps hook: UnsupportedFilterShape: <pipeline>.call at line 0:0 — cannot lower Hook. Hint: Hooks do not run in the fast mode. Compile with observe=True (top-level Hooks only), or remove them.
- python:static-inline hook: UnsupportedFilterShape: <pipeline>.call at line 0:0 — cannot lower Hook. Hint: Hooks do not run in the fast mode. Compile with observe=True (top-level Hooks only), or remove them.
10 changes: 5 additions & 5 deletions conformance/MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ live copy is `results/latest.md` (not committed). The command exits non-zero
if any target has a ❌.

**Every target passes every case, and nothing is unsupported.** The targets
are the Python dynamic run, async run and three static exports, and the
are the Python dynamic run, async run and three compiled forms, and the
TypeScript run and its compiled form.

The fast exports (`python:static-steps`, `python:static-inline`,
The fast modes (`python:static-steps`, `python:static-inline`,
`typescript:static`) expose no State or Hooks, so they pass those cases as
✅\*. `static-observe` (export with
✅\*. `static-observe` (compile with
`observe=True`) records both and passes every case outright. See
[docs/static-export.md](../docs/static-export.md).
[docs/compile.md](../docs/compile.md).

| case | python:run | python:run_async | python:static-steps | python:static-inline | python:static-observe | typescript:run | typescript:static |
|---|---|---|---|---|---|---|---|
Expand Down Expand Up @@ -52,4 +52,4 @@ The fast exports (`python:static-steps`, `python:static-inline`,
- **typescript:run**: 27 pass, 0 pass*, 0 fail, 0 unsupported
- **typescript:static**: 11 pass, 16 pass*, 0 fail, 0 unsupported

✅* passed everything the target can observe; it does not expose State or Hooks (the fast static export). ➖ the target cannot express the case.
✅* passed everything the target can observe; it does not expose State or Hooks (the fast mode). ➖ the target cannot express the case.
8 changes: 4 additions & 4 deletions conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ The rules are in **[API.md](API.md)**, the CUP API contract. This suite
checks them.

One set of language-neutral cases, a runner, and **one judge** (`run.py`).
The dynamic run and each static export are held to exactly the same
The dynamic run and each compiled form are held to exactly the same
comparison.

```bash
Expand Down Expand Up @@ -40,8 +40,8 @@ load a number can leave the band.
| Target | How it runs (all offline) |
|---|---|
| `python:run`, `python:run_async` | `codeupipe.core` Pipeline, sync and async entry points |
| `python:static-steps`, `python:static-inline` | `codeupipe.lower` fast export, loaded from disk (no State or Hooks: ✅\*) |
| `python:static-observe` | `codeupipe.lower` export with `observe=True`: State and Hooks judged too |
| `python:static-steps`, `python:static-inline` | `codeupipe.compile` fast mode, loaded from disk (no State or Hooks: ✅\*) |
| `python:static-observe` | `codeupipe.compile` with `observe=True`: State and Hooks judged too |
| `typescript:run` | `packages/codeupipe-ts` Pipeline. Node runs the `.ts` runner as it is (Node 22.18 or later). With no arguments, `run.py` leaves this target out when the machine has no such Node |
| `typescript:static` | `packages/codeupipe-compile-ts`: the pipeline as plain code, written to a file and imported (no State or Hooks: ✅\*). Needs `npm ci` in that package and a built core package; `run.py` leaves it out when they are not there |

Expand All @@ -52,7 +52,7 @@ load a number can leave the band.
| ✅ | every expectation matched |
| ✅* | everything the target *can observe* matched. Static code has no State or Hooks, so those checks cannot run |
| ❌ | something observable differed |
| ➖ | the target cannot express the case (e.g. export refuses a nested pipeline). Never counted as a pass |
| ➖ | the target cannot express the case (e.g. the compiler refuses a nested pipeline). Never counted as a pass |

## Case format (`cases/*.json`)

Expand Down
4 changes: 2 additions & 2 deletions conformance/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
partial every expectation the target can observe matched, but some
(State, Hooks) it does not expose at all (fast static code)
FAIL something observable differed (details in the JSON report)
unsupported the target cannot express the case (e.g. static export of a
unsupported the target cannot express the case (e.g. compiled form of a
nested pipeline); reported, never counted as a pass
"""

Expand Down Expand Up @@ -221,7 +221,7 @@ def _markdown(cases, names, matrix, report):
f"{t['unsupported']} unsupported")
lines.append("")
lines.append("✅* passed everything the target can observe; it does not expose State or Hooks "
"(the fast static export). ➖ the target cannot express the case.")
"(the fast mode). ➖ the target cannot express the case.")
return "\n".join(lines) + "\n"


Expand Down
2 changes: 1 addition & 1 deletion conformance/runners/python/conformance_sink.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
``from conformance_sink import TAPS`` as an import, so every lowered step
appends to this one list. A list defined next to the ops would be *copied*
into each lowered step instead (see the known limitation in
packages/codeupipe/codeupipe/lower/filters/emit_inline_module_filter.py).
packages/codeupipe/codeupipe/compile/filters/emit_inline_module_filter.py).
"""

TAPS = []
Expand Down
8 changes: 4 additions & 4 deletions conformance/runners/python/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
Targets
run codeupipe Pipeline.run (sync, the default)
run_async codeupipe Pipeline.run_async (inside asyncio.run)
static-steps codeupipe.lower export, per-step layout, loaded from disk
static-inline codeupipe.lower export, lowering="inline", loaded from disk
static-observe codeupipe.lower export, observe=True: State and Hooks too
static-steps codeupipe.compile export, per-step layout, loaded from disk
static-inline codeupipe.compile export, lowering="inline", loaded from disk
static-observe codeupipe.compile export, observe=True: State and Hooks too

The runner never judges; conformance/run.py compares every target's results
against the same expectations. The fast static targets have no State or
Expand Down Expand Up @@ -183,7 +183,7 @@ def _dynamic(case, predicates, run_async=False):

def _load_export(p, lowering, observe, workdir):
"""Export ``p`` and import the package from disk, as a customer would."""
from codeupipe.lower import build_export_pipeline_v2
from codeupipe.compile import build_export_pipeline_v2

out = build_export_pipeline_v2(write=False).run(
Payload({"pipeline": p, "pattern": "flat", "lowering": lowering, "observe": observe}))
Expand Down
Loading
Loading