Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,19 @@ here.**

### Changed

- **A report list route answers every row up to one cap, and is refused above it, in all five
ports.** With no `limit` a served `object.report` returns all of its rows; one row past the
cap, or an explicit `limit` above it, is `400 { "error": "report_too_large", "maxRows": N }`,
before any body is written. The cap is 10,000 by default and a generator option:
`routesFile({ reportMaxRows })` / `routesFileHono` (TypeScript), `RoutesGenerator.ReportMaxRows`
(C#), the `reportMaxRows` generator arg (Java, Kotlin), `router_generator(report_max_rows=)`
(Python). **This changes served output in Java, Kotlin and Python**, whose report lists used to
stop at 50 rows with no limit; and in every port a report longer than the cap that used to answer
now answers `report_too_large`. Entity and projection lists keep their paging. A report page
bound that is not usable (`limit` below 1, a negative `offset`, not an integer) is
`400 pagination.invalid_value` in every port. Gated by the new `ReadingLog` report and the
`cap-all-rows`, `cap-refused` and `bad-page-bound` scenarios of the api-contract `report/` corpus.

- **Requirements guidance: a feature commit updates its requirement.** The agent context
(the authoring and verify skills and the always-on file `meta init` writes) now tells an
agent that a commit adding, changing or removing behaviour updates the ledger in the same
Expand All @@ -32,6 +45,10 @@ here.**
other cast. Schema comparison is TypeScript-owned, so no other port
carries this comparison.

- **A report list in Java, Kotlin and Python no longer silently drops rows past the 50th.** A
90-day daily report showed 50 days with no error when `limit` was omitted. C# also ignored a
`limit` of 0 or below on a report instead of refusing it. See the report row cap under
Changed.
- **The api reference now follows a routes generator's `filter` as well as its `expose`.**
`routesFile({ filter })` and `routesFileHono({ filter })` mount nothing for an entity the filter
rejects, but the API pages still listed all five verbs for it. The pages now document the
Expand Down
14 changes: 10 additions & 4 deletions docs/features/api-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,15 +206,15 @@ feature demand (it is a consistency/safety divergence, not a capability).
The TypeScript mount helpers (`@metaobjectsdev/runtime-ts/drizzle-fastify`,
`/fastify` and `/hono`) pin five responses the contract leaves open — HTTP 5xx is
implementation-defined below, and no corpus scenario sends a malformed body, a write the database rejects, a
malformed page bound, or a value only the generated schema's format check refuses. All five use the contract's
malformed page bound to an entity, or a value only the generated schema's format check refuses. All five use the contract's
`{ "error": "<code>" }` envelope, and all five are scoped to the
routes the helpers mount: an adopter's own routes, and a Fastify `setErrorHandler`
or Hono `onError` the adopter installed, answer exactly as they did before.

| Response | When |
|---|---|
| malformed JSON body | a `POST`/`PATCH`/`PUT` body that does not parse as JSON (an empty body sent as `application/json` included) → HTTP 400 `{ "error": "invalid_json" }`. An EMPTY body on a method that takes none (`GET`, `DELETE`, `HEAD`) is accepted even when the request declares `content-type: application/json`, as many clients do on every request; a non-empty `DELETE` body is still parsed. Before, Fastify answered its own `{ "statusCode": 400, "code": "FST_ERR_CTP_INVALID_JSON_BODY", … }` and Hono a Zod `validation` error about a missing object |
| malformed page bound | a `?limit=` or `?offset=` that is not a non-negative integer (`abc`, `-5`, `1.5`, an empty value) → HTTP 400 `{ "error": "pagination.invalid_value", "param": "limit", "value": "abc", "expected": "a non-negative integer (0 or more)" }`. Before, a non-numeric bound was dropped (every row came back) and a negative or fractional one reached SQL. There is no page-size maximum on an entity or a projection with declared columns; an opaque view (no declared columns, served by raw SQL) pages to at most 1000 rows, so there a `limit` above 1000 is refused too, with `"expected": "an integer from 0 to 1000"` (it used to be clamped silently). The same answer comes from the Drizzle mounts and the ObjectManager Fastify mount. No corpus scenario sends a malformed bound; the other ports answer it as their framework does |
| malformed page bound | a `?limit=` or `?offset=` that is not a non-negative integer (`abc`, `-5`, `1.5`, an empty value) → HTTP 400 `{ "error": "pagination.invalid_value", "param": "limit", "value": "abc", "expected": "a non-negative integer (0 or more)" }`. Before, a non-numeric bound was dropped (every row came back) and a negative or fractional one reached SQL. There is no page-size maximum on an entity or a projection with declared columns; an opaque view (no declared columns, served by raw SQL) pages to at most 1000 rows, so there a `limit` above 1000 is refused too, with `"expected": "an integer from 0 to 1000"` (it used to be clamped silently). The same answer comes from the Drizzle mounts and the ObjectManager Fastify mount. The corpus sends a malformed bound only to a report (`report/bad-page-bound`), where every port answers this same envelope and also refuses `limit=0`; on an entity or a projection the other ports answer it as their framework does |
| database constraint violation | a write the database itself rejects → `{ "error": "constraint_violation", "constraint": "<kind>" }`, HTTP **409** for `unique` and `foreign_key` (a conflict with another row) and HTTP **400** for `check` and `not_null` (the request's own value), per the status rule under "Error response". The body names the constraint KIND only — never the SQL, the table or the bound values. A `CHECK` the model derives (an enum, a `validator.numeric`/`length`/`regex` range) is normally refused earlier by the route's own validation, as the corpus's `{ "error": "validation" }` 400; this row is the database's backstop |
| schema validation failure | a body that fails the insert/update schema → HTTP 400 `{ "error": "validation", "issues": [...] }`, each issue as Zod reports it **minus `pattern`**: Zod 4 attaches a failed `.regex()` check's source there, which for a generated `field.date` / `field.time` / `field.timestamp` check is a ~300-character calendar regex (and an authored `validator.regex` pattern is the server's to keep). The generated temporal checks carry a short `message` naming the expected format instead (`"must be an ISO date (YYYY-MM-DD)"`) |
| unexpected server error | anything that is not a filter, validation, not-found or constraint answer — a query against a column the database no longer has, a driver failure → HTTP 500 `{ "error": "internal" }`, the code the cross-port reference servers already use. The body names no SQL, table, column or bound parameter; the full error goes to the server log (`console.error`). Before, Fastify's default handler echoed the driver message, which for Drizzle is the query text and its parameter values |
Expand Down Expand Up @@ -250,7 +250,13 @@ each handler they register instead; an `HTTPException` is rethrown to your
- `limit=N` — page size.
- `offset=N` — page offset. Both are non-negative integers; the TS mounts refuse
anything else with a 400 (see "TS-only error responses").
- **No page-size maximum, and the default is per port.** The contract pins what
- **A report is the exception: one cap in every port.** A served `object.report` list returns every
row when `limit` is omitted, up to 10,000 (a generator option), and answers
`400 { "error": "report_too_large", "maxRows": N }` for one row more, or for a `limit` above the cap.
A `limit` below 1, a negative `offset` or a value that is not an integer is
`400 { "error": "pagination.invalid_value", "param": "limit" | "offset", "value", "expected" }` on
a report in every port. See [reporting.md](reporting.md#how-a-report-is-served).
- **No page-size maximum on an entity, and the default is per port.** The contract pins what
an explicit `limit`/`offset` returns (`list-with-pagination`); it does not pin a
default or a ceiling, and the ports differ. With `limit` **omitted**, the
TypeScript mounts and the C# routes return **every** matching row, while the
Expand Down Expand Up @@ -528,7 +534,7 @@ the empty rows, `isNull=true` matches nothing, and a sort on it orders the same
engine.

Gated by [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/)
(27 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
(30 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
gives. The columns, their types and the per-port generated files are in
[reporting.md](reporting.md#how-a-report-is-served).

Expand Down
28 changes: 24 additions & 4 deletions docs/features/reporting.md
Original file line number Diff line number Diff line change
Expand Up @@ -556,8 +556,26 @@ rule every object uses ([api-contract.md](api-contract.md)).
| a filter or sort error | The four field-naming envelopes of [api-contract.md](api-contract.md), unchanged. |

The route lists the compiled view. No request parameter picks dimensions, measures or a
grain. The default page size is the port's own, as for any list route: TypeScript and C#
return every row when `limit` is omitted, Java, Kotlin and Python the first 50.
grain.

**Paging and the row cap.** A report is a whole answer, so its list route has no page default.
With no `limit` it returns **every row**, in all five ports, up to one cap: **10,000 rows**. One
row past the cap, or an explicit `limit` above it, is refused before any body is written with
`400 { "error": "report_too_large", "maxRows": N }`; the route never truncates silently. Narrow
the report with a filter, or page it with `limit` and `offset`. The cap is a generator option, the
same default everywhere, set where each port sets its other generator options:

| Port | Set the cap |
|---|---|
| TypeScript | `routesFile({ reportMaxRows: 500 })` (and `routesFileHono`), or `reportMaxRows` on `mountReadOnlyCrudRoutes` in an owned routes file |
| C# | `new RoutesGenerator { ReportMaxRows = 500 }` |
| Java, Kotlin | the `reportMaxRows` generator arg (`<reportMaxRows>500</reportMaxRows>` in the plugin's `<args>`) |
| Python | `router_generator(report_max_rows=500)` |

A page bound that is not usable is refused, never ignored: `limit` below 1, a negative `offset` or
a value that is not an integer answers `400 { "error": "pagination.invalid_value", "param": ... }`,
the envelope the TypeScript mounts use for a malformed bound. Entity and projection lists keep
their own paging ([api-contract.md](api-contract.md#sort--pagination)).

**What a caller may filter and sort on.** Every derived field whose type has filter operators
(string, enum, uuid, int, long, double, float, decimal, currency, date, time, timestamp,
Expand Down Expand Up @@ -716,7 +734,7 @@ compares the canonical string. D1 has no separate corpus: it is SQLite behind Cl
binding, so TypeScript runs the same scenarios and the same REST scenarios through Miniflare's
local D1 (`meta migrate --dialect d1`), with no cloud account.

The REST surface is gated by twenty-seven scenarios under
The REST surface is gated by thirty scenarios under
[`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/),
run in the **generated lane on all five ports**: list (a dimension with a segment-scoped sum
that is null for one group), a time dimension at a grain (`YYYY-MM-DD`), a no-dimension
Expand All @@ -727,7 +745,9 @@ sale keeps its row, with a filter and a sort on its defaulted measure, a dimensi
two references, one time dimension listed at two grains, a filter on a string (`like`), a boolean and a
uuid dimension, a filter on a measure that carries its own `@filter`, and a dimension over an
array and one over a map, which refuse a filter (`invalid_filter_field`) and a sort
(`invalid_sort`). The corpus model carries one sourceless report, so a port that serves every
(`invalid_sort`), and the row cap: a report of 61 rows served with a cap of 60 returns 60 rows past
the old 50-row default, refuses the whole report, an explicit limit above the cap and a bad page
bound. The corpus model carries one sourceless report, so a port that serves every
report it finds fails. Every port runs the scenarios through its generated routes against the
real views on Postgres, SQLite and MySQL (`report/schema.sqlite.sql`, `report/schema.mysql.sql`),
and TypeScript also through local D1. A report's decimal columns are asserted as canonical
Expand Down
12 changes: 11 additions & 1 deletion examples/advanced-modeling/codegen/generators/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,8 @@ function renderRoutes(
// which emits no `expose` key at all and so keeps output byte-identical for projects
// that do not use it.
expose?: readonly CrudVerb[],
// Row cap of a served report (`routesFile({ reportMaxRows })`). undefined emits no key.
reportMaxRows?: number,
): string {
// FR-017 Tier 2 — a TPH discriminator base mounts polymorphic list/get at the
// base path plus a full per-subtype CRUD route set scoped to each
Expand Down Expand Up @@ -160,6 +162,7 @@ function renderRoutes(
: "") +
(keyless ? `\n${indent}itemRoutes: false,` : "") +
(report ? `\n${indent}resource: "report",` : "") +
(report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") +
decimalColumnsLine(entity, indent, ctx.loadedRoot);
const FastifyInstanceSym = imp("t:FastifyInstance@fastify");
const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`);
Expand Down Expand Up @@ -705,6 +708,13 @@ export interface RoutesFileOpts {
* your host file. Off by default, so a project that does not ask gets no new file.
*/
registerAll?: boolean;
/**
* The row cap of a report list route (default 10000, the same in every port). With no
* `limit` a report answers every row up to the cap and is refused above it with
* `400 { "error": "report_too_large", "maxRows": N }`; an explicit `limit` above it is
* refused the same way. Absent emits no key, so the output stays byte-identical.
*/
reportMaxRows?: number;
/**
* Where the emitted routes import their mount helpers from. Absent: the adapter copy
* `meta eject routes` placed in `codegen/runtime/`, by a path computed from the target's
Expand All @@ -728,7 +738,7 @@ export const routesFile = function routesFile(opts?: RoutesFileOpts): Generator
};
return {
path: entityOutputPath(ctx.config.outputLayout ?? "flat", effectivePackage(entity), `${entity.name}.routes.ts`),
content: await formatTs(renderRoutes(entity, renderContext, resolveExpose(entity, opts?.expose))),
content: await formatTs(renderRoutes(entity, renderContext, resolveExpose(entity, opts?.expose), opts?.reportMaxRows)),
};
});
const generator: Generator = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import { FilterParseError, parsePageBound, SORT_EXPECTED } from "./list-params.j

// Re-exported so existing importers keep one door; the definitions live in the
// drizzle-free list-params module, which the ObjectManager Fastify mount also uses.
export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, SORT_EXPECTED } from "./list-params.js";
export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js";
import {
ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT,
looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ export class FilterParseError extends Error {
* no allowlist to page through and returns at most this many rows. */
export const RAW_VIEW_MAX_LIMIT = 1000;

/** The most rows a report list answers, whether or not the caller sends a `limit`. A report
* with no `limit` returns every row up to this cap and is refused above it (never
* truncated); the same default in all five ports. */
export const REPORT_MAX_ROWS = 10_000;

/** The `expected` member of a `pagination.invalid_value` answer with no upper bound. */
export const PAGINATION_EXPECTED = "a non-negative integer (0 or more)";

Expand Down Expand Up @@ -48,3 +53,47 @@ export function parsePageBound(

/** The `expected` member of an `invalid_sort` answer — the one spelling `?sort` takes. */
export const SORT_EXPECTED = "sort=<field>:asc|desc";

/** The page a report list reads: `limit` is how many rows to FETCH, which is one past the
* cap when the caller sent none, so the caller can tell exactly the cap apart from more than it. */
export interface ReportPage {
readonly limit: number;
readonly offset: number;
/** True when the caller sent no `limit`, so more than `maxRows` rows means refuse. */
readonly unbounded: boolean;
}

/** A report answered more rows than `maxRows` allows. The wire answer is
* `400 { "error": "report_too_large", "maxRows": N }`. */
export class ReportTooLargeError extends Error {
constructor(public readonly maxRows: number) {
super(`A report answers at most ${maxRows} rows; narrow it with a filter or page it with ?limit=.`);
this.name = "ReportTooLargeError";
}
}

/**
* Read a report list's `?limit=` / `?offset=`. A limit is an integer of at least 1 (a
* report with `limit=0` is a bad bound, not an empty answer) and an offset is a
* non-negative integer; anything else is `pagination.invalid_value`. A limit above
* `maxRows` throws {@link ReportTooLargeError}.
*/
export function parseReportPage(query: Record<string, unknown>, maxRows: number): ReportPage {
const badLimit = (value: string) =>
new FilterParseError(
"pagination.invalid_value",
`?limit= must be an integer from 1 to ${maxRows}, got "${value}".`,
{ param: "limit", value, expected: `an integer from 1 to ${maxRows}` },
);
let limit: number | undefined;
try {
limit = parsePageBound(query, "limit");
} catch (e) {
if (e instanceof FilterParseError) throw badLimit(String(query.limit));
throw e;
}
if (limit === 0) throw badLimit("0");
if (limit !== undefined && limit > maxRows) throw new ReportTooLargeError(maxRows);
const offset = parsePageBound(query, "offset") ?? 0;
return { limit: limit ?? maxRows + 1, offset, unbounded: limit === undefined };
}
Loading
Loading