diff --git a/CLAUDE.md b/CLAUDE.md index cd41eebd8..240580afd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,7 +84,7 @@ Every activity is a [MoonWalk](docs/moonwalk.md) process, the one home of its st ### Main and branch -**The product owner creates every branch**, and main stays releasable: [the branch](docs/moonwalk.md#branch). A change goes from [a spec and a plan](docs/moonwalk.md#pick-spec-plan) to a pull request the plan describes. **Deleting the plan is the product owner's call**, because "the code is written" is not "the plan is realized". +**The product owner creates every branch**, and main stays releasable: [the branch](docs/moonwalk.md#branch). A change goes from [a spec and a plan](docs/moonwalk.md#pick-spec-plan) to a pull request the plan describes. **Moving the plan to `docs/work/past/plans/` is the product owner's call**, because "the code is written" is not "the plan is realized". **A contributed pull request is one topic, max 400 changed lines and ten files**, tests included, based on main: [Contribute a pull request](docs/moonwalk.md#contribute-pr). The contributor vouches for every line, agent-written or not, answers reviewers in their own words, and says what an agent wrote and what was tested. **A contribution is reviewed and finished as [Review and finish a contribution](docs/moonwalk.md#review-contribution) describes**. The contributor fixes their own code first and the maintainer finishes, case by case. The merge keeps their commits and authorship. diff --git a/CMakeLists.txt b/CMakeLists.txt index af76d94bb..dd59c28b9 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -131,6 +131,7 @@ find_program(UV_EXECUTABLE NAMES uv REQUIRED add_library(mm_core STATIC src/core/module/Control.cpp src/core/util/JsonUtil.cpp + src/core/util/JsonFetch.cpp src/core/util/parse.cpp src/core/util/ScratchBuffer.cpp src/core/system/HttpServerModule.cpp @@ -262,12 +263,9 @@ add_custom_target(ui_embed DEPENDS ${CMAKE_SOURCE_DIR}/src/ui/ui_embedded.h) # library while the device holds only what someone actually picked. Globbed, so the DEPENDS is the # directory contents rather than a file list that would drift; CONFIGURE_DEPENDS re-globs when a # script is added or removed. -file(GLOB MOONLIVE_SCRIPTS CONFIGURE_DEPENDS - ${CMAKE_SOURCE_DIR}/moonlive/effects/*.mle - ${CMAKE_SOURCE_DIR}/moonlive/layouts/*.mll - ${CMAKE_SOURCE_DIR}/moonlive/modifiers/*.mlm - ${CMAKE_SOURCE_DIR}/moonlive/services/*.mls - ${CMAKE_SOURCE_DIR}/moonlive/palettes/*.mlp) +file(GLOB_RECURSE MOONLIVE_SCRIPTS CONFIGURE_DEPENDS + ${CMAKE_SOURCE_DIR}/moonlive/*.mle ${CMAKE_SOURCE_DIR}/moonlive/*.mll ${CMAKE_SOURCE_DIR}/moonlive/*.mlm + ${CMAKE_SOURCE_DIR}/moonlive/*.mls ${CMAKE_SOURCE_DIR}/moonlive/*.mlp) # The file LIST itself is a dependency, not just each file's timestamp: DEPENDS notices an edited # script but not a DELETED one, so removing a script left it in the catalog and the device went on # offering a name that no longer exists upstream. The stamp is written at configure time from the diff --git a/docs/contributing/documentation-standards.md b/docs/contributing/documentation-standards.md index 9f44d6f92..ec7492e4b 100644 --- a/docs/contributing/documentation-standards.md +++ b/docs/contributing/documentation-standards.md @@ -126,7 +126,7 @@ Every page serves one of four reader needs, and only one. This is [Diátaxis](ht The test for any page is the cell it sits in. A tutorial that stops to explain, or a reference that starts to teach, is two pages: move the other half to where it belongs. -Two kinds of page sit outside the grid on purpose. **The rules**, [coding-standards.md](coding-standards.md) and this page in `contributing/` and [CLAUDE.md](../../CLAUDE.md) at the repo root, are for contributors and bind every change. `legal/` sits outside it too. **Work not yet in the code** lives under `docs/work/`, and nothing there describes the system as it is. `future` is what does not exist, `present` is being built and deleted at its PR, `past` is what shipped. +Two kinds of page sit outside the grid on purpose. **The rules**, [coding-standards.md](coding-standards.md) and this page in `contributing/` and [CLAUDE.md](../../CLAUDE.md) at the repo root, are for contributors and bind every change. `legal/` sits outside it too. **Work on its way to the code** lives under `docs/work/`, apart from the pages that describe the system. `future` holds what is planned, `present` what is being built, which moves to `past` at its PR, and `past` the closed plans, shipped or cancelled. Two scales below a page. **A module** has one reference card and one header (see [Module pages](#module-pages)); **a line of code** carries its own reason in a comment (see [Comments](#comments)). diff --git a/docs/explanation/architecture/mooncore.md b/docs/explanation/architecture/mooncore.md index ad2152113..1e6050b7e 100644 --- a/docs/explanation/architecture/mooncore.md +++ b/docs/explanation/architecture/mooncore.md @@ -101,7 +101,7 @@ The light domain plugs into the UI at three points. **A fixed top-level tree**: Only abstract what you need: - **Time**: `millis()`, `micros()`. Monotonic, microsecond resolution. (`esp_timer` / `std::chrono`) -- **Memory**: `alloc(size)`, `free(ptr)`. Prefers PSRAM on ESP32, falls back to regular heap. `freeHeap()`, `maxAllocBlock()` for diagnostics. (`heap_caps_malloc` / `std::malloc`) +- **Memory**: `alloc(size)`, `free(ptr)`. Prefers PSRAM on ESP32, falls back to regular heap; `allocInternal(size)` where only internal RAM will do (see [Internal RAM](#internal-ram)). `freeHeap()`, `maxAllocBlock()` for diagnostics. (`heap_caps_malloc` / `std::malloc`) - **Executable memory**: `allocExec(size)` / `freeExec(ptr, size)` allocate memory the CPU can *fetch and execute* from, and `writeExec(dst, src, len)` copies emitted machine code into it safely. Used by the MoonLive live-script engine (below) to place the native code it compiles. All the W^X and instruction-cache quirks live behind these three functions. On ESP32 that is IRAM via `MALLOC_CAP_EXEC`, with 32-bit-aligned stores and a cache sync so the core fetches fresh code. On POSIX desktops it is an `mmap` `PROT_EXEC` page, with `MAP_JIT` and a write-protect toggle on macOS-arm64. On Windows it is a `VirtualAlloc` `PAGE_EXECUTE_READWRITE` page plus `FlushInstructionCache`. (`heap_caps_malloc(MALLOC_CAP_EXEC)` / `mmap` / `VirtualAlloc`) - **Networking**: `UdpSocket` for ArtNet send. `TcpConnection` / `TcpServer` for HTTP + WebSocket; `TcpConnection::writeSome` is a non-blocking partial write (returns bytes written, 0 = would-block) so a backpressured browser can't stall the render loop. (lwIP sockets / BSD sockets) - **Scheduling**: `yield()` (cooperative yield to OS/RTOS), `delayMs(ms)` (blocking sleep, off-path only), `delayUs(us)` (a microsecond busy-wait, only for sub-millisecond hardware timing a driver owns, such as the WS2812 300 µs inter-frame latch in `RmtLedDriver`), and `reboot()`. General pacing uses the non-blocking `millis()` gate instead. (`vTaskDelay` / `esp_rom_delay_us` / `esp_restart` on ESP32; `std::this_thread::sleep_for` / `std::exit` on desktop) @@ -109,6 +109,20 @@ Only abstract what you need: Abstractions are added when a concrete implementation needs them, not pre-designed. +### Internal RAM + +On an ESP32 internal RAM is the scarce memory. WiFi, lwIP, the task stacks and every DMA buffer need it, while a board with PSRAM has megabytes of the other kind. **Whatever can live in PSRAM does.** Internal RAM is kept for what cannot leave it: + +- **What a DMA engine reads**, the LED drivers' buffers. +- **What a cache-safe interrupt reads.** PSRAM sits behind the flash cache, and a flash write on either core switches that cache off, even in the middle of an interrupt on the other core. So the interrupt's data comes from `platform::allocInternal`, and an object it reads inherits `platform::InternalRamObject`, whose `new` places it there. `ParallelLedDriver` and `MoonI80Peripheral` are the examples; the LCD-MM ring refuses to run when what its refill reads fell back to PSRAM, rather than fault at the next save. +- **The code and stacks of those paths.** + +Everything else asks for PSRAM first: a module through `MoonModule`'s `operator new`, a buffer through `platform::alloc`, a plain `malloc` through `CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=0`. WiFi, lwIP and TLS do so through their options in `esp32/sdkconfig.defaults`, where each option carries its reason. + +**The floor.** `platform::kInternalFreeFloor`, 100 KB, is the internal RAM a board with PSRAM keeps free for WiFi, HTTP and the UI. Its largest free block stays at 50 KB, for a TLS handshake or a state send. What would take the rest respects it. LCD-MM's `ringAuto` sizes its ring by the lead the wire needs and stops at the floor. The HTTP server queues a send on a socket only while internal RAM stays above half of it, so a slow client waits rather than draining the board. `scenario_RAM_floor` holds the floor with the UI open on the S3 and the P4. + +**A board without PSRAM** runs the same calls, which fall back to internal RAM. Its network buffers then share that RAM, so it holds no floor, and the send back-pressure is what keeps an open UI on a weak link from exhausting it. The measurements are in [performance.md](../../reference/performance.md#internal-ram). + **Platform boundary (hard rule).** All `#ifdef`, `#if defined`, platform-specific `#include`s, and hardware API calls live exclusively in `src/platform/`. Everything outside `src/platform/` compiles on every target without modification. Compile-time platform branching uses `if constexpr` on `platform_config.h` flags, never a preprocessor `#ifdef`. **Domain boundary.** A file in `src/core` includes nothing from `src/light`: core declares an interface, and the light domain implements and registers it. Both boundaries are rules of [the code check](../../reference/tests.md#check_code), each at zero, and a crossing raises the count the ratchet refuses. `LightOutput` is the domain boundary's shape: the light domain's `Drivers` implements it, and the WLED shim, MQTT and MoonStats read the device shape and the palettes through it. diff --git a/docs/explanation/architecture/moonlight.md b/docs/explanation/architecture/moonlight.md index a3da946cc..b1f1ac557 100644 --- a/docs/explanation/architecture/moonlight.md +++ b/docs/explanation/architecture/moonlight.md @@ -199,7 +199,7 @@ A modifier is a coordinate transform, applied in one of two ways (the fold contr The blend+map step walks each layer in turn: reads each logical light, uses that layer's LUT to find the physical position(s), blends the color into the physical output buffer. This is where logical space meets physical space. -Each mapping LUT is a flat, contiguous lookup table allocated outside the hot path. It is built in `Layer::prepare()` and rebuilt whenever a Layout or Modifier control changes (the controls' `affectsPrepare` returns true) or a Modifier/Layout child is added/removed/replaced/moved; both triggers flow through the same core mechanism, see [Event triggering between modules](moonmodule.md#event-triggering-between-modules). +Each mapping LUT is a flat, contiguous lookup table allocated outside the hot path. It is built in `Layer::prepare()`. A Layout or Modifier control change rebuilds it, a Modifier's reaching its own layer alone, and so does adding, removing, replacing or moving a Modifier or Layout child. Both triggers flow through the same core mechanism: [Event triggering between modules](moonmodule.md#event-triggering-between-modules). The LUT supports four mapping types: diff --git a/docs/explanation/architecture/moonmodule.md b/docs/explanation/architecture/moonmodule.md index 373f9bcd9..81476ec32 100644 --- a/docs/explanation/architecture/moonmodule.md +++ b/docs/explanation/architecture/moonmodule.md @@ -121,7 +121,7 @@ Control values and each module's `enabled` flag are persisted to flash so settin - **Storage**: one file per top-level module under `/.config/.json`, holding that module's [state document](../../reference/integrating.md#setting-everything-at-once), the shape a preset and `PATCH /api/state` carry, with the module's secrets in it since the file never leaves the device. Children are keyed by name, so a file applies through the same engine as every other document, as a stored file: a type this build lacks and a key no control matches are skipped and logged, and a value clamps to its control's bounds. - **Lifecycle**: the five phases the diagram above draws, read from persistence's side. `defineControls` binds every module's control set. The load hook then applies each file, creating children with their controls bound and setting values twice around a `rebuildControls`, so a control set that depends on a value (a driver's `peripheral`) restores. Each module's own `setup()` runs with the values already in place, and `applyState` sizes the buffers. The fifth phase sets the controls a MoonLive script declares as it compiles, which the load held until then. Modules themselves know nothing about persistence; they bind their variables. -- **Save trigger**: HttpServerModule marks the target module dirty on every successful control mutation. FilesystemModule debounces 2 s in `tick1s()`, walks the tree, writes any subtree containing a dirty descendant via atomic write-and-rename. +- **Save trigger**: `Scheduler::setControl` marks the target module dirty on every successful control mutation, except a live one. A live write is to a control declared live, or one a service script makes, which drives a value rather than configures it. FilesystemModule debounces 2 s in `tick1s()`, walks the tree, and writes any subtree holding a dirty descendant, with an atomic write-and-rename. - **Conditional controls**: every conditional control is always bound; the module sets a `hidden` flag (`controls_.setHidden(i, …)`) to tell the UI not to render it. The load path can therefore find persisted values regardless of the live conditional state. - **Code-wired children survive a stale file**: some children aren't created by the user; `main.cpp`'s boot wiring attaches them (`ImprovProvisioningModule` under `NetworkModule`; `NetworkSendDriver`, `PreviewDriver` under their parents). Each such child calls `markWiredByCode()` after `addChild()`, a one-bit flag meaning *"I belong here because the code put me here, not because a saved file or a user asked for me."* The problem it solves is a trim on load. A stored document says its children are exactly the ones it lists, so a child that exists in code but is absent from an older file, written before that child was added, would be dropped. The flag tells the apply step to keep it, and to keep its type where the file names another. Children added through the HTTP API or recreated from a file stay unmarked; those follow the file's tree shape exactly, so UI deletes still take effect. @@ -168,10 +168,10 @@ Both shapes extend to any future producer/consumer pair (a sensor owning a state A control changes, or the module tree is mutated (a child added, deleted, replaced, moved), and other modules may need to react. The framework provides a three-tier split so each change costs only as much as it has to, from cheapest to most expensive: 1. **`onControlChanged(controlName)`**: runs on *every* control change, but only on the module whose own control changed. A cheap, in-place, per-control reaction that touches nothing else: recompute a small derived table, re-bind a socket. Default no-op. -2. **`affectsPrepare(controlName)`**: a gate, default `false`. A module returns `true` only for controls that change the size or shape of its derived state (and thus may ripple to other modules); for controls that tweak a value in place it stays `false`. When `true`, the framework runs the tree-wide rebuild; when `false`, it doesn't. +2. **`affectsPrepare(controlName)`**: a gate, default `false`. A module returns `true` only for a control that changes the size or shape of its derived state, which may ripple to other modules. A control that tweaks a value in place stays `false`. When `true`, the framework runs the rebuild; when `false`, it doesn't. How far the rebuild reaches is `prepareScope()`: the whole tree by default, the module's container for a modifier, whose mapping belongs to its layer alone. 3. **`prepare()`**: the module (re)builds its derived state (buffers, tables) for the current control values. Reached via `Scheduler::prepareTree()`, the coordinator-driven sweep that walks every module's `prepare`. -`Scheduler::prepareTree()` fires from two triggers: a tier-2 gate returning true after a control change, **and** any tree mutation (HTTP add/delete/replace/move handlers all call it unconditionally, since a structural change is rare and unambiguously needs a rebuild). Both triggers funnel through the same sweep; each module's `prepare` is idempotent (such as an effect only reallocs when its grid count changed), so over-rebuilding is wasted work, not a correctness hazard. +A rebuild runs at the next frame boundary. Up to four scopes are pending per frame, and a fifth widens the rebuild to the tree. After a scoped rebuild every top module still gets `onTreePrepared()`. A module reading what the scope reshaped then prepares itself again when that changed, as `Drivers` does with the first layer's shape it sized from. `Scheduler::prepareTree()` fires from two triggers: a tier-2 gate whose scope is the tree, **and** any tree mutation. The HTTP add, delete, replace and move handlers call it unconditionally, since a structural change is rare and needs a rebuild. Both triggers funnel through the same sweep. Each module's `prepare` is idempotent, an effect reallocating only when its grid count changed, so over-rebuilding wastes work without breaking anything. **`quiesce()`, the structural path's thread guard.** A module may hand work to another thread: `Drivers` ticks its Driver children on a core-1 task, see [Parallelism](#parallelism). That makes a *structural* mutation dangerous, and switching a module on or off with it. `addChild` reallocates the child array a worker may be walking, and `removeChild` is followed by the caller's `release()` and `deleteTree()`, which frees a module a worker may be inside `tick()` on. So `MoonModule` declares `virtual void quiesce()` (default no-op) and **core calls it on the parent before every child-array mutation** (`addChild` / `removeChild` / `replaceChildAt`); a module owning a worker overrides it to park that worker. The control path already funnels through `applyState()`/`prepareTree()`, where the owner quiesces itself, this is the same rule extended to the sibling (structural) path, and it lives in core so no HTTP handler has to remember it (CLAUDE.md § *when core already owns a mechanism for one path, extend it to the sibling path*). Deleting a driver from the UI mid-encode is therefore safe by construction, not by handler discipline. diff --git a/docs/help/help.json b/docs/help/help.json index fd3bbad81..1642a4cb8 100644 --- a/docs/help/help.json +++ b/docs/help/help.json @@ -3702,6 +3702,12 @@ } ] }, + "clockPin": { + "line": "The i80 write strobe (WR): the pixel clock, and the expander's shift clock." + }, + "dcPin": { + "line": "The i80 bus's data/command select, which `LCD-IDF` mandates though WS2812 ignores it." + }, "pinExpander": { "line": "A 74HCT595 per data pin: each pin then drives 8 strands.", "more": "It buys pins, not memory: the frame grows eightfold, about 1.1 KB per light, up to 64 strands. `i80` stops at about 96 lights per strand, `MoonI80` drives 128.", @@ -3715,6 +3721,30 @@ "latchPin": { "line": "The 74HCT595 latch (RCLK) pin, needed with the expander." }, + "shiftOverclock": { + "line": "Shift the '595 at 26.67 MHz, not 20, for short wires; off if a panel scrambles." + }, + "useRing": { + "line": "Stream the frame as a ring of small internal buffers, or send it whole from one block." + }, + "ringSnapshot": { + "line": "Copy each frame before the ring sends it, so a frame never mixes with the next." + }, + "ringAuto": { + "line": "Size the ring by itself and show the numbers it picked in the controls below." + }, + "ringRows": { + "line": "Lights per DMA buffer: the ring's grain, and the RAM against interrupt-rate lever." + }, + "ringBufs": { + "line": "How many buffers the DMA circulates, each `ringRows` rows of internal RAM." + }, + "ringPadUs": { + "line": "Inter-buffer zero-pad in µs, buying refill deadline at a linear frame-time cost." + }, + "ringDbg": { + "line": "The ring's live counters, once a second; `ab` counts lost frames, 0 on a healthy ring." + }, "loopbackTest": { "line": "TX to RX self-test: send a known pattern, bit-verify the capture." }, @@ -3802,6 +3832,9 @@ "maxBlock": { "line": "The largest free internal block, which bounds one allocation." }, + "missing scripts": { + "line": "Shipped scripts a module names and the device lacks; an open page downloads them." + }, "flash": { "line": "The flash chip size." }, @@ -4342,6 +4375,10 @@ { "title": "Writing the script", "text": "Everything the script declares appears as a control on the card, and a slider move lands without a recompile. A compile error shows on the status line, and the service does nothing until it is fixed. A script runs on the 50 Hz poll rather than the render tick, so a heavy one costs its own tick." + }, + { + "title": "Reading and sending JSON", + "text": "A service reads JSON from the internet with `getJson`, and sends its own documents with `pushJson` to this device or `pushJsonTo` to another. A show builds its own look this way, live and never saved. The calls: [the MoonLive language](https://moonmodules.org/MoonLight/moonmodules/light/moonlive.html#json-documents-in-a-service)." } ], "image": "https://moonmodules.org/MoonLight/assets/core/MoonLiveService.png", diff --git a/docs/moonmodules/light/moonlive.md b/docs/moonmodules/light/moonlive.md index 2cedc6aca..b1ea2cdce 100644 --- a/docs/moonmodules/light/moonlive.md +++ b/docs/moonmodules/light/moonlive.md @@ -55,13 +55,14 @@ void tick() { ``` `string` names what comes back rather than introducing a string type: a script returns a literal, and building, joining or comparing text is out of scope. +Text a script assembles, such as a temperature with its unit, lives in a [JSON document](#json-documents-in-a-service), which the host holds. ### Members, and which become controls `byte bpm = 30;` is state the script owns: visible in every function, surviving every tick. Naming it in `defineControls()` with `addControl("bpm", bpm, 1, 240)` also puts it on the UI as a slider, which is the same call a compiled module makes. A member no `addControl` names stays private to the script, which is how a stateful effect holds a value the user should not see. The default comes from the declaration, the range from the call, and the quoted name is the UI label, free to differ from the member's name. -`addSelect("shape", shape, "disc|ball|egg")` makes a dropdown instead: `shape` holds 0, 1 or 2, the index of the name picked. The quoted names count against the script's 128 bytes of strings. The engine keeps every dropdown's names together in 96 bytes, each name with one byte for its end, up to 16 names in all. +`addSelect("shape", shape, "disc|ball|egg")` makes a dropdown instead: `shape` holds 0, 1 or 2, the index of the name picked. The engine keeps every dropdown's names together in 96 bytes, each name with one byte for its end, up to 16 names in all. ### A member can be written @@ -130,7 +131,7 @@ class RainEffect { Both reach the picker before a factory script is downloaded, because the build extracts them from the source into the catalog. That copy is for display only: once a script is on the device, the compiled script is what decides. -## The five types +## The six types A type says what a value means, and the storage is the compiler's business. Every scalar occupies the same 4-byte slot whatever its type, and only arrays pack by element. @@ -143,10 +144,14 @@ That is where width still earns its keep: a `byte[]` heat map costs a quarter of | `bool` | `true` or `false` | a flag | | `fixed` | -32,768.0 to 32,767.99998, in steps of 1/65,536 | coordinates and anything fractional | | `string` | one of the script's own literals | a name passed to a builtin | +| `json` | a document the host holds, or none | data a service reads from the internet or sends to a device | An initializer is range-checked against its type, so `byte n = 300;` is a compile error naming the member rather than a silent 44. A `string` array is refused, since there is no runtime string to fill one with. +A script's literals live in a pool sized to them, so a long one costs only the script that holds it, up to 16 KB of text a script. +`R"( ... )"` is C++'s raw literal: it spans lines and keeps every quote, so a JSON document pastes in as it is, and `R"x( ... )x"` takes a delimiter for text holding `)"`. + ## RGB and Coord3D A color and a position travel as one value, under the names the compiled modules use for them. @@ -279,11 +284,11 @@ The core owns the grammar and a generic call mechanism, and nothing else. | `addSelect(name, member, "a\|b\|c")` | surface a byte member as a dropdown of those names, its value the index picked | | `gpioRead(pin)`, `gpioWrite(pin, v)` | read and drive a pin, from a service script | | `adcRead(pin)`, `adcMv(pin)`, `adcMax()` | an analog reading: raw, in millivolts, and its full scale | -| `setControl(module, control, v)` | drive another module's control, which is what a service does | +| `setControl("fader2", v)` | move a slot of the Control surface, which drives whatever the slot targets: what a service does | Every audio call answers 0 without audio, so a script written for a rig with a microphone still runs on one without. A motion call reaches nothing on a light that carries no such channel, so one script serves a moving head and a strip. -The last four belong to a service script, which runs on the poll rather than the render tick. +The last three belong to a service script, which runs on the poll rather than the render tick, and so do the JSON calls below. Each particle call is one pass over the whole pool per frame rather than one per light. A 300-spark script therefore costs far less than a shader touching every pixel. @@ -298,6 +303,53 @@ Its time axis must be monotonic, so feeding it a `beat()` sawtooth reads as a hi `turn(n)` exists because a full revolution is 65536, one past the largest number a script can write. +## JSON documents in a service + +A service script reads JSON from the internet, builds documents of its own, and sends them to this device or another one. +A `json` value is a handle to a document the host holds, in an int's slot, so text never passes through the script: it moves from document to document, and a number comes out as an `int` or a `fixed`. +A path into a document is a JSON Pointer ([RFC 6901](https://www.rfc-editor.org/rfc/rfc6901)): `/current/temperature_2m`, `/results/0/title`. + +```c +class Weather { + json w; + json show; + + void tick20ms() { + w = getJson("https://api.open-meteo.com/v1/forecast?latitude=52.37&longitude=4.90¤t=temperature_2m", 600); + if (jsonChanged(w)) { + show = R"({"Effects": {"Layer": {"Text": {"text": ""}}}})"; + jsonSetText(show, "/Effects/Layer/Text/text", "Amsterdam "); + jsonAppend(show, "/Effects/Layer/Text/text", w, "/current/temperature_2m"); + pushJson(show); + } + } +} +``` + +| call | does | +|---|---| +| `json d = R"({...})";` | a document from the script's own text, parsed once: a string literal where a `json` value belongs is parsed into one | +| `getJson(url, seconds)` | a document fetched from `url`, fetched again once `seconds` passed and it is read; it answers at once, with the last document that arrived | +| `jsonChanged(d)` | true when a new document arrived since the script last asked | +| `jsonStatus(d)` | the HTTP status of the last answer, 0 while none came | +| `jsonInt(d, "/p")`, `jsonFixed(d, "/p")` | the number at a pointer, a quoted one such as `"252.11"` included, 0 when absent | +| `jsonSize(d, "/p")` | how many elements or members the array or object at a pointer holds | +| `jsonSetInt(d, "/p", v)`, `jsonSetFixed(d, "/p", v)`, `jsonSetText(d, "/p", "text")` | write a value, creating the members on the way | +| `jsonAppendText(d, "/p", "text")`, `jsonAppend(d, "/p", src, "/q")` | join text onto a string: the script's own, or another document's value as it reads | +| `jsonCopy(d, "/p", src, "/q")` | copy a value from one document to another, a whole object included | +| `pushJson(d)` | apply a state document to this device, and answer the reply | +| `pushJsonTo("MM-eye1.local", d)` | send it to another MoonLight, or `POST` it to a URL, and answer the reply | + +A fetch runs on a task of its own, so a DNS lookup or a TLS handshake never stalls the poll. +A refresh that fails keeps the last good document, and an answer past 16 KB, 4 KB on a board without PSRAM, is refused rather than cut. +On a board without PSRAM an HTTPS request waits while internal RAM cannot hold a TLS session, and `jsonStatus` reads 0 until it went. + +A push is live, never saved: it applies the way `PATCH /api/state?live=1` does, at the next frame boundary, so a script building its look pushes it again at start. +A push to a target with one still on its way waits, replacing any that waited before it, so a script may push whenever something changed. +A document naming a shipped script the device lacks lists that script on the System card, and an open page downloads it, as the picker does. + +`jsondemo.mls` in `moonlive/services` shows four feeds, the weather, a price, a match and a headline, as scrolling text, and builds its own four layers to do it. + ## Debugging: print `print(v)` writes a value to the serial log and returns it, so it wraps any part of an expression without changing the result. @@ -308,4 +360,5 @@ tick, so a print costs frame time every frame it survives. Each compile grants a ## Source -[MoonLive](../../../src/core/moonlive/MoonLive.h) is the engine, [MoonLiveCompiler](../../../src/core/moonlive/MoonLiveCompiler.h) the front-end, and [MoonLiveBuiltins_light](../../../src/light/moonlive/MoonLiveBuiltins_light.h) the table registering every call above. [The lowering](../../../src/core/moonlive/moonlive_lower.h) turns the intermediate form into machine bytes, written once for every backend. +[MoonLive](../../../src/core/moonlive/MoonLive.h) is the engine, [MoonLiveCompiler](../../../src/core/moonlive/MoonLiveCompiler.h) the front-end, and [MoonLiveBuiltins_light](../../../src/light/moonlive/MoonLiveBuiltins_light.h) the table registering every call above. +The service and JSON calls are [MoonLiveBuiltins_service](../../../src/core/moonlive/MoonLiveBuiltins_service.h), each a thin wrapper over [JsonFetch](../../../src/core/util/JsonFetch.h) and [JsonUtil](../../../src/core/util/JsonUtil.h), which compiled code calls for the same reach. [The lowering](../../../src/core/moonlive/moonlive_lower.h) turns the intermediate form into machine bytes, written once for every backend. diff --git a/docs/moonmodules/light/power-functions.md b/docs/moonmodules/light/power-functions.md index ac8c656bb..bad2e47c4 100644 --- a/docs/moonmodules/light/power-functions.md +++ b/docs/moonmodules/light/power-functions.md @@ -11,6 +11,7 @@ Three consumers share this vocabulary, and each uses a different slice of it: (`core/moonlive/MoonLiveBuiltins.h`), which carries plain scalar arguments, so a script sees the flat form of a function, not the C++ callback form a compiled effect can use. Sources: [draw.h](../../../src/light/powerfunctions/draw.h) (drawing), `core/util/math16.h` (16-bit math), `core/util/noise.h` (fields). +Reading and sending JSON serves every domain, so it lives in core rather than here: [JsonUtil](../../../src/core/util/JsonUtil.h) addresses and edits a document by JSON Pointer, and [JsonFetch](../../../src/core/util/JsonFetch.h) fetches one and pushes one to a device, the calls a [service script's JSON values](moonlive.md#json-documents-in-a-service) wrap. The caller lists below are generated by reading the call sites, so they record what the code does rather than what it intends. Many of the names here are **[FastLED](https://github.com/FastLED/FastLED)'s**, deliberately: diff --git a/docs/reference/MIGRATING.md b/docs/reference/MIGRATING.md index 3184c0a87..b025bc15b 100644 --- a/docs/reference/MIGRATING.md +++ b/docs/reference/MIGRATING.md @@ -24,12 +24,36 @@ For a device still on old firmware (no Backup button yet), the [installer page]( | *re-set a control* | One value resets to its default; set it again in the UI if you had changed it. | | *re-add a module* | The module vanishes from the tree on boot; add it again and re-enter its controls. | | *update a file* | An on-device file must be edited or replaced. | +| *install over USB* | The update cannot come over the air: one install from the installer page with a cable, after which updates over the air work again. | | *erase flash* | A full flash erase is required (the heaviest, a full reconfigure follows). | --- ## Unreleased +### The StadBeest autopilot builds its looks, a layer each + +**Action: *nothing*, unless a desk or a preset switched the looks on `switch3` to `switch5`: use `encoder6` to `encoder8`.** + +`stadbeest-autopilot.mls` pushes every part its look at start, a layer per look. It crossfades between them through `encoder6` to `encoder8`, which set the three layers' opacity, where `switch3` to `switch5` turned three effects in one layer on and off, and it clears those three switch targets. The eyes and the mouth are reached by name, `MM-eye1.local`, `MM-eye2.local` and `MM-mouth.local`; a part named otherwise is renamed on its System card, or the names are edited in the script. [The StadBeest](examples/multi-board-installation.md#3-one-surface-layout-for-every-device) has the table. + +### LCD-MM's ring is smaller, and refuses PSRAM + +**Action: *nothing*, unless `ringSnapshot` was off on a board with PSRAM: turn it on.** + +`ringAuto` now sizes the ring by the lead the wire needs, 3-row buffers, where it took every byte of internal RAM above 64 KB. On a 3,840-light S3 that is 21 small buffers instead of 28 large ones, a ring of 37 KB where it took about 112 KB. Turn `ringAuto` off to set `ringRows` and `ringBufs` by hand as before. +The ring refuses to run when its refill would read PSRAM, since a flash save can cut PSRAM off in the middle of a refill and crash the board. With `ringSnapshot` off it reads the live frame, which a board with PSRAM keeps there, so the card shows `ring refused: … turn ringSnapshot on`. + +### ESP32-S3 boards read their flash over four lines + +**Action: *install over USB* once, on an `esp32s3-n16r8`, `esp32s3-n8r8` or `esp32s3-zero` device; the others need *nothing*.** + +These three firmwares read code from flash over four data lines (QIO) where they used two (DIO), which makes an S3's tick about 16% faster. +The flash mode is the bootloader's, and an update over the air replaces only the app, so ESP-IDF refuses an app whose mode differs from the running one. +The Firmware card then reports `ESP_ERR_OTA_SPI_MODE_MISMATCH`. +Install this release once over USB from the [installer page](https://moonmodules.org/MoonLight/install/), which writes the bootloader too. +The partition table is unchanged, so with *Erase chip first* left unticked the config stays; take a backup first anyway. + ### Rings 241 places a few lights one unit over **Action: *nothing*.** @@ -82,7 +106,7 @@ Switching a device on and off four times, each within a few seconds, opens its a **Action: *nothing*.** The palette picker lists every factory palette at its place in the catalog, held or not, and your own scripted palettes after them. A saved selection is kept by name. -A knob or a slot that stepped to a scripted palette by number may land on another one. +A knob or a slot that stepped to a scripted palette by number may land on another one. `stadbeest-fire` is 65 and `temperature` 64, since the StadBeest palette sits in its project folder. One the device does not hold keeps the last palette until the UI downloads it. ### `stadbeest-legs` is `stadbeest-walk`, `stadbeest-eyes` is `stadbeest-frog-eye` diff --git a/docs/reference/examples/multi-board-installation.md b/docs/reference/examples/multi-board-installation.md index 0a8d27c53..26356b558 100644 --- a/docs/reference/examples/multi-board-installation.md +++ b/docs/reference/examples/multi-board-installation.md @@ -34,10 +34,10 @@ This is the case study of one multi-device installation. The steps for your own | part | hardware | lights | layout | effect | |---|---|---|---|---| -| legs | LightCrafter 16 (ESP32-S3 N8R8) | 10 SK6812 RGBW strips of 144 | Tubes, `tubeDistance` 7 | [`stadbeest-walk.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-walk.mle), [`stadbeest-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-orb.mle), [`stadbeest-thrusters.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-thrusters.mle) | -| left eye | ESP32-S3-Zero, 4 MB | a 241-light ring disc | Rings241 | [`stadbeest-frog-eye.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-frog-eye.mle), [`stadbeest-eye-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-eye-orb.mle), [`stadbeest-sauron.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-sauron.mle) | +| legs | LightCrafter 16 (ESP32-S3 N8R8) | 10 SK6812 RGBW strips of 144 | Tubes, `tubeDistance` 7 | [`stadbeest-walk.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-walk.mle), [`stadbeest-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-orb.mle), [`stadbeest-thrusters.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-thrusters.mle) | +| left eye | ESP32-S3-Zero, 4 MB | a 241-light ring disc | Rings241 | [`stadbeest-frog-eye.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-frog-eye.mle), [`stadbeest-eye-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-eye-orb.mle), [`stadbeest-sauron.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-sauron.mle) | | right eye | ESP32-S3-Zero, 4 MB | a 241-light ring disc | Rings241 | the same three as the left eye | -| mouth | ESP32-C3 RGBWW PWM bulb | one RGBWW light | Grid 1×1×1 | [`stadbeest-mouth.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-mouth.mle) | +| mouth | ESP32-C3 RGBWW PWM bulb | one RGBWW light | Grid 1×1×1 | [`stadbeest-mouth.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-mouth.mle) | The three looks, as each device's preview shows them; the mouth, one light, breathes along with each. @@ -139,7 +139,7 @@ So they look the same way at the same moment, with no message between them. **Use them yourself.** The six looks and the autopilot are part of MoonLight's [script library](https://github.com/MoonModules/MoonLight/tree/main/moonlive). Every device running MoonLight lists them in its script picker and downloads one when it is chosen. -The full code: [`stadbeest-walk.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-walk.mle), [`stadbeest-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-orb.mle), [`stadbeest-thrusters.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-thrusters.mle), [`stadbeest-frog-eye.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-frog-eye.mle), [`stadbeest-eye-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-eye-orb.mle), [`stadbeest-sauron.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/effects/stadbeest/stadbeest-sauron.mle) and [`stadbeest-autopilot.mls`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/services/stadbeest-autopilot.mls); to write your own, start with [your first script](../../moonwalk.md#first-script). +The full code: [`stadbeest-walk.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-walk.mle), [`stadbeest-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-orb.mle), [`stadbeest-thrusters.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-thrusters.mle), [`stadbeest-frog-eye.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-frog-eye.mle), [`stadbeest-eye-orb.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-eye-orb.mle), [`stadbeest-sauron.mle`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-sauron.mle) and [`stadbeest-autopilot.mls`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-autopilot.mls); to write your own, start with [your first script](../../moonwalk.md#first-script). --- @@ -151,15 +151,12 @@ The first slot of each bank on a [Control](../../moonmodules/core/system.md#cont Every StadBeest effect that hears music declares `bpm`, `sensitivity` and `pulse` first, so those slots mean the same on the legs and on the eyes in every look. The rest are the look's own: on the walk `fader4` is `swing`, on the frog eye it is `pupil`. -A slot assigned by hand keeps its target, which is how the look switches and the autopilot stay put. +A slot assigned by hand keeps its target, which is how the look opacities and the autopilot stay put. | slot | legs | each eye | mouth | |---|---|---|---| | `switch1` | `Drivers.on` | `Drivers.on` | `Drivers.on` | | `switch2` | `pulse` | `pulse` | `pulse` | -| `switch3` | `walk.enabled` | `frog-eye.enabled` | | -| `switch4` | `orb.enabled` | `eye-orb.enabled` | | -| `switch5` | `thrusters.enabled` | `sauron.enabled` | | | `switch8` | autopilot `run` | | | | `fader1` | `Drivers.brightness` | `Drivers.brightness` | `Drivers.brightness` | | `fader2` | `bpm` | `bpm` | `bpm` | @@ -167,11 +164,14 @@ A slot assigned by hand keeps its target, which is how the look switches and the | `fader4` to `fader7` | the look's next numbers | the look's next numbers | `rest` | | `fader8` | autopilot `seconds` | | | | `encoder1` | `Drivers.palette` | `Drivers.palette` | `Drivers.palette` | -| `encoder2` and on | the look's choices, such as the orb's `shape` | the same | | +| `encoder2` to `encoder5` | the look's choices, such as the orb's `shape` | the same | | +| `encoder6` | `WalkLayer.opacity` | `FrogLayer.opacity` | | +| `encoder7` | `OrbLayer.opacity` | `EyeOrbLayer.opacity` | | +| `encoder8` | `ThrustersLayer.opacity` | `SauronLayer.opacity` | | A slot holds the value of the control it drives, so `fader2` at 33 means `bpm` 33. -Each look is an effect of its own on every device, named after its script without the `stadbeest-` prefix, and `switch3` to `switch5` turn them on and off through their `enabled`. -So one switch changes the look on the legs and on both eyes at once, and the desk follows it. +Each look is a layer of its own on every device, its effect named after its script without the `stadbeest-` prefix. `encoder6` to `encoder8` set the three layers' opacity. +So one encoder fades a look on the legs and on both eyes at once, and the desk follows it. A layer at opacity 0 costs nothing, so the beast renders two looks only while they cross. --- @@ -227,13 +227,16 @@ Every device runs the [Audio](../../moonmodules/core/services.md#audio) service ## 6. Running unattended -A [MoonLiveService](../../moonmodules/core/services.md#moonliveservice) on the leader runs [`stadbeest-autopilot.mls`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/services/stadbeest-autopilot.mls), which plays the beast through a whole museum night without anyone at the desk. +A [MoonLiveService](../../moonmodules/core/services.md#moonliveservice) on the leader runs [`stadbeest-autopilot.mls`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-autopilot.mls), which plays the beast through a whole museum night without anyone at the desk. The autopilot service card: script, run and seconds +- **Building:** at start it pushes every part its look, a layer per look, with [`pushJson` and `pushJsonTo`](../../moonmodules/light/moonlive.md#json-documents-in-a-service). It reaches the legs, itself, and the eyes and the mouth by name: `MM-eye1.local`, `MM-eye2.local` and `MM-mouth.local`. + A push is live and never saved, so it pushes them again every half minute, and a part that boots late or reboots gets its look back. The same document applied again changes nothing. + A part whose look names a script it lacks lists it on its System card, and the page open on it downloads it. - **Scenes:** every `seconds`, three minutes by default, it picks a new scene. - A scene moves on to the next look, walking, the orb or the thrusters, by switching `switch3` to `switch5`. - It picks a built-in palette, in a third of the thruster scenes [`stadbeest-fire.mlp`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/palettes/stadbeest-fire.mlp), a mood from calm to wild, how much the music counts and `pulse` on or off. + A scene crossfades to the next look, walking, the orb or the thrusters, over three seconds through `encoder6` to `encoder8`. + It picks a built-in palette, in a third of the thruster scenes [`stadbeest-fire.mlp`](https://github.com/MoonModules/MoonLight/blob/main/moonlive/projects/stadbeest/stadbeest-fire.mlp), a mood from calm to wild, how much the music counts and `pulse` on or off. That palette is computed every frame: louder music burns hotter, a beat flares the hot end white, and the flames flicker. A palette has the same number on every device, so the eyes and the mouth pick the same one. - **Breathing:** within a scene the energy rises and falls once a minute, through `fader2`, the tempo of whatever look runs. @@ -266,10 +269,10 @@ The browser is the bridge between desk and device, so the laptop keeps the leade |---|---| | track faders 1-7 | brightness, then the look's `bpm`, `sensitivity` and its own numbers | | track fader 8 | the autopilot's `seconds`, how long a scene lasts | -| track knobs 1-8 | the palette, then the look's choices and the numbers past the faders | +| track knobs 1-5 | the palette, then the look's choices and the numbers past the faders | +| track knobs 6-8 | the looks, faded in and out: walking, the orb, the thrusters | | activator 1 | on and off | | activator 2 | `pulse`, legs and eyes together | -| activators 3 to 5 | the look: walking, the orb, the thrusters | | activator 8 | the autopilot's `run` | | clip pads | the presets, the top-left pad being preset 1; a stored preset glows white and the applied one green | diff --git a/docs/reference/hardware/led-drivers.md b/docs/reference/hardware/led-drivers.md index 7bc28ce9a..a8d192a85 100644 --- a/docs/reference/hardware/led-drivers.md +++ b/docs/reference/hardware/led-drivers.md @@ -38,6 +38,34 @@ On an S3, P4 or S31 with `i80` or `MoonI80` as the `peripheral`, `pinExpander` o The expander buys pins, not memory: the register is serial-in, so the DMA frame grows eightfold, about 1.1 KB per light. `i80` holds the whole frame, which past about 96 lights per strand falls to PSRAM too slow for the expander's clock. `MoonI80` streams the frame and drives 128 reliably. +## Sizing the MoonI80 ring + +`MoonI80` streams the frame through a ring of small buffers in internal RAM, refilled from an interrupt as the wire drains each one. The refill encodes faster than the wire drains, so the ring only absorbs the moments the interrupt is held off, up to about 1.1 ms. What prevents a lost frame is the ring's lead over the wire in rows, not its number of buffers. On a 3,840-light expander rig, 45 rows ran clean and 40 lost frames. + +`ringAuto` cuts the ring into 3-row buffers and takes enough for 54 rows of lead. It stops where free internal RAM would fall under 100 KB, and the status says when that cuts it short. With it off, `ringRows` and `ringBufs` are yours to set, at about 600 bytes a row on an 8-pin expander. + +`ringSnapshot` copies each frame into internal RAM before the ring sends it, 3 bytes per RGB light (11.5 KB for 3,840 lights). Off, the ring reads the live frame, and refuses where that frame is in PSRAM. A flash save on the other core switches PSRAM off in the middle of a refill. + +`ringDbg` shows the ring's counters once a second. `ab` and `ml` say whether the depth holds. + +| Field | Meaning | +|---|---| +| `sn`, `lv` | where the snapshot and the live frame live: `I` internal RAM, `P` PSRAM | +| `sl/bf` | slices per frame, over buffers | +| `co`, `cr` | refills deferred while the flash cache was off, and their longest run | +| `ab` | stalls abandoned, each a lost frame: 0 on a healthy ring | +| `dn`, `ld` | frames done, and the last drain | +| `lt` | slices refilled after their drain began; the lead prevents it, so it stays 0 even while frames are lost | +| `tx` | the frame's wire time, in µs | +| `ipb`, `ci` | DMA items per buffer, and consumed | +| `tn`, `de` | the chain terminator, and descriptor errors | +| `enc`, `ea` | the slowest and the average refill, in µs | +| `sg`, `se` | gather and emit cycles per row | +| `tw`, `ts`, `tp` | the tick's wait, snapshot and prime, in µs | +| `gap@` | the longest gap between refills within a frame in µs, and the slice it ended at | +| `ml` | the fewest slices the refill stayed ahead of the drain | +| `c` | the core the refill runs on | + ## Testing the wire with the loopback `loopbackTest` sends a known pattern out of a strand and reads it back through a jumper to `loopbackRxPin`, with its verdict in the status field. A `PASS` means the chip and the peripheral are clean, so flicker that remains is [signal integrity](../troubleshooting.md#stray-pixels-or-wrong-colors-on-lights-that-should-be-off). diff --git a/docs/reference/integrating.md b/docs/reference/integrating.md index 8e16b5ab9..4c44248e1 100644 --- a/docs/reference/integrating.md +++ b/docs/reference/integrating.md @@ -52,6 +52,7 @@ The answer is `{"ok":true,"changes":N}`, or a 400 naming the first failure and w A failure only writing finds, such as a value out of range, leaves what came before it applied. A document is at most 32 KB, and a larger one is refused with a 413. `POST /api/state` takes the same document, for a client that cannot send `PATCH`. +`PATCH /api/state?live=1` applies a document that drives rather than configures, as a show building its look on another device does. It is checked and applied the same way, and nothing it changes is marked to save, so a reboot returns the device to its saved state. Adding, replacing and removing a module through `/api/modules` applies a one-member document, so the same rules hold on both paths. `GET /api/modules//document` reads a module back as a document, its path from the top level around it, and a card's `{ }` button shows the same. diff --git a/docs/reference/metrics/code.json b/docs/reference/metrics/code.json index f8d07970b..97d904f1b 100644 --- a/docs/reference/metrics/code.json +++ b/docs/reference/metrics/code.json @@ -1,18 +1,18 @@ { "#define used as a constant": 1, - "(duplicated lines)": 3379, - "(total)": 1672, + "(duplicated lines)": 3315, + "(total)": 1654, "blocking call on the render path": 194, - "complex function": 296, - "deeply nested": 111, + "complex function": 295, + "deeply nested": 110, "double in firmware code": 2, - "duplicated block": 752, + "duplicated block": 738, "large file": 25, - "large stack frame": 33, + "large stack frame": 32, "long function": 49, "long parameter list": 35, "module type in the UI": 78, - "mutable static state": 44, + "mutable static state": 43, "name spread": 22, "raw allocation in light": 30 } diff --git a/docs/reference/metrics/docgen.json b/docs/reference/metrics/docgen.json index fefbbcfae..0a16ebeb2 100644 --- a/docs/reference/metrics/docgen.json +++ b/docs/reference/metrics/docgen.json @@ -1,11 +1,11 @@ { "(appendix lines)": 4376, - "(total)": 3051, + "(total)": 3044, "appendix sections nothing refers to": 199, - "member deep dives": 39, + "member deep dives": 38, "multi-line comment blocks": 307, - "over-long sentences": 76, - "over-wide comment lines": 1541, + "over-long sentences": 75, + "over-wide comment lines": 1536, "over-wide doc lines": 34, "split paragraphs": 855 } diff --git a/docs/reference/metrics/prose.json b/docs/reference/metrics/prose.json index f20310b1a..d95a31214 100644 --- a/docs/reference/metrics/prose.json +++ b/docs/reference/metrics/prose.json @@ -1,12 +1,12 @@ { - "(total)": 2336, - "MoonLight.EmDash": 1117, + "(total)": 2327, + "MoonLight.EmDash": 1114, "MoonLight.LinkText": 7, "MoonLight.NegatedHeading": 6, "MoonLight.SelfReference": 17, - "MoonLight.SentenceLength": 320, + "MoonLight.SentenceLength": 316, "MoonLight.Spelling": 39, - "MoonLight.Temporal": 185, + "MoonLight.Temporal": 183, "MoonLight.TonePadding": 1, "MoonLight.VagueClose": 28, "MoonLight.Weasel": 616 diff --git a/docs/reference/metrics/repo-health.json b/docs/reference/metrics/repo-health.json index 7255af929..32310e515 100644 --- a/docs/reference/metrics/repo-health.json +++ b/docs/reference/metrics/repo-health.json @@ -1,43 +1,43 @@ { - "commit": "80e19948", + "commit": "bfcc83f5", "flash": { - "esp32s3-n16r8": 2474896, - "esp32": 2281504, + "esp32s3-n16r8": 2491984, + "esp32": 2299696, "esp32p4rev1-eth": 2404800, "esp32p4rev1-eth-wifi": 2657056, "esp32s3-n8r8": 2437792, "esp32s31": 2744032, - "esp32-16mb": 2356048, + "esp32-16mb": 2399680, "esp32-eth": 1681040, "esp32-wrover": 1843760, "qemu": 1383648, "esp32p4rev3-eth": 2423840, "esp32s3-zero": 2224816, "esp32-pico": 2107168, - "desktop-macos": 2423016, + "desktop-macos": 2480264, "desktop-windows": 1925120, "esp32c3": 2335072, "desktop-linux": 2548000, - "esp32s3-n32r16": 2484400, + "esp32s3-n32r16": 2513856, "teensy41": 1271788 }, "measured": { "esp32p4rev1-eth": "2026-10-19", "esp32s31": "2026-10-13", - "esp32": "2026-10-21", + "esp32": "2026-10-23", "esp32-pico": "2026-09-09", - "esp32s3-n16r8": "2026-10-21", + "esp32s3-n16r8": "2026-10-23", "esp32s3-n8r8": "2026-10-13", "esp32s3-zero": "2026-10-13", - "esp32-16mb": "2026-10-13", + "esp32-16mb": "2026-10-22", "esp32p4rev1-eth-wifi": "2026-10-19", "esp32-eth": "2026-09-11", "esp32p4rev3-eth": "2026-10-21", - "desktop-macos": "2026-10-21", + "desktop-macos": "2026-10-23", "desktop-windows": "2026-09-30", "esp32c3": "2026-10-10", "desktop-linux": "2026-10-12", - "esp32s3-n32r16": "2026-10-16", + "esp32s3-n32r16": "2026-10-23", "teensy41": "2026-10-15" }, "perf": { @@ -365,6 +365,20 @@ "last": "2026-10-14" } }, + "RAM_floor": { + "esp32s3-n16r8": { + "p50": 11732, + "p95": 12527, + "n": 3, + "last": "2026-10-22" + }, + "esp32p4rev1-eth": { + "p50": 2168, + "p95": 2168, + "n": 1, + "last": "2026-10-22" + } + }, "perf_full": { "desktop-macos": { "p50": 256, @@ -781,8 +795,8 @@ } }, "desktop-macos": { - "tick_us": 7, - "fps": 142857, + "tick_us": 8, + "fps": 125000, "scenario_p50": { "Effects_pipeline_builds_and_renders": { "p50": 7, @@ -822,54 +836,54 @@ } }, "loc": { - "core": 30448, - "light": 31568, - "platform": 21750, - "ui": 12957, - "test": 71673, - "moondeck": 31938 + "core": 32227, + "light": 31562, + "platform": 22123, + "ui": 12980, + "test": 73772, + "moondeck": 32132 }, "comments": { "core": { - "lines": 6336, - "ratio": 0.231 + "lines": 6547, + "ratio": 0.225 }, "light": { - "lines": 6666, + "lines": 6653, "ratio": 0.237 }, "platform": { - "lines": 4104, - "ratio": 0.212 + "lines": 4145, + "ratio": 0.21 }, "ui": { - "lines": 3489, + "lines": 3492, "ratio": 0.285 }, "test": { - "lines": 7277, - "ratio": 0.118 + "lines": 7389, + "ratio": 0.116 }, "moondeck": { - "lines": 4460, - "ratio": 0.16 + "lines": 4462, + "ratio": 0.159 } }, "tests": { - "cases": 2643, - "scenarios": 32 + "cases": 2741, + "scenarios": 35 }, "docs": { - "md_files": 162, - "md_lines": 30528, - "plans_files": 53, - "backlog_lines": 4031, - "lessons_lines": 535, + "md_files": 164, + "md_lines": 30957, + "plans_files": 55, + "backlog_lines": 4083, + "lessons_lines": 546, "claude_md_lines": 119 }, "complexity": { - "functions": 5171, - "over_threshold": 349, + "functions": 5344, + "over_threshold": 351, "worst_ccn": 128 } } diff --git a/docs/reference/metrics/repo-health.md b/docs/reference/metrics/repo-health.md index c96038956..e9424c8e3 100644 --- a/docs/reference/metrics/repo-health.md +++ b/docs/reference/metrics/repo-health.md @@ -1,6 +1,6 @@ # Repo health -Measured at `80e19948`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** +Measured at `bfcc83f5`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** Current state only; the trend is this file's git history (`git log -p docs/reference/metrics/repo-health.md`). Nothing here fails a build: the numbers make growth visible, the judgment stays human. @@ -9,29 +9,29 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | Metric | v6.0.0 | Now | Change | |---|---:|---:|---:| | flash desktop-linux | - | 2,488 KB | new | -| flash desktop-macos | 1,963 KB | 2,366 KB | +403.2 KB (+20.5%) | -| flash esp32 | 2,061 KB | 2,228 KB | +166.8 KB (+8.1%) | -| flash esp32-16mb | 2,012 KB | 2,301 KB | +288.8 KB (+14.4%) | +| flash desktop-macos | 1,963 KB | 2,422 KB | +459.1 KB (+23.4%) | +| flash esp32 | 2,061 KB | 2,246 KB | +184.6 KB (+9.0%) | +| flash esp32-16mb | 2,012 KB | 2,343 KB | +331.4 KB (+16.5%) | | flash esp32c3 | - | 2,280 KB | new | | flash esp32p4rev1-eth | 2,022 KB | 2,348 KB | +326.7 KB (+16.2%) | | flash esp32p4rev1-eth-wifi | 2,304 KB | 2,595 KB | +291.2 KB (+12.6%) | | flash esp32p4rev3-eth | 2,025 KB | 2,367 KB | +341.8 KB (+16.9%) | -| flash esp32s3-n16r8 | 2,111 KB | 2,417 KB | +305.4 KB (+14.5%) | -| flash esp32s3-n32r16 | - | 2,426 KB | new | +| flash esp32s3-n16r8 | 2,111 KB | 2,434 KB | +322.1 KB (+15.3%) | +| flash esp32s3-n32r16 | - | 2,455 KB | new | | flash esp32s3-n8r8 | 2,038 KB | 2,381 KB | +342.4 KB (+16.8%) | | flash esp32s3-zero | 1,977 KB | 2,173 KB | +195.9 KB (+9.9%) | | flash esp32s31 | 2,395 KB | 2,680 KB | +284.3 KB (+11.9%) | | flash teensy41 | - | 1,242 KB | new | | tick Effects_pipeline_builds_and_renders (desktop-macos) | 7 µs | 7 µs | +0 (+0.0%) | | tick Layouts_resize_reallocates_live (desktop-macos) | 12 µs | 10 µs | -2 (-16.7%) | -| lines core | 22,299 | 30,448 | +8,149 (+36.5%) | -| lines light | 31,054 | 31,568 | +514 (+1.7%) | -| lines platform | 16,798 | 21,750 | +4,952 (+29.5%) | -| lines ui | 11,394 | 12,957 | +1,563 (+13.7%) | -| lines test | 57,759 | 71,673 | +13,914 (+24.1%) | -| lines moondeck | 29,325 | 31,938 | +2,613 (+8.9%) | -| unit tests | 2,120 | 2,643 | +523 (+24.7%) | -| scenarios | 21 | 32 | +11 (+52.4%) | +| lines core | 22,299 | 32,227 | +9,928 (+44.5%) | +| lines light | 31,054 | 31,562 | +508 (+1.6%) | +| lines platform | 16,798 | 22,123 | +5,325 (+31.7%) | +| lines ui | 11,394 | 12,980 | +1,586 (+13.9%) | +| lines test | 57,759 | 73,772 | +16,013 (+27.7%) | +| lines moondeck | 29,325 | 32,132 | +2,807 (+9.6%) | +| unit tests | 2,120 | 2,741 | +621 (+29.3%) | +| scenarios | 21 | 35 | +14 (+66.7%) | Against the release `v6.0.0`, the fixed point a user compares with; the tables below compare against the last commit. A firmware not rebuilt since `v6.0.0` is left out, and a measure the release did not record yet, such as the code check, joins from the next one. @@ -39,25 +39,25 @@ Against the release `v6.0.0`, the fixed point a user compares with; the tables b | Target | Flash | Capacity | Used | Built | |---|---:|---:|---:|:--:| -| desktop-linux | 2,488 KB | - | - | **STALE 9d** | -| desktop-macos | 2,366 KB | - | - | yes | -| desktop-windows | 1,880 KB | - | - | **STALE 21d** | -| esp32 | 2,228 KB | 2,496 KB | 89% | yes | -| esp32-16mb | 2,301 KB | 4,096 KB | 56% | **STALE 8d** | -| esp32-eth | 1,642 KB | 2,496 KB | 66% | **STALE 40d** | -| esp32-pico | 2,058 KB | 3,072 KB | 67% | **STALE 42d** | +| desktop-linux | 2,488 KB | - | - | **STALE 11d** | +| desktop-macos | 2,422 KB (+15 KB) ⚠ | - | - | yes | +| desktop-windows | 1,880 KB | - | - | **STALE 23d** | +| esp32 | 2,246 KB | 2,496 KB | 90% | yes | +| esp32-16mb | 2,343 KB | 4,096 KB | 57% | carried 1d | +| esp32-eth | 1,642 KB | 2,496 KB | 66% | **STALE 42d** | +| esp32-pico | 2,058 KB | 3,072 KB | 67% | **STALE 44d** | | esp32-wrover | 1,801 KB | 2,496 KB | 72% | carried (age?) | -| esp32c3 | 2,280 KB | 2,496 KB | 91% | **STALE 11d** | -| esp32p4rev1-eth | 2,348 KB | 4,096 KB | 57% | carried 2d | -| esp32p4rev1-eth-wifi | 2,595 KB | 4,096 KB | 63% | carried 2d | -| esp32p4rev3-eth | 2,367 KB | 4,096 KB | 58% | yes | -| esp32s3-n16r8 | 2,417 KB (+80 B) ⚠ | 4,096 KB | 59% | yes | -| esp32s3-n32r16 | 2,426 KB | 4,096 KB | 59% | carried 5d | -| esp32s3-n8r8 | 2,381 KB | 3,072 KB | 77% | **STALE 8d** | -| esp32s3-zero | 2,173 KB | 2,496 KB | 87% | **STALE 8d** | -| esp32s31 | 2,680 KB | 4,096 KB | 65% | **STALE 8d** | +| esp32c3 | 2,280 KB | 2,496 KB | 91% | **STALE 13d** | +| esp32p4rev1-eth | 2,348 KB | 4,096 KB | 57% | carried 4d | +| esp32p4rev1-eth-wifi | 2,595 KB | 4,096 KB | 63% | carried 4d | +| esp32p4rev3-eth | 2,367 KB | 4,096 KB | 58% | carried 2d | +| esp32s3-n16r8 | 2,434 KB (+1 KB) ⚠ | 4,096 KB | 59% | yes | +| esp32s3-n32r16 | 2,455 KB | 4,096 KB | 60% | yes | +| esp32s3-n8r8 | 2,381 KB | 3,072 KB | 77% | **STALE 10d** | +| esp32s3-zero | 2,173 KB | 2,496 KB | 87% | **STALE 10d** | +| esp32s31 | 2,680 KB | 4,096 KB | 65% | **STALE 10d** | | qemu | 1,351 KB | - | - | carried (age?) | -| teensy41 | 1,242 KB | 3,456 KB | 36% | carried 6d | +| teensy41 | 1,242 KB | 3,456 KB | 36% | **STALE 8d** | `Built: yes` was measured this run. `carried (age?)` was not rebuilt either and predates this record, so its age is unknown: it dates itself on the next build. `carried Nd` was NOT rebuilt and its number is N days old, so an absent delta says nothing about the change. **STALE** marks a carry older than 7 days: the number has gone unchecked long enough that growth will surface later as one jump, blamed on whichever commit happens to rebuild that target. `Used` is against the app slot in the firmware's own partition table. @@ -65,15 +65,15 @@ Against the release `v6.0.0`, the fixed point a user compares with; the tables b | Target | Net | New | Removed | Grown | Shrunk | Other | Since | |---|---:|---:|---:|---:|---:|---:|---| -| esp32 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `80e19948` | -| esp32p4rev3-eth | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `80e19948` | -| esp32s3-n16r8 | +0.1 KB | +0.0 KB | +0.0 KB | +0.1 KB | +0.0 KB | -0.0 KB | `80e19948` | +| esp32 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `bfcc83f5` | +| esp32s3-n16r8 | +1.1 KB | +2.9 KB | -2.3 KB | +1.5 KB | -1.0 KB | +0.1 KB | `bfcc83f5` | +| esp32s3-n32r16 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `bfcc83f5` | Largest moves: - **esp32**: none -- **esp32p4rev3-eth**: none -- **esp32s3-n16r8**: `mm::ui::file10` +83 B +- **esp32s3-n16r8**: `mm::Drivers::live_` +804 B, `mm::(anonymous namespace)::writeValue(mm::JsonSink&, mm::jso` -745 B, `mm::Drivers::liveNames_` -656 B, `mm::JsonFetch::worker(void*)` -384 B, `mm::JsonFetch::serve(mm::JsonFetch::Job const&, char*)` +316 B +- **esp32s3-n32r16**: none What this commit's flash went to, symbol by symbol against the measurement at `Since`. New is new functionality, removed and shrunk are what a cleanup saved, and other is what no symbol owns, such as merged strings and alignment. A rename reads as removed plus new, which the largest moves expose. @@ -82,7 +82,7 @@ What this commit's flash went to, symbol by symbol against the measurement at `S | Target | Tick | FPS | |---|---:|---:| | desktop-linux | 5 µs | 200,000 | -| desktop-macos | 7 µs | 142,857 | +| desktop-macos | 8 µs | 125,000 | | desktop-windows | 4 µs | 250,000 | | esp32 | 1,716 µs | 582 | @@ -106,6 +106,7 @@ What this commit's flash went to, symbol by symbol against the measurement at `S | NetworkSend_one_sender_many_receivers | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 3,847 ? | - | - | | Network_hardware_reconfigures_live | 21 ? | 2,834 ? | - | 8,692 ? | - | - | - | - | 2,147 ? | 237 ? | - | - | - | - | - | - | - | - | | Palettes_a_script_drives_the_effects | 7 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | +| RAM_floor | - | - | - | 11,732 ? | 2,168 ? | - | - | - | - | - | - | - | - | - | - | - | - | - | | State_one_document_adds_an_effect_with_its_controls | 7 | - | - | 2,505 ? | - | - | - | - | - | - | - | - | - | - | - | - | - | - | | perf_full | 256 | 592 ? | 5,313 | 25,340 ? | 11,711 ? | 13,373 ? | - | - | 6,797 ? | 2,438 ? | 5,469,660 ? | - | - | - | - | - | - | - | | perf_light | 16 | 35 ? | 2,510 | 3,044 | 1,117 ? | 1,378 ? | - | - | 662 ? | 780 ? | 5,385,763 ? | - | - | - | - | - | - | - | @@ -114,7 +115,7 @@ What this commit's flash went to, symbol by symbol against the measurement at `S Microseconds. `?` marks a cell backed by fewer than 4 samples, which is a first impression rather than a percentile; several are months old and were captured during a network reconfigure, so they read as whole milliseconds. `-` means that target has never run that scenario. -**Coverage: 115/378 cells measured (30%), 28 of them with 4+ samples (7%).** The blanks are the point: a regression on a target that has never run a scenario cannot be DETECTED in it, and the target cannot be compared against the others. Filling the matrix means running the scenario suite on each bench board, which is a standing task rather than a one-off. +**Coverage: 117/396 cells measured (29%), 28 of them with 4+ samples (7%).** The blanks are the point: a regression on a target that has never run a scenario cannot be DETECTED in it, and the target cannot be compared against the others. Filling the matrix means running the scenario suite on each bench board, which is a standing task rather than a one-off. ### desktop-macos: isolated scenarios (p50 of the sample window) @@ -138,36 +139,36 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Area | Lines | Comments | Comment share | |---|---:|---:|---:| -| core | 30,448 | 6,336 | 23.1 % | -| light | 31,568 | 6,666 | 23.7 % | -| platform | 21,750 | 4,104 | 21.2 % | -| ui | 12,957 (+2) ⚠ | 3,489 | 28.5 % | -| test | 71,673 (+9) ⚠ | 7,277 | 11.8 % | -| moondeck | 31,938 (+117) ⚠ | 4,460 | 16.0 % | +| core | 32,227 (+60) ⚠ | 6,547 | 22.5 % (−0.1 %) ✓ | +| light | 31,562 (+2) ⚠ | 6,653 | 23.7 % | +| platform | 22,123 (+30) ⚠ | 4,145 | 21.0 % | +| ui | 12,980 | 3,492 | 28.5 % | +| test | 73,772 (+17) ⚠ | 7,389 | 11.6 % | +| moondeck | 32,132 | 4,462 | 15.9 % | ## Tests | Kind | Count | |---|---:| -| unit cases | 2,643 | -| scenarios | 32 | +| unit cases | 2,741 (+2) ✓ | +| scenarios | 35 | ## Complexity | Metric | Value | |---|---:| -| functions | 5,171 | -| over threshold | 349 | +| functions | 5,344 (+29) ✓ | +| over threshold | 351 (−8) ✓ | | worst CCN | 128 | ## Documentation | Metric | Value | |---|---:| -| markdown files | 162 | -| markdown lines | 30,528 (+47) ⚠ | -| plan files | 53 | -| backlog lines | 4,031 | -| lessons lines | 535 | +| markdown files | 164 | +| markdown lines | 30,957 (−6) ✓ | +| plan files | 55 | +| backlog lines | 4,083 | +| lessons lines | 546 | | CLAUDE.md lines | 119 | diff --git a/docs/reference/openapi.yaml b/docs/reference/openapi.yaml index 1352cf05e..33fd4735d 100644 --- a/docs/reference/openapi.yaml +++ b/docs/reference/openapi.yaml @@ -56,6 +56,12 @@ paths: tags: [State] operationId: applyState summary: Applies a state document as a JSON Merge Patch (RFC 7386); its rules are in Integrating. + parameters: &stateParameters + - name: live + in: query + required: false + description: 1 applies a document that drives rather than configures, such as a show building its look on another device, checked the same way and marking nothing to save. + schema: { type: integer, enum: [0, 1] } requestBody: { $ref: "#/components/requestBodies/StateDocument" } responses: &stateResponses "200": { $ref: "#/components/responses/StateResult" } @@ -68,6 +74,7 @@ paths: tags: [State] operationId: applyStatePost summary: The same as PATCH, for a client that cannot send PATCH. + parameters: *stateParameters requestBody: { $ref: "#/components/requestBodies/StateDocument" } responses: *stateResponses @@ -359,6 +366,7 @@ paths: responses: "200": { $ref: "#/components/responses/Ok" } "400": { $ref: "#/components/responses/Error" } + "404": { $ref: "#/components/responses/Error" } "500": { $ref: "#/components/responses/Error" } /api/file: diff --git a/docs/reference/performance.md b/docs/reference/performance.md index 7534f4c86..63e814a63 100644 --- a/docs/reference/performance.md +++ b/docs/reference/performance.md @@ -300,6 +300,35 @@ The exec block is the emitted machine code, so its size varies by ISA. `lines.ml --- +## Internal RAM + +What the ESP32's scarce memory holds and what freed it, measured 2026-10-21 and 2026-10-22 with BouncingBalls held fixed. The rule behind it is in [MoonCore](../explanation/architecture/mooncore.md#internal-ram); `scenario_RAM_floor` measures it on any board. + +**The S3** (shiffy, `esp32s3-n16r8`, 3,840 lights through 2 pins and the 74HCT595 expander on LCD-MM), with a browser on the UI: + +| | Free internal | Largest block | fps | +|---|---|---|---| +| Start, `ringAuto` taking all but 64 KB (28 buffers of 7 rows) | 66 KB | 30 KB | 75 | +| WiFi, lwIP and TLS buffers in PSRAM, WiFi's code in flash | 110 KB | 31 KB | 72 | +| ESP-IDF's 32 KB DMA reserve pool off (it capped the block at its own size) | 111 KB | 59 KB | 72 | +| Every plain `malloc` asking PSRAM first, smaller stacks | 132 KB | 88 KB | 71 | +| `ringAuto` sizing by lead, 3-row buffers (3 × 21, a 37 KB ring) | 145 KB | 94 KB | 81 | +| The prime's core-0 helper removed | 150 to 155 KB | 108 KB | 102 to 107 | + +**The ring's depth** (`ringAuto` off, 40 s each, frames lost): what counts is the lead over the wire in rows. Before the encode sped up, 7 × 10 and 3 × 24 ran clean and 7 × 8 lost a frame in five. After it, 3 × 18 and 2 × 26 run clean, 3 × 14 loses one every few seconds, and 1-row buffers send nothing. + +| Per row of 64 strands | Before | After | +|---|---|---| +| Emit (transpose and store) | 5,800 cycles | 3,650 (its eight stores unrolled to constant shifts, and the empty second byte of a 16-bit bus skipped) | +| Gather (correction per light) | 1,850 cycles | 1,300 (a plain RGB strip skips every optional fixture role) | +| Refill of a slice, against its wire time | 220 µs vs 207 (7 rows) | 63 µs vs 89 (3 rows) | + +**The P4** (64 lights) holds 368 to 373 KB free with a 192 KB block through the same scenario. + +**The classic ESP32 without PSRAM** (the Olimex, WiFi, 64 lights) rests at about 70 KB free. A page load over a slow link once took it to 3.6 KB. The RAM went to 34 TCP segments of 1,536 bytes waiting for acknowledgement and 8 of WiFi's receive buffers. With the sends' back-pressure an open UI and preview hold 40 to 55 KB with a 40 KB block, where they held 12 to 23 KB. On the classic, WiFi's code in flash frees only the code-only region (22 to 55 KB, MoonLive's scripts use it), no data RAM. + +**Network receive** is unaffected by WiFi's code in flash. In `scenario_NetworkReceive_two_boards`, the P4 sending to the S3, every row matched the run with it in IRAM. That is 38 to 40 frames a second up to 4,096 lights, in Art-Net, E1.31 and DDP. + ## Multi-pin LED driving (all three peripherals, 128×128 grid) The rows name the `ParallelLedDriver` backends by the class names they were measured under: `MultiPinLedDriver` is the `i80` peripheral, `MoonLedDriver` is `MoonI80`, and `ParlioLedDriver` is `Parlio`. The `peripheral` control selects the backend at one vtable dispatch per frame, so the timings hold. diff --git a/docs/reference/testing.md b/docs/reference/testing.md index ca3a1a656..54c5925c6 100644 --- a/docs/reference/testing.md +++ b/docs/reference/testing.md @@ -263,9 +263,9 @@ The run: [Run a scenario against a live device](../moonwalk.md#run-live-scenario #### Several boards -A scenario naming `"boards": ["sender", ...]` drives those boards too, each mapped to a device by `--board NAME=HOST`; it is skipped while one is missing. A step with `"device": ""` runs on that board, as an `apply_state`, a `set_control` or a `measure`. Every step's strings fill `{host}` and `{}` with the boards' addresses, so a sender's `hosts` points at its receivers. Each board is restored from its own snapshot, before the device under test. The observations are keyed by the whole combination, such as `host=esp32s3-n16r8 sender=esp32p4rev1-eth`, so a WiFi pair and an Ethernet pair keep apart. +A scenario naming `"boards": ["sender", ...]` drives those boards too, each mapped to a device by `--board NAME=HOST`; it is skipped while one is missing. A step with `"device": ""` runs on that board, as an `apply_state`, a `set_control` or a `measure`. Every step's strings fill `{host}` and `{}` with the boards' addresses, so a sender's `hosts` points at its receivers. Each board is restored from its own snapshot, before the device under test. `--keep` leaves the device under test as the last step set it, to watch or deploy what the scenario built. The observations are keyed by the whole combination, such as `host=esp32s3-n16r8 sender=esp32p4rev1-eth`, so a WiFi pair and an Ethernet pair keep apart. -A `measure` step's `read` list names extra values to report: `"Module.control"` from the state, `"Module.tick"` for a module's tick time, or `"fps"`; `":"` in front reads another board, such as `"sender:fps"`. The run ends with one row per such step, which is what a sweep is read from. +A measuring step waits `--settle` seconds before it reads, or its own `settle`. A `measure` step's `read` list names extra values to report: `"Module.control"` from the state, `"Module.tick"` for a module's tick time, or `"fps"`; `":"` in front reads another board, such as `"sender:fps"`. The run ends with one row per such step, which is what a sweep is read from. The run passes `--board sender=` per board: [MoonDeck.md](../../moondeck/MoonDeck.md#live_scenario). @@ -336,6 +336,12 @@ The runner compares each measurement to the contract and fails if the device mis - `tick_us` is a **ceiling**, the device must deliver this tick or faster. Going faster than contract is good news, never a failure. - `free_heap` is a **floor**, the device must deliver at least this much free heap. Growing heap is good news. +- `max_alloc_block`, `free_internal` and `min_stack_free` are **floors** a step opts into, one for each of these: + - the largest free block, which an allocator needing one piece depends on; + - the internal RAM WiFi, HTTP and the UI need on a board with PSRAM, where `free_heap` counts the PSRAM too; + - the headroom of the tightest task stack, which the runner names. + + The first two take the heap tolerance; a stack's headroom is exact. `scenario_RAM_floor` holds them with the UI open in a real browser, through the live runner's `browser` step (`action` open, refresh or close, with a `settle` in seconds). - Both are **hand-set promises**, not auto-captured last readings. Renegotiating a contract requires `--update-contract --reason "..."`, see below. - `set_by` records when the contract was last (re)negotiated; `reason` records why. Both stamped automatically by `--update-contract`. @@ -425,7 +431,7 @@ Every `scenario_*.json` carries top-level metadata plus a `description` per step } ``` -`name` matches the filename stem exactly; `module` drives the doc grouping and the MoonDeck filter; `also` lists peer modules. `mode` is `construct` or `mutate` (see above). `fixture` is required for `mutate` scenarios that need to run in-process. `reset` (optional but recommended for mutate scenarios) normalizes controls before measurement so contract assertions aren't coupled to run order. `contract[]` carries the per-target performance contract, see § Performance contracts. `observed[]` carries the published summary of the observations per metric (tick_us, free_heap, max_alloc_block), written on request; see § Persistent observations. Per-step `description` is recommended (a missing one falls back to the step's `name`/`op` in the generated view). Optional top-level `"live_only": true` keeps the scenario out of the in-process runner. Optional per-step `"measure": true` triggers a measurement after the step, checked against the step's `contract`. +`name` matches the filename stem exactly; `module` drives the doc grouping and the MoonDeck filter; `also` lists peer modules. `mode` is `construct` or `mutate` (see above). `fixture` is required for `mutate` scenarios that need to run in-process. `reset` (optional but recommended for mutate scenarios) normalizes controls before measurement so contract assertions aren't coupled to run order. `contract[]` carries the per-target performance contract, see § Performance contracts. `observed[]` carries the published summary of the observations per metric (tick_us, free_heap, max_alloc_block, and on a live device free_internal and min_stack_free), written on request; see § Persistent observations. Per-step `description` is recommended (a missing one falls back to the step's `name`/`op` in the generated view). Optional top-level `"live_only": true` keeps the scenario out of the in-process runner. Optional per-step `"measure": true` triggers a measurement after the step, checked against the step's `contract`. Unknown JSON keys are ignored by both runners (C++ and Python), so adding a new field is safe. @@ -460,11 +466,11 @@ Steps to run them: [Run a scenario in-process on the desktop](../moonwalk.md#run **Step ops** the in-process runner understands today: -- `set_control`: write a control on an already-added module (`id` + `key` + `value`). Mirrors `handleSetControl`: applies the typed write, calls `onControlChanged()`, and triggers `Scheduler::prepareTree()` if `affectsPrepare` returns true. Today supports Uint8 / Uint16 / Int16 / Bool / Text / Password / Select. A step may carry `"optional": true`, a best-effort write (e.g. shrink a grid that may not exist) that's skipped, not failed, when the target is absent. +- `set_control`: write a control on an already-added module (`id` + `key` + `value`). Mirrors `handleSetControl`: applies the typed write, calls `onControlChanged()`, and rebuilds the scope `prepareScope()` names if `affectsPrepare` returns true. It supports Uint8 / Uint16 / Int16 / Bool / Text / Password / Select. A step may carry `"optional": true`, a best-effort write (e.g. shrink a grid that may not exist) that's skipped, not failed, when the target is absent. - `apply_state`: a state document (`document`) through the engine `PATCH /api/state` runs, from its text as written so key order holds; it is how a scenario changes the tree. A member with a `type` adds a module and a new `type` under the same name replaces one, `null` removes one, and `"$patch":"replace"` keeps only the children listed, so with none it clears a container (the "prepare my own canvas" primitive), keeping the ones main.cpp wires. With `error`, and optionally `at`, the step expects that failure; with `optional`, an unknown type is a skip, as for a peripheral another chip has. The live runner removes the modules a document created, and restores each container's document at the end. - `round_trip_state`: read every card back as its document and apply them as one document, which must apply. What the device shows is then proven to apply as it is, and live, the body outgrows the request buffer, so the streaming route runs. - `list_row`: reach a row of a list control as the list API does. `add` appends one with those fields, and `to` moves it there. Otherwise `match` finds a row by its fields, then the step writes a `field` (a button field presses it), runs the row's action (`"apply": true`, a pad click), deletes it (`"delete": true`), or with none of those only expects it. -- `write_file` / `delete_file`: stage a file the way the editor saves one, and remove it again, so a scenario leaves the device as it found it. Both re-prepare the tree, as the live API does after a file changes. +- `write_file` / `delete_file`: stage a file the way the editor saves one, and remove it again, so a scenario leaves the device as it found it. The text is in `value`, or in a repository file named in `from`. Both re-prepare the tree, as the live API does after a file changes. - `measure`: pure measurement step. Runs warmup + measure frames, prints per-step tick / FPS / lights / heap-delta, and checks the step's `contract`. A step can also set `"measure": true` on a non-measure op (e.g. mark the last `apply_state` as the one to measure after); the runner treats either shape identically. diff --git a/docs/reference/troubleshooting.md b/docs/reference/troubleshooting.md index 8bc62a4c5..44a111c5d 100644 --- a/docs/reference/troubleshooting.md +++ b/docs/reference/troubleshooting.md @@ -44,6 +44,14 @@ Confirm the firmware is innocent before reaching for the soldering iron: [Diagno How the bench found this is in [lessons.md](../work/past/lessons.md). +## The lights stutter once a second + +At **logLevel** `Info` or above, the device writes a status line to its serial console once a second, from the loop that renders the frames. On an ESP32-S3 driving a large setup, that write stalls the loop long enough to see. Set **logLevel** on the System card to `Warn`: the line stops and the stutter with it. The first minute after boot always prints, since the installer reads the device's address from that line. + +## Free memory is low with the interface open + +The status bar's 🧠 is free internal RAM and 🧱 its largest block, the memory WiFi, the web server and the LED drivers share. A board with PSRAM keeps 100 KB free and a 50 KB block. A classic ESP32 without PSRAM runs with less. An open interface over a weak WiFi link takes more there, since what the board sends waits in that RAM until the browser acknowledges it. Ethernet, a better WiFi link or a closed preview gives it back. Where the RAM goes and what each board holds is in [performance.md](performance.md#internal-ram). + ## Colors are wrong everywhere Red where you expect green usually means the channel order does not match the strip. Set it on the driver: the [fixture profile](../moonmodules/light/supporting.md#fixtureprofiles) names which channel carries which color. diff --git a/docs/work/future/backlog-core.md b/docs/work/future/backlog-core.md index 5da703968..d425f2b30 100644 --- a/docs/work/future/backlog-core.md +++ b/docs/work/future/backlog-core.md @@ -2,6 +2,22 @@ Forward-looking to-build items for the **core / infrastructure** domain (`src/core/`, `src/platform/`, build, CI, network, persistence, UI). The light-domain counterpart is [backlog-light.md](backlog-light.md); items that genuinely span both are in [backlog-mixed.md](backlog-mixed.md). Index + overview: [README.md](index.md). Completed items are removed. +### The `.deb` ships the systemd unit, raw Ethernet included (2026-10-22, from robwomp on Discord) + +The package holds only the binary, so a Pi or NanoPi user writes `/etc/systemd/system/moonlight.service` by hand from MoonWalk's "Run at boot" step. That unit grants no `CAP_NET_RAW`, so the panel card driver's `AF_PACKET` socket fails unless MoonLight runs as root, and the `setcap` alternative is lost whenever apt replaces `/usr/bin/MoonLight`. `CAP_NET_RAW` is all MoonLight needs, since it sets no interface flags. The fix: `package_desktop.py` installs the unit with `AmbientCapabilities=CAP_NET_RAW` and a dedicated `moonlight` user, so installing is `apt install` plus `systemctl enable --now moonlight`. MoonWalk's unit step shrinks to that enable, and the `setcap` step stays for a binary run by hand. + +### Downloads off the tick path (2026-10-13) + +Every upload moves through the transfer slots; a flash download still writes its whole file on one tick, bounded per chunk by the write deadline. Moving downloads onto the slots too needs a look at how many files the UI reads at once, since two slots answer a third request with 503. Firmware uploads drain whole by design and feed the watchdog. + +### An update's connection that would not open until a reset (2026-10-22, from the internal RAM plan) + +On 3.0.0 a board could not open the update's connection (`ESP_ERR_HTTP_CONNECT`) until it was reset, with 80 KB of internal RAM free. TLS has lived in PSRAM since then (`CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC`), which may be what cured it; it has not recurred. Recorded so a recurrence is recognized: the free internal RAM and largest block at that moment are the first readings to take. + +### The HTTP server is audited for the two PHP request-handling bug classes (2026-10-21) + +PHP's critical advisory of 2026-10-21 (CVE-2026-109618, CVE-2026-109619) names two bug classes any C++ server can have. **Out of memory mid-request**: an allocation fails while a request is half built, and cleanup or a later step treats the partial state as valid. That path is live here: a classic ESP32 fell to 916 bytes free during a page load. **State outliving its request**: a reused connection slot (`wsClients_`, `streams_`) keeps a pointer, length or buffer from its previous client. The audit is a read-only pass over `HttpServerModule` and the platform TCP layer, every allocation-failure path and every slot reuse, and each finding is pinned by a desktop test that forces the failure. + ### Headers link the docs by page, not by host (2026-10-21, from the PR 150 review) Seventeen headers under `src/` carry absolute `https://moonmodules.org/MoonLight/...` links, so a docs move edits firmware sources and marks every firmware stale, as PR 150's move from `how-to/` to `reference/` did. A header names the page site-relative (`reference/hardware/panel-cards.md#...`) and `gen_help.py` resolves the host once, so the next docs move touches no source file. @@ -260,7 +276,7 @@ The "public member has no `///`" rule reads a declaration's shape with a regex a ## A capture device on Linux feeds the Picture service (robwomp, 2026-10-21) -The [Streams plan](../present/Plan-20261013%20-%20Streams,%20every%20frame%20from%20outside.md) brings capture to the ESP32: a USB grabber on the P4 ([#151](https://github.com/MoonModules/MoonLight/pull/151), [#152](https://github.com/MoonModules/MoonLight/pull/152)) and a MIPI-CSI camera. +The [Streams plan](../present/Plan-20261013%20-%20Streams%2C%20every%20frame%20from%20outside%20(partial).md) brings capture to the ESP32: a USB grabber on the P4 ([#151](https://github.com/MoonModules/MoonLight/pull/151), [#152](https://github.com/MoonModules/MoonLight/pull/152)) and a MIPI-CSI camera. On the desktop build a capture device is read through Video4Linux (V4L2), which covers USB grabbers and the HDMI and CSI inputs of boards such as Rockchip SBCs. It is one more source of the picture family: the platform layer reads frames, and the Picture service takes them as it takes every other source. robwomp has such boards and offered to test on Discord. @@ -335,7 +351,7 @@ declared rather than for a buffer to fill, and to time out on stall rather than - **Live RMII Ethernet reconfigure** — runtime PHY/pin config shipped (`ethType` + pin controls in NetworkModule, per-board defaults in `deviceModels.json`, `platform::setEthConfig`/`ethInit` dispatch). W5500 (SPI) on S3 applies **live** — `ethStop()` tears down the SPI bus and `ethInit()` re-runs on the next `loop1s()` with no reboot. RMII (classic/P4 internal EMAC) still saves config and asks for a restart to apply, because the EMAC bring-up is fiddlier to hot-cycle cleanly. Make RMII live too: a hot `esp_eth_stop` + EMAC/netif teardown + re-init on config change, matching the W5500 path, so every interface honours the no-reboot principle. - **GCC below 16 needs four warnings demoted, and nothing exercises those versions** - `-Wnull-dereference`, `-Wrestrict`, `-Wstringop-overflow` and `-Wformat-truncation` fire on provably correct code from GCC 12 through 15 (five of the twelve inside libstdc++ and glibc headers, unreachable from our source), so CMakeLists demotes them to non-fatal there and keeps them fatal on 16+. That unblocks CI and from-source builds on Debian and Raspberry Pi OS alike, but it is a suppression, not an understanding: nobody routinely compiles with 12-15, so a REAL instance of one of these on those versions is now a warning nobody reads. Revisit when the runner's default GCC reaches 16, at which point the whole block can be deleted. - **Installer UX polish** — clear "Pre-release (beta)" warning on RC/latest picks, yank-by-asset-tag instead of yank-by-release-deletion. -- **Offer MoonLight/MoonLight as a library**: a downstream sketch where another firmware/app consumes the light pipeline (or a subset) as an embeddable dependency rather than running the whole binary. `library.json` is already a PlatformIO *library* manifest, so the seed exists. When this is designed, give it a small public **identity surface**: one runtime constant the consumer reads (a `kProjectName`, likely a `ProjectInfo` bundle of name + version + url) that the network wire-strings (ArtNet/E1.31 source-name + CID), the UI banner, and any "About" string all *derive from*: the one place a consumer queries "what am I embedding." This is the genuine home for the name-centralisation that the rename ([the MoonLight plan](../past/plans/Plan-20260922%20-%20MoonLight,%20from%20v5.0.0%20to%20the%20rename%20(shipped).md)) deliberately *didn't* do: the rename is a one-time sweep (a constant would just split it), but a library consumer references the identity ongoing and widely, which is the test a constant must pass. Build it *then*, against the real library API, not speculatively now. +- **Offer MoonLight/MoonLight as a library**: a downstream sketch where another firmware/app consumes the light pipeline (or a subset) as an embeddable dependency rather than running the whole binary. `library.json` is already a PlatformIO *library* manifest, so the seed exists. When this is designed, give it a small public **identity surface**: one runtime constant the consumer reads (a `kProjectName`, likely a `ProjectInfo` bundle of name + version + url) that the network wire-strings (ArtNet/E1.31 source-name + CID), the UI banner, and any "About" string all *derive from*: the one place a consumer queries "what am I embedding." This is the genuine home for the name-centralisation that the rename ([the MoonLight plan](../past/plans/Plan-20260922%20-%20MoonLight%2C%20from%20v5.0.0%20to%20the%20rename%20(shipped).md)) deliberately *didn't* do: the rename is a one-time sweep (a constant would just split it), but a library consumer references the identity ongoing and widely, which is the test a constant must pass. Build it *then*, against the real library API, not speculatively now. - **HTTP: a request split across packets, or a body, that arrives a few ms late is dropped, intermittently (2026-08-20).** `handleConnection` runs SYNCHRONOUSLY inside `tick20ms`, so its waits are kept short to protect the render loop. A connection with no bytes yet is parked and served on a later tick (2026-10-09), but a request whose headers arrive in two parts gets **~5 ms** for the rest, and a body **~50 ms** (`empties > 50`). A client that misses either budget gets a request processed as truncated, or a `400 {"error":"incomplete request body"}`. Parking these too needs a buffer per parked connection. @@ -859,7 +875,7 @@ Today the eth-only build profile compiles WiFi out (`MM_NO_WIFI`). Turning WiFi **Hardware-limit tail (not covered by the pin check).** Pin-uniqueness rejects the common case but not the controller-count limit: the S3 has **2 I2S controllers** regardless of pins, so a 3rd mic on distinct pins passes the pin check yet fails `i2s_new_channel` at runtime. That tail is already handled — the platform I2S init returns false on failure (no panic, module stays `inited_=false`); verified live (4 pinned AudioModules → error spam, no crash). So scope = pin-uniqueness check + the existing graceful-degrade; don't try to make the pin check also model controller counts. -**Related:** the shipped "disabling releases resources" work (see [past plans](../past/plans/index.md)) — a disabled module freeing its pins is what lets the same GPIO be reassigned live without a conflict-reject. +**Related:** the shipped "disabling releases resources" work (see [past plans](../past/index.md)) — a disabled module freeing its pins is what lets the same GPIO be reassigned live without a conflict-reject. ### PinsModule — strict reject-on-add mode (the one remaining increment) diff --git a/docs/work/future/backlog-light.md b/docs/work/future/backlog-light.md index c7d52a2fb..bb5e6c327 100644 --- a/docs/work/future/backlog-light.md +++ b/docs/work/future/backlog-light.md @@ -20,6 +20,22 @@ Forward-looking to-build items for the **light domain** (`src/light/`: drivers, ## Effects +### The ring's refill jitter (2026-10-22, from the internal RAM plan) + +LCD-MM's refill interrupt fires every 89 µs at 3-row slices, yet almost every second it goes about 1.1 ms without firing. +So `ringAuto` keeps 54 rows of lead (3 × 21 buffers, 37 KB on shiffy), where hpwit's S3 driver runs on 10. + +Measured with `ringDbg`'s `gap` (within a frame, `@` the slice it ended at), `ml` (the fewest slices of lead) and `c` (the interrupt's core). The gap ends mid-frame as often as at the idle tail. + +Ruled out: +- the flash cache: `co` stays 0; +- the core: registered on core 1 it was the same, and it broke the cache guard, so it stays on core 0; +- `mmSnap`: removed. + +Next, the gap measured against the frame's own events, the arm and the snapshot, then against other interrupts on core 0. + +A lead of 10 rows would free most of the ring's internal RAM. + ### A preview GIF for every MoonLive script (2026-10-19) The 140 compiled effects each show a GIF on their catalog card (`docs/assets/light/effects/.gif`, 25 MB in all); the 49 MoonLive effect scripts show none. @@ -896,7 +912,7 @@ The LED-driver increments **shipped**: increment 1 (RMT/WS2812B single-strand on **Why it waits.** It is fourteen effects' worth of change across audio-reactive and simulation families, each needing its own judgement about whether to clear, fade, or seed differently — not a mechanical sweep. `unit_Effects_gridsweep.cpp` already measures it (`afterFirst`) and asserts only the settled frame, so the number is visible without blocking. -- **A scripted modifier that animates every frame** (2026-10-11): `MoonLiveModifier` compiles a `modifyLive()` entry, so a script moves lights per frame with `t` (scroll, wobble, sweep) as the compiled Rotate does. Geometry only: dimming is a blend mode or an effect's `mulRGB`, per [the layer compositing plan](../present/Plan-20261011%20-%20Layer%20compositing,%20masks%20and%20transitions.md). Built once a script needs it; the cost to measure is one compiled-script call per light per frame, on an S3 at its largest grid. +- **A scripted modifier that animates every frame** (2026-10-11): `MoonLiveModifier` compiles a `modifyLive()` entry, so a script moves lights per frame with `t` (scroll, wobble, sweep) as the compiled Rotate does. Geometry only: dimming is a blend mode or an effect's `mulRGB`, per [the layer compositing plan](../present/Plan-20261011%20-%20Layer%20compositing%2C%20masks%20and%20transitions%20(partial).md). Built once a script needs it; the cost to measure is one compiled-script call per light per frame, on an S3 at its largest grid. - **A scripted modifier needs a way to drop a light** (2026-08-10). A coordinate slot is a byte, so a script that computes past 255 wraps: `shift.mlv` with a large `amount` lands lights back at the left edge instead of walking them off it. The Layer already drops an out-of-bounds position, but a script has no way to SAY out-of-bounds — every value it can write is a valid coordinate. Needs a sentinel the binding recognizes (or a wider coordinate slot), at which point the "walks off the edge" behavior a scroll modifier wants becomes expressible. - **A scripted layout loses its control values across a reboot** (2026-08-11). Persistence SAVES them correctly (`/.config/Layouts.json` holds `"0.width":64`), but on boot they are loaded into a control set that does not exist yet: a script's controls are created by `defineControls`, which can only publish what the ENGINE declared, and the engine does not compile until `prepare()` — Scheduler phase 4, after the phase-2b re-bind. So the loaded value has nowhere to land and the script's declared default wins; a saved 64×64 grid comes back 16×16. Verified on the P4 bench. diff --git a/docs/work/future/backlog-moonlive.md b/docs/work/future/backlog-moonlive.md index ba1137525..b74cd8fa0 100644 --- a/docs/work/future/backlog-moonlive.md +++ b/docs/work/future/backlog-moonlive.md @@ -531,7 +531,7 @@ Two effects written in one session on real hardware: `stadbeest-walk.mle` (a wal ### 12. A one-to-one port of hpwit's scripts, for a like-for-like comparison (2026-10-19) -hpwit's compiler (`hpwit/new-parser`, `asmparser2`) runs whole C-like programs. His `multi_effect_controller.sc` (800 lines) and the five `ProceduralEffects` were ported to MoonLive on 2026-10-18 (`moonlive/effects/hpwit/`), but as the same LOOK rather than the same program, so code size and frame time do not compare like for like. A one-to-one port needs the language to accept his programs as written. Each construct was compiled on the desktop on 2026-10-19: +hpwit's compiler (`hpwit/new-parser`, `asmparser2`) runs whole C-like programs. His `multi_effect_controller.sc` (800 lines) and the five `ProceduralEffects` were ported to MoonLive on 2026-10-18 (`moonlive/projects/hpwit/`), but as the same LOOK rather than the same program, so code size and frame time do not compare like for like. A one-to-one port needs the language to accept his programs as written. Each construct was compiled on the desktop on 2026-10-19: | his scripts use | MoonLive today | what it takes | work | |---|---|---|---| @@ -548,7 +548,7 @@ hpwit's compiler (`hpwit/new-parser`, `asmparser2`) runs whole C-like programs. Work is in days of agent implementation with the product owner's review, for the parser, the IR and all five backends, each with its tests; S is under a day, M one to two, L a week, XL two. In all about five weeks with `float` mapped onto `fixed`, about seven with a real `float`, the structs and the arena being most of it. -Most rows are general language growth the MoonLive scripts want anyway; the `main()` shim and his host bindings exist only for the comparison, and stay behind a compatibility table rather than entering the standard vocabulary. **Decide first** whether the comparison is worth the compatibility layer, or whether comparing on the ported looks, now in `moonlive/effects/hpwit/`, is enough. +Most rows are general language growth the MoonLive scripts want anyway; the `main()` shim and his host bindings exist only for the comparison, and stay behind a compatibility table rather than entering the standard vocabulary. **Decide first** whether the comparison is worth the compatibility layer, or whether comparing on the ported looks, now in `moonlive/projects/hpwit/`, is enough. ## How to know a step landed diff --git a/docs/work/future/input-mapping-analysis.md b/docs/work/future/input-mapping-analysis.md index 58cdf211c..8ac88f8a5 100644 --- a/docs/work/future/input-mapping-analysis.md +++ b/docs/work/future/input-mapping-analysis.md @@ -251,7 +251,7 @@ category confusing: - **Ultrasonic (HC-SR04)** is a trigger pulse plus an echo whose *width* is the distance. Neither I2C nor a plain read: it needs a **pulse-timing seam** that does not exist. The textbook answer on ESP32 is the same PCNT/RMT machinery an encoder wants, so the two features share a dependency. -- **ToF** is I2C, so it rides the layer above. A **mmWave radar** such as the HLK-LD2450 is UART, so it needs the UART seam the [audio-reactive plan](../present/Plan-20261013%20-%20Audio%20reactive%20live%20performance.md) names. +- **ToF** is I2C, so it rides the layer above. A **mmWave radar** such as the HLK-LD2450 is UART, so it needs the UART seam the [audio-reactive plan](../present/Plan-20261013%20-%20Audio%20reactive%20live%20performance%20%28not%20started%29.md) names. **I2S is audio-shaped, and should stay that way.** The platform's I2S surface is `audioMicInit` / `audioMicRead`, deliberately named for what it does rather than for the peripheral, and diff --git a/docs/work/future/moonwalk.md b/docs/work/future/moonwalk.md index bbd5854d9..fe9fc622e 100644 --- a/docs/work/future/moonwalk.md +++ b/docs/work/future/moonwalk.md @@ -1,6 +1,6 @@ # MoonWalk -Step 2 of [Processes, described once](../present/Plan-20261015%20-%20Processes%2C%20described%20once.md): MoonWalk, the model every process is written in and the map that shows them, chosen against [the inventory](processes-inventory.md) and approved on 2026-10-16. The schema is [moonwalk/moonwalk.schema.json](../../../moonwalk/moonwalk.schema.json). +Step 2 of [Processes, described once](../present/Plan-20261015%20-%20Processes%2C%20described%20once%20(partial).md): MoonWalk, the model every process is written in and the map that shows them, chosen against [the inventory](processes-inventory.md) and approved on 2026-10-16. The schema is [moonwalk/moonwalk.schema.json](../../../moonwalk/moonwalk.schema.json). ## What it must do diff --git a/docs/work/future/processes-inventory.md b/docs/work/future/processes-inventory.md index 4949c2fa6..9e2e6693c 100644 --- a/docs/work/future/processes-inventory.md +++ b/docs/work/future/processes-inventory.md @@ -1,6 +1,6 @@ # Processes inventory -Step 1 of [Processes, described once](../present/Plan-20261015%20-%20Processes%2C%20described%20once.md), the Extract phase: every process MoonLight has, recorded by its content rather than the format it is stored in, counted on 2026-10-16 by three read-only Researchers. Part 1 is the end user's journey, Part 2 the contributor's, product owner's, agents' and CI's, and Part 3 the executable step lists: the scenarios and the MoonTube clips. +Step 1 of [Processes, described once](../present/Plan-20261015%20-%20Processes%2C%20described%20once%20(partial).md), the Extract phase: every process MoonLight has, recorded by its content rather than the format it is stored in, counted on 2026-10-16 by three read-only Researchers. Part 1 is the end user's journey, Part 2 the contributor's, product owner's, agents' and CI's, and Part 3 the executable step lists: the scenarios and the MoonTube clips. ## What it found diff --git a/docs/work/past/index.md b/docs/work/past/index.md index 8117330f1..23fd97bfc 100644 --- a/docs/work/past/index.md +++ b/docs/work/past/index.md @@ -4,7 +4,7 @@ Shipped plans and dated records. **A shipped plan is a working document, not testimony.** Whatever its PR already carries is duplication and can be trimmed or deleted. What earns keeping is the part no PR and no commit holds: the alternatives considered and rejected, and the reason the chosen shape won. -`plans/` holds the plans that shipped before the current rule, when a plan was archived rather than folded into its PR. They map onto merged PRs, and the mapping is the next thing to write here. From now on a plan ends at its PR ([CLAUDE.md, Branch](../../../CLAUDE.md#main-and-branch)), so this folder stops growing. +`plans/` holds every plan that left `present`, each named with its status: `(shipped)` once every step is built and verified, the judgment steps included, or `(superseded)`, `(cancelled)` or `(attempted, reverted)`. A plan moves here in the pull request that ships or closes it, the product owner's call ([CLAUDE.md, Branch](../../../CLAUDE.md#main-and-branch)). The prior-version inventories beside it state what was true at a moment and stay unrewritten. Release notes live on [the GitHub release](https://github.com/MoonModules/MoonLight/releases) itself, which is their one home: a copy here drifts from what shipped, and the links in one pinned to an older tag went stale exactly that way. @@ -15,31 +15,3 @@ The prior-version inventories beside it state what was true at a moment and stay The surveys of v1, v2 and MoonLight that once sat here are gone. They were written to decide what to harvest into v3, that decision shipped, and each described code that lives in its own repository and in git. The LED-driver and shift-register design analyses moved to [future](../future/index.md), where the other research documents sit: they were written to decide what to build, and that is what future holds. - -## Plans and their PRs - -21 plans, each mapped to the merged pull request that carries the same design plus the diff that implemented it. The mapping is derived from title words and merge date, so it is a starting point rather than a record: a `?` means no confident match, and any row is worth checking before it is trusted. 4 need a hand. - -| Date | Plan | PR | -|---|---|---| -| 2026-08-13 | [MoonLive on a stack machine: the frame is where values live](plans/Plan-20260813%20-%20MoonLive%20on%20a%20stack%20machine%20%E2%80%94%20the%20frame%20is%20where%20values%20live%20%28shipped%29.md) | [#63](https://github.com/MoonModules/MoonLight/pull/63) | -| 2026-08-17 | [MoonLive scripts are classes](plans/Plan-20260817%20-%20MoonLive%20scripts%20are%20classes%20%28shipped%29.md) | [#65](https://github.com/MoonModules/MoonLight/pull/65) | -| 2026-08-18 | [A file editor control and a filesystem change seam](plans/Plan-20260818%20-%20A%20file%20editor%20control%20and%20a%20filesystem%20change%20seam%20%28shipped%29.md) | [#67](https://github.com/MoonModules/MoonLight/pull/67) | -| 2026-08-21 | [MoonLive on Windows](plans/Plan-20260821%20-%20MoonLive%20on%20Windows%20%28x86_64%20host-JIT%20backend%29%20%28shipped%29.md) | [#73](https://github.com/MoonModules/MoonLight/pull/73) | -| 2026-08-21 | [Particles in MoonLive, plus a fade builtin](plans/Plan-20260821%20-%20Particles%20in%20MoonLive%2C%20plus%20a%20fade%20builtin%20%28shipped%29.md) | [#71](https://github.com/MoonModules/MoonLight/pull/71) | -| 2026-08-22 | [Raw L2 Ethernet on Windows](plans/Plan-20260822%20-%20Raw%20L2%20Ethernet%20on%20Windows%20%28shipped%29.md) | [#73](https://github.com/MoonModules/MoonLight/pull/73) | -| 2026-08-23 | [A Windows installer, and settings that persist](plans/Plan-20260823%20-%20A%20Windows%20installer%2C%20and%20settings%20that%20persist%20%28shipped%29.md) | [#74](https://github.com/MoonModules/MoonLight/pull/74) | -| 2026-08-23 | [Five types for MoonLive scripts](plans/Plan-20260823%20-%20Five%20types%20for%20MoonLive%20scripts%20%28shipped%29.md) | [#77](https://github.com/MoonModules/MoonLight/pull/77) | -| 2026-08-24 | [NDI output](plans/Plan-20260824%20-%20NDI%20output%20%28shipped%29.md) | [#80](https://github.com/MoonModules/MoonLight/pull/80) | -| 2026-08-25 | [A lossy channel for the preview](plans/Plan-20260825%20-%20A%20lossy%20channel%20for%20the%20preview%20%28shipped%29.md) | [#81](https://github.com/MoonModules/MoonLight/pull/81) | -| 2026-08-25 | [Client-driven preview adaptation](plans/Plan-20260825%20-%20Client-driven%20preview%20adaptation%20%28superseded%29.md) | [#81](https://github.com/MoonModules/MoonLight/pull/81) | -| 2026-08-25 | [Lean preview transport](plans/Plan-20260825%20-%20Lean%20preview%20transport%20%28shipped%29.md) | [#81](https://github.com/MoonModules/MoonLight/pull/81) | -| 2026-08-27 | [HLS on ESP32-P4](plans/Plan-20260827%20-%20HLS%20on%20ESP32-P4%20%28shipped%29.md) | [#78](https://github.com/MoonModules/MoonLight/pull/78) | -| 2026-08-27 | [HLS streaming driver](plans/Plan-20260827%20-%20HLS%20streaming%20driver%20%28shipped%29.md) | [#85](https://github.com/MoonModules/MoonLight/pull/85) | -| 2026-08-27 | [Raw-L2 interface dropdown](plans/Plan-20260827%20-%20Raw-L2%20interface%20dropdown%20%28shipped%29.md) | [#85](https://github.com/MoonModules/MoonLight/pull/85) | -| 2026-08-27 | [Sprites and flying toasters](plans/Plan-20260827%20-%20Sprites%20and%20flying%20toasters%20%28shipped%29.md) | [#86](https://github.com/MoonModules/MoonLight/pull/86) | -| 2026-08-30 | [Ship the MoonLive script library](plans/Plan-20260830%20-%20Ship%20the%20MoonLive%20script%20library%20%28shipped%29.md) | [#89](https://github.com/MoonModules/MoonLight/pull/89) | -| 2026-08-31 | [Scripts declare dimensions and tags](plans/Plan-20260831%20-%20Scripts%20declare%20dimensions%20and%20tags%20%28shipped%29.md) | [#89](https://github.com/MoonModules/MoonLight/pull/89) | -| 2026-09-22 | [MoonLight, from v5.0.0 to the rename](plans/Plan-20260922%20-%20MoonLight%2C%20from%20v5.0.0%20to%20the%20rename%20%28shipped%29.md) | [#119](https://github.com/MoonModules/MoonLight/pull/119) | -| 2026-09-22 | [RTSP video out on the P4 and the desktop](plans/Plan-20260922%20-%20RTSP%20video%20out%20on%20the%20P4%20and%20the%20desktop%20%28shipped%29.md) | [#110](https://github.com/MoonModules/MoonLight/pull/110) | -| 2026-10-02 | [One way to send to many receivers](plans/Plan-20261002%20-%20One%20way%20to%20send%20to%20many%20receivers%20%28shipped%29.md) | [#123](https://github.com/MoonModules/MoonLight/pull/123) | diff --git a/docs/work/past/lessons.md b/docs/work/past/lessons.md index 16cd0a1f6..95fae7669 100644 --- a/docs/work/past/lessons.md +++ b/docs/work/past/lessons.md @@ -3,7 +3,7 @@ Hard-won debugging lessons and gotchas, recorded with the code that proved them. The PR-merge *carry-forward* gate writes new entries here (CLAUDE.md § Lifecycle Events). This is the **lesson** record: a bug, its root cause, and the fix. Two neighbours hold the other genres, and a lesson belongs in whichever fits: - A durable **rule** that graduated from a lesson lives in [CLAUDE.md](../../../CLAUDE.md) or [coding-standards.md](../../contributing/coding-standards.md), not here. -- The forward-looking **design intent** of a feature is a [plan](plans/index.md). +- The forward-looking **design intent** of a feature is a [plan](index.md). Entries run oldest-first by branch. A lesson fully absorbed into a rule doc or the code is pruned (per *Mandatory subtraction*): the git history is the permanent record, this file is the working narrative on top. @@ -533,3 +533,14 @@ Two tests were written and both passed against the unfixed code before one final Three weeks without a Windows run left three build breaks waiting, while CI's `build-windows-x64` job was green on the same commit. They were not one bug: `if (anyKnownChip)` and `if (i2sLanes > 0)` are compile-time constants that **this** MSVC (19.44.35227) reports as C4127 and the toolset `windows-latest` pins does not, and `/WX` turns either into C2220. A non-ASCII literal raised C4566, which is not a toolset question at all but the system code page: MSVC alone assumes it for both the source and execution charset, so `/utf-8` settles it the way GCC and clang already behave. On the Python side the same code page is the default for `read_text`, `write_text`, `open` and `subprocess(text=True)`, and `PYTHONIOENCODING` masks only stdio, so a UTF-8 doc read crashes a script whose output looked fine. **The rule: a platform in CI is one toolchain on that platform, not the platform.** `windows-latest` is a moving pin, so green there says the code built with that toolset on that runner image, which is a weaker claim than it reads as. `if constexpr` is the textbook answer to C4127 and says what was meant anyway; explicit `encoding="utf-8"` is the answer everywhere Python touches a file. The second half of the lesson is cheaper than the first: the drift is silent and the cost scales with how long the platform goes unexercised, so the fix is to run it, not to reason about it. + +## Lessons from the internal RAM plan (2026-10-22) + +- **Measure at the low point, not on average.** Free RAM read once a second said 70 KB on a classic board whose page load dropped it to 3.6 KB; a task sampling every tick and recording the heap's block sizes at each new low named the owner (1,536-byte TCP segments) in one run. +- **A block size is a fingerprint.** 32 KB blocks were ESP-IDF's DMA reserve pool, 1,536 bytes a TCP segment, 1,664 WiFi's receive buffers, 4,096 the ring's DMA nodes; matching sizes to owners attributed most of the heap without a tracer. +- **The obvious owner can be wrong.** Thirty-two queued packets matched WiFi's transmit queue of 32 exactly, and halving that queue changed nothing: they were TCP's. +- **A tick figure is wall-clock.** Control read 1.4 ms per tick because a helper task on its core preempted it; its own work was 150 µs. The helper itself, meant to halve the ring's prime, cost 12 fps by preempting the effects. +- **A guard that holds on one core can fail on the other.** A cache-safe interrupt's cache-off check held on the core doing flash writes; moved to the other core, the cache went off mid-refill and the board panicked. Whatever such an interrupt reads must be internal, whichever core it runs on. +- **Check the compiler's output before optimizing the algorithm.** Removing branches and a whole wasted transpose gained nothing measurable; the disassembly showed the store loop's variable shifts costing more than the transpose, and unrolling them gained 37%. +- **A depth that runs clean is the measurement, not free RAM.** `ringAuto` took 28 buffers because RAM allowed it; 21 smaller ones run clean, and the RAM went back to WiFi and the UI. +- **Two fix attempts, then stop.** The ring's 1.1 ms of jitter survived two; it is in the backlog with its measurements rather than chased further. diff --git a/docs/work/present/Plan-20260915 - Native HUB75 output (shipped).md b/docs/work/past/plans/Plan-20260915 - Native HUB75 output (shipped).md similarity index 78% rename from docs/work/present/Plan-20260915 - Native HUB75 output (shipped).md rename to docs/work/past/plans/Plan-20260915 - Native HUB75 output (shipped).md index 97e07a006..a5ed92f31 100644 --- a/docs/work/present/Plan-20260915 - Native HUB75 output (shipped).md +++ b/docs/work/past/plans/Plan-20260915 - Native HUB75 output (shipped).md @@ -4,7 +4,7 @@ Answers [issue #102](https://github.com/MoonModules/MoonLight/issues/102). Phase ## The gap -MoonLight drives HUB75 panels one way: [PanelCardDriver](../../../src/light/drivers/PanelCardDriver.h) emits ColorLight frames over raw Ethernet to a 5A-75B/E receiving card, which does the HUB75 driving. That path is excellent above roughly 16,384 pixels and it is what the [panel-cards guide](../../reference/hardware/panel-cards.md) documents. +MoonLight drives HUB75 panels one way: [PanelCardDriver](../../../../src/light/drivers/PanelCardDriver.h) emits ColorLight frames over raw Ethernet to a 5A-75B/E receiving card, which does the HUB75 driving. That path is excellent above roughly 16,384 pixels and it is what the [panel-cards guide](../../../reference/hardware/panel-cards.md) documents. Below that size it is the wrong shape. The user buys a receiving card, configures it with LEDvision (a Windows tool, not ours), and runs a dedicated Ethernet link, all to light one 64×64 panel that a $10 board could drive from its own pins. WLED does this natively; MoonLight does not. That is the gap, and it is the whole of Phase 1. @@ -18,7 +18,7 @@ frame bytes = (height / 2) × width × bitDepth Measured against the peripherals MoonLight already has: -Measured from the encoder's own formula (`Hub75Geometry::frameBytes`, pinned by [unit_Hub75Slots.cpp](../../../test/unit/light/unit_Hub75Slots.cpp)), at 1/32 scan: +Measured from the encoder's own formula (`Hub75Geometry::frameBytes`, pinned by [unit_Hub75Slots.cpp](../../../../test/unit/light/unit_Hub75Slots.cpp)), at 1/32 scan: | Panels | Geometry | 4-bit | 6-bit | Parlio (65,535 B cap) | LCD_CAM (internal RAM) | |---|---|---:|---:|:--|:--:| @@ -28,9 +28,9 @@ Measured from the encoder's own formula (`Hub75Geometry::frameBytes`, pinned by Two bytes a slot, since the bus word is 16 bits. **Four panels miss the Parlio cap even at 4-bit, by one byte**, so four panels are an LCD_CAM job at any depth, and Parlio carries one panel at up to 6-bit. The encoder offers 2 to 6 bits; 8-bit waits on repeating the upper planes (backlog). -**This is the fact the issue does not account for.** #102 proposes the P4-Nano as lead candidate because Espressif's reference uses PARLIO, but [platform_esp32_parlio.cpp:238](../../../src/platform/esp32/platform_esp32_parlio.cpp) records the hardware cap: `kParlioMaxTransferBytes = 0x7FFFF / 8` = 65,535 bytes, width-invariant. PARLIO carries one panel comfortably, reaches its limit at four, and cannot do sixteen at any useful depth. +**This is the fact the issue does not account for.** #102 proposes the P4-Nano as lead candidate because Espressif's reference uses PARLIO, but [platform_esp32_parlio.cpp:238](../../../../src/platform/esp32/platform_esp32_parlio.cpp) records the hardware cap: `kParlioMaxTransferBytes = 0x7FFFF / 8` = 65,535 bytes, width-invariant. PARLIO carries one panel comfortably, reaches its limit at four, and cannot do sixteen at any useful depth. -The route that scales is the one the [shift-register analysis](../future/shift-register-driver-analysis.md) already established for a structurally identical problem: **i80/LCD_CAM, whose DMA reaches PSRAM**, proven on this exact path to 16,384 lights. `hasLcdCam` is true on S3, P4 and S31. +The route that scales is the one the [shift-register analysis](../../future/shift-register-driver-analysis.md) already established for a structurally identical problem: **i80/LCD_CAM, whose DMA reaches PSRAM**, proven on this exact path to 16,384 lights. `hasLcdCam` is true on S3, P4 and S31. ### GPIO requirements @@ -43,21 +43,21 @@ HUB75 is pin-hungry in a way the memory table hides. One port needs: | Control | `clk lat oe` | 3 | | | **Total** | **12-14** | -Against the per-chip free sets in [gpio-usage.md](../../reference/hardware/gpio-usage.md), which exclude flash/PSRAM, input-only, USB, UART0 and strapping pins: +Against the per-chip free sets in [gpio-usage.md](../../../reference/hardware/gpio-usage.md), which exclude flash/PSRAM, input-only, USB, UART0 and strapping pins: | Chip | Usable output GPIOs | 1/16 scan (13 pins) | 1/32 scan (14 pins) | |---|---:|:--:|:--:| | **ESP32-S3** (N16R8) | 16 | ✅ 3 spare | ✅ 2 spare | | **ESP32-P4** (P4-NANO) | 20 | ✅ 7 spare | ✅ 6 spare | -| **ESP32-S31** | board-specific | take from the [coreboard reference](../../reference/hardware/esp32-s31-coreboard.md) | | +| **ESP32-S31** | board-specific | take from the [coreboard reference](../../../reference/hardware/esp32-s31-coreboard.md) | | | **ESP32 classic** | 13 | ❌ | ❌ | **This settles the target list, and it removes a chip the issue implies.** The classic ESP32 has 13 usable output GPIOs and would need every one for a 1/16-scan port, leaving nothing for the LED strand, a button or a mic; a 1/32 panel does not fit at all. gpio-usage.md already records the same squeeze for 16-lane i80 ("a 16-lane set must borrow strap pins, and GPIO 12 is the flash-voltage strap: driving it at reset can brick the boot"). So HUB75 is an **S3, P4 and S31 feature**, and the classic is out on pins before memory is even considered. Two consequences for the plan: -- **The board catalog entries in step 5 are load-bearing, not polish.** An S3 has exactly 3 spare pins at 1/16 scan, so a user picking 13 by hand will land on octal-PSRAM (33-37), USB (19-20), UART0 (43-44) or a strap (0, 3, 45, 46). [PinsModule](../../moonmodules/core/system.md) flags a conflict after the fact; a catalog entry prevents it. -- **Defaults stay unset.** Even with the pins available, a default would guess the user's wiring. Same rule the LED drivers follow, and the P4's own history is the argument: its first LED-driver default landed on strapping pins ([lessons.md](../past/lessons.md)). +- **The board catalog entries in step 5 are load-bearing, not polish.** An S3 has exactly 3 spare pins at 1/16 scan, so a user picking 13 by hand will land on octal-PSRAM (33-37), USB (19-20), UART0 (43-44) or a strap (0, 3, 45, 46). [PinsModule](../../../moonmodules/core/system.md) flags a conflict after the fact; a catalog entry prevents it. +- **Defaults stay unset.** Even with the pins available, a default would guess the user's wiring. Same rule the LED drivers follow, and the P4's own history is the argument: its first LED-driver default landed on strapping pins ([lessons.md](../lessons.md)). ### Refresh @@ -69,7 +69,7 @@ frame time = (height / 2) x bitDepth x line time refresh = 1 / frame time ``` -Predicted at a 20 MHz shift clock (the rate the i80 path already runs, per the [shift-register analysis](../future/shift-register-driver-analysis.md)), ignoring latch overhead, so these are ceilings rather than promises: +Predicted at a 20 MHz shift clock (the rate the i80 path already runs, per the [shift-register analysis](../../future/shift-register-driver-analysis.md)), ignoring latch overhead, so these are ceilings rather than promises: | Geometry | 4-bit | 6-bit | 8-bit | |---|---:|---:|---:| @@ -85,11 +85,11 @@ Predicted at a 20 MHz shift clock (the rate the i80 path already runs, per the [ ### The driver -`Hub75Driver : DriverBase`, registered in [main.cpp](../../../src/main.cpp) beside the others, gated on `platform::hasLcdCam || platform::parlioLanes` so it is offered only where it can run — the same inert-on-wrong-chip rule [ParlioLedDriver](../../../src/light/drivers/ParlioLedDriver.h) follows. That gate happens to exclude the classic ESP32, which § GPIO requirements shows could not host a port anyway. +`Hub75Driver : DriverBase`, registered in [main.cpp](../../../../src/main.cpp) beside the others, gated on `platform::hasLcdCam || platform::parlioLanes` so it is offered only where it can run — the same inert-on-wrong-chip rule [ParlioLedDriver](../../../../src/light/drivers/ParlioLedDriver.h) follows. That gate happens to exclude the classic ESP32, which § GPIO requirements shows could not host a port anyway. It is **not** a `LedPeripheral` backend. That seam is shaped around the parallel WS2812 orchestrator (`supportsPinExpander`, `lanesAvailable`, slot encoding), and HUB75 shares none of it: different wire protocol, different buffer shape, different control set. Reusing it would mean a base class whose every method one subclass ignores, which is the abstraction CLAUDE.md's minimalism rule exists to prevent. What they legitimately share is the **platform DMA seam**, and that is where the sharing belongs. -Controls, all defaulting to unset because a soldered pin must never be guessed ([lessons.md](../past/lessons.md)): +Controls, all defaulting to unset because a soldered pin must never be guessed ([lessons.md](../lessons.md)): | Control | Why | |---|---| @@ -101,7 +101,7 @@ Controls, all defaulting to unset because a soldered pin must never be guessed ( | `sPWM` | The [WLED-MM sPWM commit](https://github.com/MoonModules/WLED-MM/commit/778ae558) case | | `peripheral` | Which silicon block drives the panel, where the chip has more than one | -**Geometry is not a driver control.** [PanelLayout](../../../src/light/layouts/PanelLayout.h) and [PanelsLayout](../../../src/light/layouts/PanelsLayout.h) already model a serpentine panel and an M×N tiling of them, with configurable axis order and per-axis direction. A layout emits coordinates and the driver owns pins: that separation is the architecture, and HUB75 must not re-litigate it. The issue asks for "UI features to set panel dimensions and arrangement", and the honest answer is that MoonLight already has them. +**Geometry is not a driver control.** [PanelLayout](../../../../src/light/layouts/PanelLayout.h) and [PanelsLayout](../../../../src/light/layouts/PanelsLayout.h) already model a serpentine panel and an M×N tiling of them, with configurable axis order and per-axis direction. A layout emits coordinates and the driver owns pins: that separation is the architecture, and HUB75 must not re-litigate it. The issue asks for "UI features to set panel dimensions and arrangement", and the honest answer is that MoonLight already has them. ### The platform seam @@ -115,7 +115,7 @@ bool hub75Transmit(Hub75Handle& h, uint8_t buffer); void hub75Deinit(Hub75Handle& h); ``` -Two backends behind it, and **the user picks which**, through a `peripheral` select exactly like the one [ParallelLedDriver](../../../src/light/drivers/ParallelLedDriver.h) already carries. Each backend self-registers at static init, and `main.cpp` links it only where its SOC macro says the silicon exists, so the dropdown's options are per-chip rather than a fixed list: LCD_CAM on an S3 or S31, LCD_CAM and PARLIO on a P4, nothing on a classic. +Two backends behind it, and **the user picks which**, through a `peripheral` select exactly like the one [ParallelLedDriver](../../../../src/light/drivers/ParallelLedDriver.h) already carries. Each backend self-registers at static init, and `main.cpp` links it only where its SOC macro says the silicon exists, so the dropdown's options are per-chip rather than a fixed list: LCD_CAM on an S3 or S31, LCD_CAM and PARLIO on a P4, nothing on a classic. **An earlier draft of this plan had the platform choose, on memory alone. That was wrong, and the reason is contention rather than capacity.** A P4 has both peripherals and only one of each. A user driving WS2812 strips from PARLIO needs HUB75 on LCD_CAM; another user wants the reverse. Both are correct, the difference is what else is plugged into that board, and the platform cannot know it. The sibling claim guard stops two drivers colliding on one block, but it cannot guess which driver should win — that is the user's call, and a select is how the repo already asks it. @@ -127,7 +127,7 @@ A desktop stub returning false from everything keeps `mm_tests` and every non-HU The one genuinely new piece: rendered RGB → bit-plane-major scan buffer. Per bit plane, per scan row, pack six colour bits per pixel-column into the parallel word alongside the row address and the latch/OE timing. -This is the same class of work as [ParallelSlots.h](../../../src/light/drivers/ParallelSlots.h) (serialise a frame through a parallel bus under a DMA ceiling) and is testable the same way: a host unit test that walks a known pattern through the encoder and asserts the exact bytes, including a walking-one per colour line. The '595 investigation's hardest lesson was that an encoder proven byte-for-byte on the host still left a wall-visible artifact, so the test pins the encoder and hardware verification stays the PO's. +This is the same class of work as [ParallelSlots.h](../../../../src/light/drivers/ParallelSlots.h) (serialise a frame through a parallel bus under a DMA ceiling) and is testable the same way: a host unit test that walks a known pattern through the encoder and asserts the exact bytes, including a walking-one per colour line. The '595 investigation's hardest lesson was that an encoder proven byte-for-byte on the host still left a wall-visible artifact, so the test pins the encoder and hardware verification stays the PO's. ## End states considered @@ -146,7 +146,7 @@ This is the same class of work as [ParallelSlots.h](../../../src/light/drivers/P 3. **`Hub75Driver`.** Controls, pin claims through the existing `PinsModule` registry, status line reporting geometry and achieved refresh, registration in `main.cpp`. 4. **A backend registry, then the PARLIO backend.** The seam built in step 1 picks its peripheral internally; this turns it into an interface the driver selects, mirroring `LedPeripheral` without reusing it (that one is WS2812-shaped — same pattern, own interface). Then PARLIO behind it, and the `peripheral` select on the card. This REVISES step 1 rather than only adding to it. 5. **Board catalog entries**: Adafruit S3 Portal, Waveshare RGB Matrix, Lilygo S3, ESP32-S3-16MB-PSRAM. Catalog data in `deviceModels.json`, not driver work, and it is what makes the feature findable. -6. **Docs**: a how-to beside [panel-cards.md](../../reference/hardware/panel-cards.md), the driver's catalog card, and a line in panel-cards.md saying which of the two paths a given wall wants. +6. **Docs**: a how-to beside [panel-cards.md](../../../reference/hardware/panel-cards.md), the driver's catalog card, and a line in panel-cards.md saying which of the two paths a given wall wants. Steps 1-3 are the shippable unit. Steps 4-6 each stand alone. @@ -178,7 +178,7 @@ The PO's MoonHub75 (a LILYGO T7-S3 on the passive adapter) drove a 64×64 1/32-s ## Risks - **Refresh at depth may disappoint on 16 panels.** The encoder cost is linear in bit planes and the scan is serial. **Settled by publishing the prediction, not by measuring first**: § Refresh computes the ceiling per geometry and depth, the driver reports what it actually achieves, and the docs name PanelCard as the path above whatever that turns out to be. A user meets the limit with an explanation rather than a surprise. -- **The [48×256 white-flash](../future/backlog-light.md) is open on the i80 path.** Six theories ruled out, cause unfound. A HUB75 driver rides the same peripheral, so it may inherit it. **Settled by shipping and watching**: Phase 1 targets 1-4 panels, a size at which the flash has never been seen, so it does not block. If a tester on a large HUB75 wall reports it, that is the seventh theory the backlog entry is missing — the flash following a non-WS2812 encoder through the same peripheral would exonerate the WS2812 slot encoding, which six hardware tests could not do. +- **The [48×256 white-flash](../../future/backlog-light.md) is open on the i80 path.** Six theories ruled out, cause unfound. A HUB75 driver rides the same peripheral, so it may inherit it. **Settled by shipping and watching**: Phase 1 targets 1-4 panels, a size at which the flash has never been seen, so it does not block. If a tester on a large HUB75 wall reports it, that is the seventh theory the backlog entry is missing — the flash following a non-WS2812 encoder through the same peripheral would exonerate the WS2812 slot encoding, which six hardware tests could not do. - **Pin count is the tightest constraint on the classic ESP32 and shapes the target list.** See § GPIO requirements: the classic cannot host a full port without a strap pin, so it is excluded rather than half-supported. ## What this plan does not do diff --git a/docs/work/present/Plan-20261003 - Ethernet, WiFi and access point as Network submodules (shipped, partly verified).md b/docs/work/past/plans/Plan-20261003 - Ethernet, WiFi and access point as Network submodules (shipped).md similarity index 98% rename from docs/work/present/Plan-20261003 - Ethernet, WiFi and access point as Network submodules (shipped, partly verified).md rename to docs/work/past/plans/Plan-20261003 - Ethernet, WiFi and access point as Network submodules (shipped).md index 9811b275c..7943462fd 100644 --- a/docs/work/present/Plan-20261003 - Ethernet, WiFi and access point as Network submodules (shipped, partly verified).md +++ b/docs/work/past/plans/Plan-20261003 - Ethernet, WiFi and access point as Network submodules (shipped).md @@ -150,7 +150,7 @@ Waiting for more devices and a cable: ## Out of scope at first -The advanced settings users will ask for sooner or later are listed in the backlog's [advanced network settings](../future/backlog-core.md) item: roaming, enterprise WiFi, pinning to one access point, power save, the country code, the access point's address, and IPv6 beyond link-local. +The advanced settings users will ask for sooner or later are listed in the backlog's [advanced network settings](../../future/backlog-core.md) item: roaming, enterprise WiFi, pinning to one access point, power save, the country code, the access point's address, and IPv6 beyond link-local. ## Decided diff --git a/docs/work/present/Plan-20261005 - The audio codec as runtime configuration (in PR 129, M3 unverified).md b/docs/work/past/plans/Plan-20261005 - The audio codec as runtime configuration (shipped).md similarity index 100% rename from docs/work/present/Plan-20261005 - The audio codec as runtime configuration (in PR 129, M3 unverified).md rename to docs/work/past/plans/Plan-20261005 - The audio codec as runtime configuration (shipped).md diff --git a/docs/work/present/Plan-20261009 - Safe mode and factory reset.md b/docs/work/past/plans/Plan-20261009 - Safe mode and factory reset (shipped).md similarity index 100% rename from docs/work/present/Plan-20261009 - Safe mode and factory reset.md rename to docs/work/past/plans/Plan-20261009 - Safe mode and factory reset (shipped).md diff --git a/docs/work/present/Plan-20261011 - MoonLive Coord3D and RGB types.md b/docs/work/past/plans/Plan-20261011 - MoonLive Coord3D and RGB types (shipped).md similarity index 89% rename from docs/work/present/Plan-20261011 - MoonLive Coord3D and RGB types.md rename to docs/work/past/plans/Plan-20261011 - MoonLive Coord3D and RGB types (shipped).md index 4553a6d5e..31dd9f3f3 100644 --- a/docs/work/present/Plan-20261011 - MoonLive Coord3D and RGB types.md +++ b/docs/work/past/plans/Plan-20261011 - MoonLive Coord3D and RGB types (shipped).md @@ -1,10 +1,10 @@ # Plan: MoonLive `Coord3D` and `RGB` types -Most of what a script handles is a position or a color, and today both are loose integers: a coordinate is three values or an index the script computes by hand, and a color is three bytes that cannot travel together. This plan adds the two predefined types the [language roadmap](../future/backlog-moonlive.md) names as item 4b, so a script reads as its compiled counterpart does: `addRGB(i, getRGB(j))`. It lands now because every script written before it is a script to revisit after it. +Most of what a script handles is a position or a color, and today both are loose integers: a coordinate is three values or an index the script computes by hand, and a color is three bytes that cannot travel together. This plan adds the two predefined types the [language roadmap](../../future/backlog-moonlive.md) names as item 4b, so a script reads as its compiled counterpart does: `addRGB(i, getRGB(j))`. It lands now because every script written before it is a script to revisit after it. ## The rule it follows -The vocabulary follows the compiled modules ([MoonLive architecture](../../explanation/architecture/moonlive.md)): `Coord3D` and `RGB` are the C++ types of the same names, and every builtin taking or returning one is a one-line call into the `draw` function that does the same for a compiled effect. +The vocabulary follows the compiled modules ([MoonLive architecture](../../../explanation/architecture/moonlive.md)): `Coord3D` and `RGB` are the C++ types of the same names, and every builtin taking or returning one is a one-line call into the `draw` function that does the same for a compiled effect. ## The design: a struct is a named fixed-size array diff --git a/docs/work/present/Plan-20261013 - State by subscription, streamed.md b/docs/work/past/plans/Plan-20261013 - State by subscription, streamed (shipped).md similarity index 100% rename from docs/work/present/Plan-20261013 - State by subscription, streamed.md rename to docs/work/past/plans/Plan-20261013 - State by subscription, streamed (shipped).md diff --git a/docs/work/present/Plan-20261014 - Help in the UI, from the docs.md b/docs/work/past/plans/Plan-20261014 - Help in the UI, from the docs (shipped).md similarity index 92% rename from docs/work/present/Plan-20261014 - Help in the UI, from the docs.md rename to docs/work/past/plans/Plan-20261014 - Help in the UI, from the docs (shipped).md index b7a5d9a59..7a19327d7 100644 --- a/docs/work/present/Plan-20261014 - Help in the UI, from the docs.md +++ b/docs/work/past/plans/Plan-20261014 - Help in the UI, from the docs (shipped).md @@ -114,7 +114,7 @@ The device serves the UI; the **browser** fetches the help whenever it has inter - **Without internet**, as on a device's own access point, it uses the cache; with no cache either, the cards show as they do today. - **Help per module** opens from the card's title in its levels: the line, then the summary, the picture and the controls, then the details, then the links to the source and the tests. - **Help per control** is Material's helper text, one persistent line under the field, never a hover-only tooltip, which Nielsen Norman Group's guidelines warn hides what people need and fails on touch. The pilot compared it with Pickering's ⓘ toggletip and kept the line, since the status bar's ⓘ already shows or hides all help at once. A control with more opens it from its line, behind the module help's arrow. -- **Off until asked**: the ⓘ in the status bar turns help on, and the first time it asks before downloading, since the request shows GitHub, which hosts the docs site, the browser's address. Privacy by default, GDPR article 25's term; nothing is fetched before the yes. The [privacy policy](../../legal/privacy-policy.md) says so. +- **Off until asked**: the ⓘ in the status bar turns help on, and the first time it asks before downloading, since the request shows GitHub, which hosts the docs site, the browser's address. Privacy by default, GDPR article 25's term; nothing is fetched before the yes. The [privacy policy](../../../legal/privacy-policy.md) says so. ## Steps @@ -125,13 +125,13 @@ The change is gradual, the textbook strangler fig: the new path grows module by 3. ✅ **Layouts, the second catalog.** All seventeen layouts carry their help in their headers, and `layouts.md` is generated from `help.json`: its intro and clip from `LayoutBase` through `@video`, its cards from the headers, and its details from `MoonLiveLayout`'s `@guide`. The layouts credit what they draw on, Boustrophedon for serpentine wiring, polar coordinates for the rings and a conical spiral, through the attribution catalog. A layout is built from others as objects again, its public settings set and placed through `LightEmitter::place`, the relay the Layouts container uses for its children: CarLights and Rings241 from rings, which gain a center, Tubes from columns, Panels from panels and GridBlacks from a grid. Done in 436f37e9. 4. ✅ **Layouts composed in the tree.** A layout built from other layouts in code, as CarLights is from rings, has a runtime twin: the Layouts container placing each child at a position, so a user builds a car's lights from Ring children without code. To analyze: whether the position is the child's own control, such as a ring's center, or the container's per child, and how the bounding box and the light numbering follow. Done in 436f37e9: the position is the child's own control, Ring's `centerX` and `centerY`, so a user places rings side by side under the Layouts container. 5. ✅ **Effects, the third catalog.** All seventy effects carry their help in their headers, and `effects.md` is generated from `help.json`: its intro from `EffectBase`, its cards from the headers, and its details from each effect's `@guide`. Each header gets the step 13 sweeps as it moves, as the layouts did: `@attribution` for its `Author:` and `Prior art:` lines, no predecessor links, one comment kind per declaration, a line per control and a trimmed `@technical`. Done in f2e1b2ee. -6. ✅ **Drivers, the fourth catalog.** All eleven drivers carry their help in their headers, and `drivers.md` is generated from `help.json`: its intro and its two clips from `DriverBase`, its cards from the headers, and the shared controls read through `DriverBase`'s `defineControls`, as the virtual call resolves it. The card's details sections move by who reads them. In the app the user is at the card with a setting in hand, so the help is a glance: what the driver is, what to pick, the one fix when it does not work. On the website the user is sorting something out, which is a how-to page: [choosing and wiring an LED driver](../../reference/hardware/led-drivers.md), [driving a HUB75 panel](../../reference/hardware/hub75-panels.md), [watching the output as video](../../reference/watching-the-output.md) and the panel-card tutorial. So a control's longer help goes to a `/// @guide` line on its member, the choice to a `@guide` section of the module, which never explains a control, tables, steps and reasons to the how-to page the guide links, the mechanism to `@technical`, credit to `@attribution`, and what the code or another page holds is cut. `check_docgen` caps the app help on every generated catalog: 40 words for a control's longer help, 60 for a guide section, 250 for the whole. Done in ef20e444. +6. ✅ **Drivers, the fourth catalog.** All eleven drivers carry their help in their headers, and `drivers.md` is generated from `help.json`: its intro and its two clips from `DriverBase`, its cards from the headers, and the shared controls read through `DriverBase`'s `defineControls`, as the virtual call resolves it. The card's details sections move by who reads them. In the app the user is at the card with a setting in hand, so the help is a glance: what the driver is, what to pick, the one fix when it does not work. On the website the user is sorting something out, which is a how-to page: [choosing and wiring an LED driver](../../../reference/hardware/led-drivers.md), [driving a HUB75 panel](../../../reference/hardware/hub75-panels.md), [watching the output as video](../../../reference/watching-the-output.md) and the panel-card tutorial. So a control's longer help goes to a `/// @guide` line on its member, the choice to a `@guide` section of the module, which never explains a control, tables, steps and reasons to the how-to page the guide links, the mechanism to `@technical`, credit to `@attribution`, and what the code or another page holds is cut. `check_docgen` caps the app help on every generated catalog: 40 words for a control's longer help, 60 for a guide section, 250 for the whole. Done in ef20e444. 7. ✅ **System, Services and Supporting, the last catalogs.** Their modules carry their help in their headers by step 6's rules, and the three pages are generated: Services from its container's class comment, which is then no card, and System and Supporting from their own markdown kept as the intro the deck follows. Supporting's classes that are no module become one linked list. Control's details move into the control-surface, presets and home-automation how-tos, and Audio's into a new audio-input how-to. The separate deck pages go, every catalog page being a deck. Done in 18d069d1. 8. ✅ **The header is the technical reference.** Doxygen, npx moxygen and `gen_api` go, with their installs in the docs job and the 262 pages they generate. The header is what a developer reads, in the repo or on GitHub, so a second rendering of it is a copy to keep in step. Every link into `moxygen/` becomes a link to the header, which the docs build already rewrites to its file on GitHub, and the help's Technical link goes, Source being the same file. `check_docgen` follows: an `@xref` still names a heading in its own file, read as text, and the rules that measured the generated pages (a page nothing links to, an appendix section no page links) go or read the header alone. The supporting lists link headers the same way. Done. 9. ✅ **The migration**, module by module, each card deleted as its header takes over, so the site and the app stayed complete throughout. All 145 modules take their help from their headers, and a card's details section moved as step 6 sets out. The card bridge in `gen_help` is gone, `help.json` coming out byte for byte the same without it. Done. -10. ✅ **The checks.** `check_docgen` checks the help in headers. Done: a stale `help.json` fails it, drawn cards meet the card limits, a control line may not state its default or range, every module and every control has its help line, the app help stays within its caps, a control's guide does not repeat its line, the module guide explains no control, a function carrying a macro such as `MM_NONBLOCKING` has its `///`, and every tag emoji groups at least two modules or factory scripts; `check_specs` counts a module documented when the help reads its header. Reading declarations from the clang AST rather than by pattern is in [the core backlog](../future/backlog-core.md). +10. ✅ **The checks.** `check_docgen` checks the help in headers. Done: a stale `help.json` fails it, drawn cards meet the card limits, a control line may not state its default or range, every module and every control has its help line, the app help stays within its caps, a control's guide does not repeat its line, the module guide explains no control, a function carrying a macro such as `MM_NONBLOCKING` has its `///`, and every tag emoji groups at least two modules or factory scripts; `check_specs` counts a module documented when the help reads its header. Reading declarations from the clang AST rather than by pattern is in [the core backlog](../../future/backlog-core.md). 11. ✅ **The end state.** The `?` button is gone, the help's Docs link taking its place. A catalog page writes each module under its deck as plain markdown from `help.json`, its name bold rather than a heading so the table of contents lists no card, which search indexes and a reader without JavaScript reads. `split_blocks`, `fill_cards_from_help`, the table renderer, `update_module_docs.py` and the table styles are gone. `check_docgen`'s card rules read `help.json`: the picture and its format, a control line's width, and a line stating its default or range. `check_specs` counts a module documented when the help reads its header, so its page lookup and drift guards are gone. -12. ✅ **The attribution catalog as a deck.** [The attribution catalog](../../reference/attributions.md) is a card deck drawn from its own table. The table stays the one home, which search indexes and a reader without JavaScript sees, and the deck script turns its rendered rows into cards. A card shows the entry's picture, name and what it is known for, and its back the modules that name it. Its chips are a Kind column: 🧑 a person, 🏢 a company or shop, 🧰 a project or library, 📜 a standard or protocol, 🧮 an algorithm, 🎨 an art form or culture. 🧰 replaces the planned 📦, which the app's chips use for a compiled module. One card builder and one deck serve both kinds of card. +12. ✅ **The attribution catalog as a deck.** [The attribution catalog](../../../reference/attributions.md) is a card deck drawn from its own table. The table stays the one home, which search indexes and a reader without JavaScript sees, and the deck script turns its rendered rows into cards. A card shows the entry's picture, name and what it is known for, and its back the modules that name it. Its chips are a Kind column: 🧑 a person, 🏢 a company or shop, 🧰 a project or library, 📜 a standard or protocol, 🧮 an algorithm, 🎨 an art form or culture. 🧰 replaces the planned 📦, which the app's chips use for a compiled module. One card builder and one deck serve both kinds of card. 13. 🚧 **The sweeps**, the last commits before this branch merges: each a mechanical commit of its own, apart from behavioral changes, and under 100 files so a review still reads it whole. In order of size, counted on 2026-10-15: | # | Sweep | Status or size | @@ -144,5 +144,5 @@ The change is gradual, the textbook strangler fig: the new path grows module by | 5 ✅ | One comment kind per declaration, and its rule | 561 notes in 92 headers, class and file scope; `check_docgen` reports a `//` on a declaration | | 6 ✅ | `@defgroup` goes | 137 files and three generators; a file's lead is its first `///` block before any code | | 7 ✅ | `@moreinfo` becomes `@technical` | 273 files: the marker, the generators, `check_docgen` and `check_clang_query`, the standards and the tests; the past plans keep the old name as history | - | 8 → | `@technical` trimmed, Guides that add | moved to [the processes plan](Plan-20261015%20-%20Processes%2C%20described%20once.md), step 12 | + | 8 → | `@technical` trimmed, Guides that add | moved to [the processes plan](../../present/Plan-20261015%20-%20Processes%2C%20described%20once%20(partial).md), step 12 | 14. ✅ **Publishing.** Each release attaches its `help.json` as `MoonLight-help.json`, which the Pages deploy mirrors beside the release's binaries, and the app reads the copy of the release its device runs, falling back to the site's `help/help.json`. Both answered after the merge's run, and the fallback to the branch's copy is gone. diff --git a/docs/work/present/Plan-20261020 - Test engine.md b/docs/work/past/plans/Plan-20261020 - Test engine (shipped).md similarity index 93% rename from docs/work/present/Plan-20261020 - Test engine.md rename to docs/work/past/plans/Plan-20261020 - Test engine (shipped).md index 172a64fd4..f4d30c52a 100644 --- a/docs/work/present/Plan-20261020 - Test engine.md +++ b/docs/work/past/plans/Plan-20261020 - Test engine (shipped).md @@ -2,7 +2,7 @@ Plan, for the product owner's approval (2026-10-20). Automated tests are one of MoonLight's core strengths, and they run as several separate machines: C++ unit tests, scenarios on the desktop and on a live device, JS and Python tests, and the Python checks (docgen, prose, code, specs). Each has its own runner, its own output and its own report. This plan finds one model above them, so that what exists, what ran and what it found are described once and read by everyone it concerns: the help, MoonWalk, MoonDeck, the docs and CI. -It starts from [Processes, described once](Plan-20261015%20-%20Processes%2C%20described%20once.md), step 17: the map should show which tests cover a step, and the help which tests cover a module and a control. That needs a machine-readable index of the tests, which does not exist. The index is this plan's first result, and the rest follows from it. +It starts from [Processes, described once](../../present/Plan-20261015%20-%20Processes%2C%20described%20once%20(partial).md), step 17: the map should show which tests cover a step, and the help which tests cover a module and a control. That needs a machine-readable index of the tests, which does not exist. The index is this plan's first result, and the rest follows from it. ## Why @@ -52,7 +52,7 @@ What the inventory settled: ## The model (step 2, approved 2026-10-20) -[The schema](../../../moondeck/docs/tests.schema.json) holds the records. Three, all JSON; JUnit and SARIF are what runners and GitHub exchange, and the working model is ours: +[The schema](../../../../moondeck/docs/tests.schema.json) holds the records. Three, all JSON; JUnit and SARIF are what runners and GitHub exchange, and the working model is ours: 1. **The registry, `tests.json`**: every test by a derived id (`unit:test/unit/core/unit_X.cpp::the digest matches`, `scenario:`, `scenario:#`, `python:`, `js:`, `check:`), with its kind, size, title, file, the **subjects** it is about, the **controls** it sets or expects, the **steps** it proves, and the call that runs it. Generated like `help.json`. 2. **A run**, under `build/`, never committed: the test id, the git tree id it saw, a status, seconds, and **measurements** (target, metric, value, window). A runner that writes JUnit is read into it. @@ -84,8 +84,8 @@ No runner is a prerequisite. doctest, the scenario runner, pytest and each check Each step ends in something that works and can be seen, and nothing waits for a later step to be useful. 1. ✅ **Inventory**, done 2026-10-20: above. -2. ✅ **The model**, approved 2026-10-20 with the derived id: this section and [the schema](../../../moondeck/docs/tests.schema.json). -3. ✅ **`tests.json` and a drawn page on the website**, done 2026-10-20. `moondeck/docs/gen_tests.py` writes the sharded registry (2,607 tests, 195 subjects, an index of 13 KB and a largest shard of 235 KB, 44 KB gzipped), the docs build publishes it as `tests/index.json` and a shard per subject, and `docs/assets/tests.js` draws [the test page](../../reference/tests.md): subjects grouped by the help page of the module (Light · Effects, Core · Services) or by the domain of the tests for internals, with counts, a search, kind filters, each test with its file and line and the command that runs it, and a scenario's steps with the contract and observed range of each measured step, with every test's name in the page as plain text for search. `unit-tests.md` and `scenario-tests.md` are gone, redirected, and the help's Tests links point at the new page. The build found and the fix removed eleven unit tests duplicated in `unit_MoonLiveLayout.cpp`. MoonDeck still draws its own scenario views from `_test_metadata.py` and the perf table in `generate_test_docs.py`, which step 6 replaces with the registry. +2. ✅ **The model**, approved 2026-10-20 with the derived id: this section and [the schema](../../../../moondeck/docs/tests.schema.json). +3. ✅ **`tests.json` and a drawn page on the website**, done 2026-10-20. `moondeck/docs/gen_tests.py` writes the sharded registry (2,607 tests, 195 subjects, an index of 13 KB and a largest shard of 235 KB, 44 KB gzipped), the docs build publishes it as `tests/index.json` and a shard per subject, and `docs/assets/tests.js` draws [the test page](../../../reference/tests.md): subjects grouped by the help page of the module (Light · Effects, Core · Services) or by the domain of the tests for internals, with counts, a search, kind filters, each test with its file and line and the command that runs it, and a scenario's steps with the contract and observed range of each measured step, with every test's name in the page as plain text for search. `unit-tests.md` and `scenario-tests.md` are gone, redirected, and the help's Tests links point at the new page. The build found and the fix removed eleven unit tests duplicated in `unit_MoonLiveLayout.cpp`. MoonDeck still draws its own scenario views from `_test_metadata.py` and the perf table in `generate_test_docs.py`, which step 6 replaces with the registry. 4. ✅ **Help and MoonWalk read it**, done 2026-10-20. A control's help lists the scenarios that set or expect it (46 controls, each link opening that test on the page, proved in the device app as well as on the site's decks), scenarios declare their instances' types once (`instances`, and the generator refuses a step naming an undeclared one), and a step shows "Tested by" for the tests that name it (`steps_proved` in a scenario, `// @step` above a unit test) and for the scenarios that share its control. The build refuses a test naming a step no process has. The unit tests ran 2,585 cases with none failing and the scenario sweep 17 passed with none failing, after the new keys went in. *Result: the links in the help and the map, on the site and in the device's app.* 5. ✅ **The gaps**, done 2026-10-20: the page's Gaps view lists the 25 device-changing steps no test covers, the 18 help modules no test is about, and the 9 controls a scenario sets that the help does not know. *Result: the list of what to test next.* 6. ✅ **Runs**, done 2026-10-20. MoonDeck's runners keep each test's last result as a run, in the model's shape (the derived id, a status, seconds, the git tree it saw), under `build/moondeck-logs/`, never committed (`moondeck/docs/test_runs.py`). The page shows it beside the test, "passed in 0 ms, on this tree" or "on an older tree", where the file exists; the public site shows none until CI records its own (step 10). Recorded from: doctest's JUnit (followed live, so MoonDeck's log still streams; subcase runs count for their case, and a shared `.inc` body is one test per target), pytest's and node's JUnit through their flags, each scenario's `PASS`, `FAIL` or `SKIP` from `run_scenario.py`, and MoonDeck's record of each check script. The registry now holds every kind, 3,526 tests: unit, scenario, live, Python, JS and the 15 checks as `check: