diff --git a/.cursor/rules/sql-editor-agent-memory.mdc b/.cursor/rules/sql-editor-agent-memory.mdc index 82b8b4d7d..af61e6208 100644 --- a/.cursor/rules/sql-editor-agent-memory.mdc +++ b/.cursor/rules/sql-editor-agent-memory.mdc @@ -27,7 +27,7 @@ FoxSchema = schema diff/migration + SQL Editor + workspaces. Destinations checkl - Admins see all workspaces but open one only by joining (#495 / Admin → Workspaces). - Public vs private: signed-in users can join public (#496). - Isolation proven e2e both ways (#497). -- UI: `workspace-switcher`, `workspace-menu`, `workspace-settings`, profile embeds WorkspaceMenu. +- UI: rail `WorkspaceRailButton` (under the Fox mark) opens `WorkspaceMenu` / settings; launch sessions hide it. Profile menu does not list workspaces. ## Auth / admin (Claude + follow-ons) - **Launch sessions** (#490): `foxschema open` one-time local launch token → session with `launch: true`; skip signup wizard; profile **"Create your account"**; registration grace then blocks; admin/SSO gated until registered. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index b9c130556..a51e7ee67 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -155,6 +155,13 @@ connection grant follows the connection’s workspace. (`GET /api/workspaces/discover`, `POST /:id/join`). Join role is viewer or editor, never owner. An invite never adds: the invitee accepts (`POST /api/workspaces/invites/:id/accept`, 14-day TTL). +- **UI.** One workspace is open. `WorkspaceRailButton` sits under the Fox mark + on the activity rail (not the profile menu) and opens the same + `WorkspaceMenu` (switch, invites, settings, public join, create). A launch + session hides it. Switching is still `POST /api/workspaces/:id/select` then + reload. Editor tabs/scripts in this browser are keyed by workspace id + (`editorWorkspaceStorage.ts`); the unscoped blob is claimed once by the first + workspace the rail sees. Startup `backfillWorkspaces` gives each account a personal workspace and moves rows that still have `workspace_id` null or `'local'` into their author’s. @@ -285,8 +292,10 @@ largest files when it fails. Everything else loads when it is used: `prefetchView` on hover or focus, so a view's code is usually already loaded by the time the click lands. - **Panels that open on a click** (admin console, credentials, applies history, new - connection) are `lazy()` inside `MountWhenOpened`. That mounts a panel on its first open - and keeps it mounted afterwards, so its state survives closing exactly as before. + connection, the workspace menu) are `lazy()` inside `MountWhenOpened`. That mounts a + panel on its first open and keeps it mounted afterwards, so its state survives + closing exactly as before. The workspace button is shell, not a view — it is + not in the view registry. - **Heavy libraries** (sql-formatter, Prettier, faker, lodash, date-fns, zod, ajv) load through `shared/lib/loadOnce.ts`: one shared promise, retried after a failure. A component that needs one while rendering uses `useLoaded(loader)`, which re-renders diff --git a/docs/CODE_MAP.md b/docs/CODE_MAP.md index 795666e47..b627faab3 100644 --- a/docs/CODE_MAP.md +++ b/docs/CODE_MAP.md @@ -89,7 +89,7 @@ modules/ One folder per domain, named to match the frontend feature | `lokee-weave` | Content-addressed schema versioning and revert | | `sql-editor` | FoxScript parsing, code cells, SELECT aliasing, the SQL subset | | `access` | Permission intent, effective access, GRANT/REVOKE and account DDL (facades; emitters live in `providers/`) | -| `utilities` | DBA queries: server insights, index fragmentation, backup/restore command builders | +| `utilities` | DBA queries: server insights, index fragmentation (`isSafeIndexFragmentationCustomSql`: one read-only SELECT; Db2 defrag is `providers/db2/db2.index-fragmentation.ts`), backup/restore command builders | `dialect` and `sql-text` are the foundations: the other folders build on them, never the reverse. @@ -140,7 +140,7 @@ database/ The metadata store and its migrations. | Folder | What it covers | |---|---| -| `access` | Database permission inspection and DBA utilities | +| `access` | Database permission inspection and DBA utilities. Index-fragmentation custom SQL is gated read-only in `index-fragmentation.service.ts` (same tokenizer as `/sql/execute`). | | `system` | Health and version, updates, app info, the metadata-database probe, activity | | `admin` | Users and roles, secrets, cloud credentials, policy | | `backup` | Per-user backup defaults (`GET`/`PUT /api/backup-settings`). Commands themselves are built in `@foxschema/sql` (`modules/utilities/backup.ts` + `providers//*.backup.ts`); the panel runs only a server-side SQL backup, after confirmation, and lists the backups the server recorded (`backupHistoryQuery`); restores are never run. | @@ -153,10 +153,10 @@ database/ The metadata store and its migrations. | `history` | Schema history and revert (Lokee Weave) | | `migration` | Applying DDL migrations, and their run history | | `schema` | Reading a schema | -| `sql-editor` | SQL editor, code cells, sandboxed execution | +| `sql-editor` | SQL editor, code cells, sandboxed execution. `sql-execute.service.ts` (`runStatements`) applies the saved schema (`adapter.setCurrentSchema`) then optionally `BEGIN`s one transaction for the batch; paging wrap is `sql-page-wrap.service.ts`. | | `users` | Profile, preferences, first-run wizard | | `workspaces` | Shared workspaces: create, switch, members, invites, public or private (`/api/workspaces`). Which workspace a request acts in is `platform/workspaces`. Admins list every workspace (`GET /all`) and open one only by joining. | -| `workflow` | Workflow engine settings and health; the engine proxy (an allowlist of engine routes, each behind a `workflow.*` permission); saved-connection grants; and the token-guarded internal routes the engine calls to resolve a granted connection and read its settings. Runbook: [workflow.md](workflow.md). | +| `workflow` | Workflow engine settings and health; the engine proxy (an allowlist of engine routes, each behind a `workflow.*` permission); saved-connection grants; and the token-guarded internal routes the engine calls to resolve a granted connection and read its settings. Save, run, and steering (cancel / resume / answer a paused run) refuse a connection the caller cannot use in this workspace (`WorkflowConnectionGrants.notUsableBy`). Runbook: [workflow.md](workflow.md). | Every feature folder has an `index.ts` exporting its `ServerFeatureModule` and its public API; other features import only that @@ -245,10 +245,10 @@ the page-epoch guard, bookmarks and recents, SQL variables. | `lokee-weave` | Schema history graph and version compare (Snapshots workspace) | | `migrations` | Migration run history (Applies on the rail) | | `schema-diff` | Diff rendering shared by compare and history | -| `sql-editor` | SQL editor, its store, results grid, data peek, the shared Monaco setup (`monaco.ts`) (Index/Clone/Query files live in `utilities`) | +| `sql-editor` | SQL editor, its store (`oneTransaction`, `selectOnlyConnection` so the schema browser's connection is the one Run hits), results grid, data peek, the shared Monaco setup (`monaco.ts`) (Index/Clone/Query files live in `utilities`) | | `utilities` | Own workspace: clone table, index management, backup & restore commands, server insights, query files, DB users & grants | -| `workflow` | Workflow designer (canvas, inspector, triggers, SQL and script editors), runs, variables, credentials and engine settings — through the engine proxy, plus linking saved connections to workflows | -| `workspaces` | Switcher and settings in the profile menu: create, invite, join public, members. Admin → Workspaces lists every workspace; opening one is joining. | +| `workflow` | Workflow designer (canvas, inspector, triggers, SQL and script editors), runs (`RunsPanel` cancel / answer a paused human gate), variables, credentials and engine settings — through the engine proxy, plus linking saved connections to workflows | +| `workspaces` | The rail's workspace button (`WorkspaceRailButton`) and the menu it opens: switch, invite, join public, members, settings. Admin → Workspaces lists every workspace; opening one is joining. Editor localStorage is keyed by workspace (`shared/lib/editorWorkspaceStorage.ts`). | ## Where does my change go? diff --git a/docs/architecture/FEATURE-MODULE-GUIDE.md b/docs/architecture/FEATURE-MODULE-GUIDE.md index 7dbe318ec..9fb782b0c 100644 --- a/docs/architecture/FEATURE-MODULE-GUIDE.md +++ b/docs/architecture/FEATURE-MODULE-GUIDE.md @@ -113,7 +113,9 @@ checks access itself; its button follows its nav item's permission (`navId`). The rail, the lazy view, prefetch-on-hover, the "you lost access, go home" redirect and the view's part of the top toolbar (`toolbar: { start, end, below }`) -all read this entry. Adding a view touches no shell file. +all read this entry. Adding a view touches no shell file. The workspace button +under the Fox mark is shell (`WorkspaceRailButton`), not a view — do not add it +to the view registry. Keep components local until a second feature needs them; then move the product-neutral part to `shared/`, not to another feature. diff --git a/docs/releases/UNRELEASED.md b/docs/releases/UNRELEASED.md index dc7e49f25..7364be37c 100644 --- a/docs/releases/UNRELEASED.md +++ b/docs/releases/UNRELEASED.md @@ -3,6 +3,34 @@ Notes for the next release. At ship time, rename this file to that version's `RELEASE_.md` and use it as the GitHub Release body. +## One workspace on the rail + +The workspace you are in sits under the Fox mark on the left rail — switch, +invites (amber dot), settings, public join, and create. It is no longer inside +the profile menu. A launch session hides the button. SQL editor tabs in this +browser follow the workspace. + +## SQL Editor: the schema you are browsing, and one transaction + +Run applies the saved connection's schema (`CURRENT SCHEMA` / `search_path`) +so unqualified names resolve to the tables the explorer shows, not the +database user's default. Picking a connection in the explorer checks that +destination alone. **One transaction** wraps a multi-statement SQL run per +destination; a failure rolls it back. Code cells and `-- @set` stay statement +by statement. + +## Workflow: steer a paused run; connections stay in their workspace + +**Runs** can cancel a live job and answer **Ask a person**. Saving or running +a workflow that names another workspace's connection is refused — the engine +is install-wide; the connection is not. + +## Index custom SQL is read-only; Db2 reorgs the table's indexes + +A custom fragmentation probe must be one `SELECT`. GRANT / REVOKE scripts stop +at the first failure. Db2 defrag reorganises every index on the table, quoting +catalog names. + ## Left-rail views - **Snapshots** is its own left-rail view (schema history / Lokee). **Applies** diff --git a/docs/user-guide.md b/docs/user-guide.md index e4c050d7e..d215fb698 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -97,9 +97,12 @@ optional — use **Skip for now** if you prefer. It only appears once per instal You land on **Home**: continue last compare / last query, **Snapshots**, **Utilities**, and saved connections (grouped by dialect). **⌘K** / **Ctrl+K** searches the same -rail workspaces the left rail shows, plus recents. The left rail is **Compare**, -**Editor**, **Utils**, **Access**, **Workflow**, **Snapshots**, then **Creds** and -**Applies** at the bottom. +views the left rail shows, plus recents. The left rail is the Fox mark (Home), +the **workspace** button (the one workspace this account is acting in), then +**Compare**, **Editor**, **Utils**, **Access**, **Workflow**, **Snapshots**, and +**Creds** and **Applies** at the bottom. A launch session (`foxschema open` +before you create an account) hides the workspace button — you are already in +the install owner's personal workspace. Saved connections, snapshots, migration runs and secrets belong to a **workspace** — every account has its own, and you can share others. See @@ -127,9 +130,14 @@ manage its members, whatever your role there. ### Switch, create, invite -1. Open the **profile menu** (avatar at the foot of the left rail). The - **Workspace** list is the workspaces you are in — click one to switch. -2. **Workspace settings** opens the current one: rename, members, invites, +One workspace is open at a time. Switching reloads the app, and the SQL +editor's saved tabs follow that workspace (they live in this browser, keyed by +workspace id). + +1. Click the **workspace** button under the Fox mark on the left rail. It + names the workspace you are in. An amber dot means you have invitations. +2. The menu is switch, invites, **Workspace settings**, browse public, and + **New workspace**. Settings opens the current one: rename, members, invites, public / private. 3. **New workspace** creates a shared one and makes you its owner. Admins always can; other accounts only when an admin turns on **Members can create @@ -137,11 +145,12 @@ manage its members, whatever your role there. has their own personal workspace. A new shared workspace starts private. To bring someone in: Workspace settings → invite by email with a role -(`viewer` / `editor` / `owner`). They see it under **Invitations** in their -profile menu and join only when they **Accept** (14 days to accept; **Decline** -drops it). Inviting an email with no account creates one as a viewer — only an -admin can, unless **Workspace owners can invite new people** is on. Only an -admin ever sees that new account’s one-time code; otherwise it is emailed. +(`viewer` / `editor` / `owner`). They see it under **Invitations** in the +workspace menu and join only when they **Accept** (14 days to accept; +**Decline** drops it). Inviting an email with no account creates one as a +viewer — only an admin can, unless **Workspace owners can invite new people** +is on. Only an admin ever sees that new account’s one-time code; otherwise it +is emailed. ### Public vs private @@ -295,6 +304,8 @@ compare / migrate). It lives in the same local web UI you open with `foxschema`. 2. On the left rail, click **Editor** (next to **Compare**). 3. Under **Destinations**, check one or more saved connections — the same SQL runs against every checked server (handy for comparing data across environments). + Picking a connection in the **Schema explorer** checks that one alone, so Run + hits the tables on screen; check more Destinations afterwards to fan out. 4. Type SQL in the editor. Multiple statements are fine; use the **statement strip** under the editor to enable/disable individual statements before Run. Or **select** a statement (or any SQL) in the editor — Run becomes **Run selection** and only @@ -355,13 +366,21 @@ Tips: `table.` and `alias.` still suggest columns. A name that is both a schema and a table keeps its **columns** (`orders.`). An unknown qualifier shows a short explanation instead of an empty popup. + Run applies that connection's saved schema the same way a migration does + (`SET CURRENT SCHEMA` / `search_path` / equivalent), so an unqualified + `ORDERS` resolves to the schema you saved — not the database user's default + (Db2 would otherwise look in the auth-id schema). **Edit table** shows each index’s fragmentation % for every dialect (physical or estimated probe; SQLite / DuckDB / ClickHouse / Redshift list indexes when no - native % exists). Paste custom SELECT if the default fails, and use the wrench - to insert rebuild/reorg/optimize/REINDEX SQL when useful. The index form opens - under the selected index (no jump to a form at the bottom of the section). - Index Management, Clone Table, and Query files live in **Utils** — see - [Utilities](#utilities). + native % exists). Paste custom SELECT if the default fails — it must be **one + read-only** `SELECT` (or `WITH … SELECT`); a write, `SELECT … INTO`, or a + second statement is refused. Use the wrench to insert rebuild/reorg/optimize/REINDEX + SQL when useful. On **Db2**, that SQL reorganises **every** index on the table + (`REORG INDEXES ALL FOR TABLE` through `SYSPROC.ADMIN_CMD`) and quotes catalog + names (folded uppercase unless the table is a mixed-case delimited identifier). + The index form opens under the selected index (no jump to a form at the bottom + of the section). Index Management, Clone Table, and Query files live in + **Utils** — see [Utilities](#utilities). - **Data peek** — two ways in: - **Schema:** hold **Cmd** (macOS) or **Ctrl** (Windows/Linux) and click a table, view or MQT to see its rows without writing a query. @@ -539,6 +558,12 @@ Tips: rows (`ON CONFLICT DO UPDATE`, `ON DUPLICATE KEY UPDATE`, `INSERT OR REPLACE`), and DDL need an extra confirmation before run. Plain INSERT (including insert CTEs and `ON CONFLICT DO NOTHING`) does not. +- **One transaction** — when on, every SQL statement in that Run is one + transaction **per destination**. A failure rolls the batch back and leaves + later statements unrun (the result row says so). Code cells and `-- @set` + stay statement by statement — the checkbox then warns instead of wrapping. + Engines that auto-commit DDL still report each statement; the rollback is + best-effort there. This is separate from Data migrate's Transaction assist. Writes and DDL are allowed when you confirm them. **SQLite** connections are read-write (the file is opened that way on purpose). **ClickHouse** grid row @@ -556,12 +581,17 @@ saved credential at the top, then a tool: - **Index Management** — indexes grouped by table; filter by name or minimum fragmentation %; fetch fragmentation in batch (`POST /schema/index-fragmentation-batch`); - defragment selected indexes or all filtered rows. + defragment selected indexes or all filtered rows. Custom probe SQL is the + same read-only rule as Edit table (one `SELECT`). **Db2** cannot reorg a + single index: defrag runs `REORG INDEXES ALL FOR TABLE` plus `RUNSTATS` + through `SYSPROC.ADMIN_CMD`, quoting names the way `SYSCAT` stores them. - **Clone Table** — archive a huge table as `name_1` / next free `name_N` (or a fixed starting number), then recreate an empty table with the original name and columns so apps keep working. Toggle **Keep indexes** and **Foreign keys (auto)** - for the new table; Insert SQL or Apply. Inbound FKs from other tables still point - at the archive until you update them. + for the new table; Insert SQL or Apply. Apply stops at the first failed + statement and is not one transaction — if it stops after rename, recover by + renaming the archive back. Inbound FKs from other tables still point at the + archive until you update them. - **Backup & Restore** — Fox Schema writes the backup and restore commands for this connection. The amber banner says *where* they run, because that is what the folder means: @@ -639,13 +669,21 @@ Developer / ops runbook: [workflow.md](workflow.md). Env vars: 4. **Designer**: drop a trigger (Manual, Schedule, Webhook, API Endpoint, …) and pipes. Picking a saved connection on a **SQL query** / **SQL write** pipe **links** it — Fox Schema records the grant; the pipe never holds a password. + A saved connection belongs to **this workspace**. Saving or running a + workflow that names another workspace's connection is refused (403) — the + engine is install-wide, so this check is what keeps A's password out of B. 5. **Run** from the designer, or let cron / webhook / poll fire. **Runs** lists - history. **Variables** and **Credentials** are per-engine; linked Fox Schema - connections show up as `foxschema-…` credentials. + history. A live run can be **cancelled**. A run paused on **Ask a person** + shows the question there: **Send answer** continues it, **Resume** without + an answer retries the gate. Steering reads the run's snapshot, so a run that + already used a foreign connection cannot be cancelled or answered from this + workspace either. **Variables** and **Credentials** are per-engine; linked + Fox Schema connections show up as `foxschema-…` credentials. Permissions (multi-user): **Open Workflow** to see the pane; **Design** to edit; -**Run** to start jobs; **Workflow admin** for the Engine tab. Viewers can open -Workflow and read runs without seeing Designer chrome. +**Run** to start, cancel, and answer a paused job; **Workflow admin** for the +Engine tab. Viewers can open Workflow and read runs without seeing Designer +chrome. ## Access control @@ -690,7 +728,10 @@ API (`POST /schema/db-access`): - **Utils → DB users & grants** (needs **Use utilities**). Either family may load the catalog. Running GRANT / REVOKE still needs -**Grant privileges** (`editor.grant`). SQLite / DuckDB have no GRANT catalog; +**Grant privileges** (`editor.grant`). The script runs statement by statement +and **stops at the first failure** — later steps assume earlier ones (USAGE ON +SCHEMA before ON ALL TABLES). Statements before the failure stay applied; +nothing is wrapped in a transaction. SQLite / DuckDB have no GRANT catalog; ClickHouse has no permission builder yet. The Access workspace opens on **Permissions** (a principal list). Pick a user @@ -861,8 +902,9 @@ the `/data` volume — make sure you didn't remove it (`docker compose down -v` volumes). See [DEPLOYMENT.md](DEPLOYMENT.md). **Saved connections disappeared after a switch.** They belong to the -**workspace** you were in. Open the profile menu → **Workspace** and switch -back; each workspace has its own connections, snapshots and runs. +**workspace** you were in. Click the workspace button under the Fox mark and +switch back; each workspace has its own connections, snapshots and runs. SQL +editor tabs follow that workspace too (they are stored in this browser). **Signed out while the UI was open.** A session unused for **8 hours** ends, whatever its 7-day expiry (`FOX_SESSION_IDLE_HOURS`; `0` turns the idle limit @@ -874,7 +916,8 @@ separate process. Check **Workflow → Engine** health. New installs default to **Disabled** — set **Enabled** and Save. `FOXFLOW_ENCRYPTION_KEY` is required at engine boot. Saved connections need both `WORKFLOW_ENGINE_TOKEN` (same value on FoxSchema and the engine) and an explicit grant (pick the connection on a SQL -pipe). The published Docker image does not start the engine. See +pipe). A 403 on save or run usually means the workflow names a connection from +another workspace. The published Docker image does not start the engine. See [workflow.md](workflow.md). Still stuck? Open a GitHub issue with what you did and the error you saw. diff --git a/docs/workflow.md b/docs/workflow.md index e10794f64..c89b1269f 100644 --- a/docs/workflow.md +++ b/docs/workflow.md @@ -142,6 +142,31 @@ The engine resolves that connection **as the owner** through `POST /api/workflow-internal/connections/resolve`. Revoke stops the next run. The pipe never stores a password. +The engine itself is install-wide (one SQLite file, every designer sees every +workflow). A saved connection is not: it belongs to one workspace. Save, run, +duplicate, read a run, and steering (cancel / resume / answer) all call +`WorkflowConnectionGrants.notUsableBy` on the connections named in the body, +the stored workflow, or the run snapshot (`ENGINE_ROUTES[].connections`). A +connection the caller is not a member of is **403** before the proxy forwards. +A grant is also ignored once the granter leaves that workspace +(`resolveForEngine`). + +## Steering a run + +`workflow.run` starts a job and also steers one that is already going. The +Runs panel (`RunsPanel.tsx`): + +| State | Control | Engine route | +|-------|---------|--------------| +| `queued` / `running` | **Cancel run** | `POST /runs/:id/cancel` | +| `paused` (Ask a person) | load the question, **Send answer** | `GET` then `POST /runs/:id/inputs` | +| `paused` | **Resume** without new values | `POST /runs/:id/resume` | + +Each of those first reads the run snapshot so the workspace check applies to +what the run actually used, not the live workflow (which may have been edited +since). A start is still refused unless engine state is `enabled`; cancel and +answer are not starts. + ## Engine settings Saved in FoxSchema (`workflow.engine_config`). The engine re-reads them every @@ -162,6 +187,10 @@ Saved in FoxSchema (`workflow.engine_config`). The engine re-reads them every connections. `FOXFLOW_ENCRYPTION_KEY` (32 bytes, hex or base64) protects the engine credential store and is **required at engine boot** — there is no plaintext fallback (`keyFromEnv()`). +- **A connection stays in its workspace.** Linking one in workspace A used to + let every designer on the install save and run with A's password. The proxy + now refuses that. Move the workflow (or re-pick a connection that belongs + here) instead of copying a `foxschema-…` credential id across workspaces. - **Docker image does not start the engine.** `Dockerfile` / `docker-compose.app.yml` run FoxSchema only. Deploy `apps/workflow-server` beside it and set the token on both. See [DEPLOYMENT.md](DEPLOYMENT.md#workflow-engine). diff --git a/packages/sql/src/providers/DIALECTS.md b/packages/sql/src/providers/DIALECTS.md index 0f568077d..d0dd3735f 100644 --- a/packages/sql/src/providers/DIALECTS.md +++ b/packages/sql/src/providers/DIALECTS.md @@ -117,6 +117,13 @@ Each dialect lives in `providers//.sql-dialect.ts`. can write to. `postColumnChangeStatements` emits `CALL SYSPROC.ADMIN_CMD('REORG TABLE …')` before the keys/indexes are rebuilt. Bare `REORG TABLE` is a CLP command, not SQL, and cannot be sent over a connection. +- **Index reorg (Utils / Edit table):** Db2 cannot reorganise a single index + (`REORG INDEX name` → SQL0270N reason 89). Defrag emits + `CALL SYSPROC.ADMIN_CMD('REORG INDEXES ALL FOR TABLE …')` then `RUNSTATS` + (`providers/db2/db2.index-fragmentation.ts`). Quote names as `SYSCAT` stores + them: fold the schema (and a single-case table) to uppercase before quoting; + keep mixed case on a delimited table name. Quoting the typed case + (`"demo_a"."ORDERS"`) is SQL2211N for a table the probe just listed. - `modifyColumnStatements` — `SET DATA TYPE` can't carry nullability; emit a **separate** `SET NOT NULL` / `DROP NOT NULL`. - `dropIndexStatement` / `dropTriggerStatement` schema-prefixed, no `ON table`.