Bind Pointer actions like Keyboard and Gamepad, and group devices in Controls

.changeset/consume-once.md

@@ -6,6 +6,6 @@
 **Draining a service is declared, checked at compile time, and observed at run time.**
 
 - **`System.Consumes<"snapshot">`** marks the methods of a service type that drain it. A system that declares the service with `Game.System.service` sees those methods as a compile error naming the fix (`snapshot() drains Game/Controls: declare it with Game.System.consumeService, or read the resource its capture writes`); `Game.System.consumeService` calls them. Its other methods stay available through `service` (a settings screen can still `rebind`).
-- Every input source is marked: `Keyboard.Actions`, `Keyboard.Presses`, `Gamepad.Actions`, `Touch.Touch`, `Pointer.Pointer`, `Controls.Device`, `Controls.Controls`, and `InputCapture.Source`. **Breaking:** a system that called `snapshot()` through `Game.System.service` no longer compiles; read the resource `InputCapture.system` writes instead.
+- Every input source is marked: `Keyboard.Actions`, `Keyboard.Presses`, `Gamepad.Actions`, `Touch.Touch`, `Pointer.Actions`, `Controls.Device`, `Controls.Controls`, and `InputCapture.Source`. **Breaking:** a system that called `snapshot()` through `Game.System.service` no longer compiles; read the resource `InputCapture.system` writes instead.
 - `Game.System.service` returns `ServiceRead<D, false>` and `consumeService` returns `ServiceRead<D, true>`.
 - The `service-consumed-twice` lint also reports what a debug runtime saw: a consumed service whose consumers ran at more than one placement in the session, in unnamed schedules too (`Game/Drain (schedule#2, step 0, unnamed)`). Named schedules are reported once, by the static lint.

.changeset/pointer-actions.md

@@ -0,0 +1,12 @@
+---
+"@typeonce/ritmo-browser": minor
+---
+
+**`Pointer` binds actions like `Keyboard` and `Gamepad` and merges into `Controls`, and `Controls.group` makes devices used together count as one.**
+
+- **Breaking: `Pointer.track(target, blurHost, options?)` is `Pointer.actions(target, blurHost, bindings, options?)`.** Bindings map actions to buttons (`{ fire: ["primary"] }`); a misspelled button or an empty list does not compile. Pass `{}` for position and raw buttons only. The `Pointer.Pointer` type is `Pointer.Actions<B>` and `TrackOptions` is `ActionsOptions`.
+- **Breaking: the snapshot is `{ actions, buttons, position, over, used }`.** Read `input.buttons.primary` (or a bound action) instead of `input.primary`. `used` is a press, a hold, or movement over the element; releases do not count.
+- **The `back` and `forward` side buttons** (`PointerEvent.button` 3 and 4) are reported and bindable. `Pointer.buttonNames` and `Pointer.isButton` list and check them.
+- `rebind(...)` swaps the buttons of the same actions; `Pointer.parseRebinding(bindings, data)` validates saved ones.
+- **Breaking: `Pointer.idle(bindings, position?)` and `Pointer.scripted(bindings, timeline, start?)`** take the bindings first. Timelines still name buttons, so a replay follows the bindings as live play does.
+- **`actions.group({ keyboard, mouse })`** groups devices into one merge entry: it binds what its devices bind between them, and `lastUsed` and `available` follow it as a whole, so walking with the keyboard while aiming with the mouse stays on one device. Grouped devices keep their own names in `devices`, `rebind`, and `idle`; `lastUsed` names the group. Pass the group to `lacking(...)` for actions none of its devices binds. A group of fewer than two devices, a `lacking` device or a group inside a group, a device binding an undeclared action, and a name used twice across devices and groups do not compile.

integration/top-down/debug.ts

@@ -11,14 +11,14 @@
  * the timeline, the invariants, and the questions.
  */
 import { HostClock } from "@typeonce/ritmo"
-import { Gamepad, Keyboard } from "@typeonce/ritmo-browser"
+import { Gamepad, Keyboard, Pointer } from "@typeonce/ritmo-browser"
 import { Invariant, Session } from "@typeonce/ritmo-devtools"
 
 import { WORLD_HEIGHT, WORLD_WIDTH } from "./constants.ts"
 import { PlayerPositionQuery } from "./queries.ts"
 import { Game, Player, Position } from "./schema.ts"
 import { createTopDownSimulation } from "./simulation.ts"
-import { inputBindings, padBindings } from "./types.ts"
+import { inputBindings, mouseBindings, padBindings } from "./types.ts"
 
 // Frame 0 is the first update. Press and release are applied before that
 // frame's input capture; a press and release in one frame is a tap.
@@ -32,7 +32,8 @@ const keyboard = Keyboard.scripted(inputBindings, [
 const hostClock = HostClock.test()
 // No controller in this run; script one the same way, by button names.
 const gamepad = Gamepad.scripted(padBindings, [])
-const simulation = createTopDownSimulation({ keyboard, gamepad, hostClock })
+const mouse = Pointer.scripted(mouseBindings, [])
+const simulation = createTopDownSimulation({ keyboard, mouse, gamepad, hostClock })
 if (!simulation.ok) throw new Error(simulation.error.message)
 const { runtime, setup, update } = simulation.value
 

integration/top-down/runtime.ts

@@ -1,8 +1,8 @@
-import { Gamepad, Keyboard } from "@typeonce/ritmo-browser"
+import { Gamepad, Keyboard, Pointer } from "@typeonce/ritmo-browser"
 
 import { pickupLayout } from "./content.ts"
 import { Facing, Game, Locomotion } from "./schema.ts"
-import { Actions, inputBindings, padBindings } from "./types.ts"
+import { Actions, inputBindings, mouseBindings, padBindings } from "./types.ts"
 import type { FrameContextValue } from "./types.ts"
 
 export const makeEmptyFocusedCollectable = () => ({
@@ -13,7 +13,7 @@ export const makeEmptyFocusedCollectable = () => ({
 
 export const initialResources = (frame: FrameContextValue) => ({
   DeltaTime: frame.deltaSeconds,
-  InputState: Actions.idle({ keyboard: Keyboard.idle(inputBindings), gamepad: Gamepad.idle(padBindings) }),
+  InputState: Actions.idle({ keyboard: Keyboard.idle(inputBindings), mouse: Pointer.idle(mouseBindings), gamepad: Gamepad.idle(padBindings) }),
   FocusedCollectable: makeEmptyFocusedCollectable(),
   CollectedCount: 0,
   TotalCollectables: pickupLayout.length

integration/top-down/simulation.ts

@@ -1,14 +1,15 @@
 /**
  * The top-down game without a browser: the simulation schedules on a
- * runtime that needs only a keyboard, a gamepad, and a frame context, with
+ * runtime that needs only a keyboard, a mouse, a gamepad, and a frame context, with
  * the debug handle attached.
  *
  * Tests, benchmarks, and coding agents drive it directly:
  *
  * ```ts
  * const keyboard = Keyboard.scripted(inputBindings, [{ frame: 0, press: ["right"] }])
+ * const mouse = Pointer.scripted(mouseBindings, [])
  * const gamepad = Gamepad.scripted(padBindings, [])
- * const simulation = createTopDownSimulation({ keyboard, gamepad, hostClock: HostClock.test() })
+ * const simulation = createTopDownSimulation({ keyboard, mouse, gamepad, hostClock: HostClock.test() })
  * if (simulation.ok) {
  *   simulation.value.runtime.tick(simulation.value.setup)
  *   simulation.value.runtime.tick(simulation.value.update)
@@ -23,10 +24,12 @@ import { initialMachines, initialResources } from "./runtime.ts"
 import { ControlsInput, FrameContext, Game } from "./schema.ts"
 import { setupWorldSchedule, simulationSchedule } from "./schedules.ts"
 import { makeControls } from "./types.ts"
-import type { FrameContextValue, GamepadInput as GamepadInputValue, KeyboardInput as KeyboardInputValue } from "./types.ts"
+import type { FrameContextValue, GamepadInput as GamepadInputValue, KeyboardInput as KeyboardInputValue, MouseInput as MouseInputValue } from "./types.ts"
 
 export interface TopDownSimulationOptions {
   readonly keyboard: KeyboardInputValue
+  /** `Pointer.actions(canvas, window, mouseBindings)` in a browser, `Pointer.scripted(mouseBindings, timeline)` headless. */
+  readonly mouse: MouseInputValue
   /** `Gamepad.actions(window, padBindings)` in a browser, `Gamepad.scripted(padBindings, timeline)` headless. */
   readonly gamepad: GamepadInputValue
   /** Times the debug traces; `HostClock.test()` keeps every timing at 0. Pass the same clock to `Session.make`. */
@@ -43,7 +46,7 @@ export const createTopDownSimulation = (options: TopDownSimulationOptions) => {
   const frame = options.frame ?? defaultFrameContext
   const made = Game.Runtime.make({
     services: Game.Runtime.services(
-      Game.Runtime.service(ControlsInput, makeControls({ keyboard: options.keyboard, gamepad: options.gamepad })),
+      Game.Runtime.service(ControlsInput, makeControls({ keyboard: options.keyboard, mouse: options.mouse, gamepad: options.gamepad })),
       Game.Runtime.service(FrameContext, frame)
     ),
     resources: initialResources(frame),

integration/top-down/test/simulation.test.ts

@@ -1,16 +1,17 @@
 import { HostClock } from "@typeonce/ritmo"
-import { Gamepad, Keyboard } from "@typeonce/ritmo-browser"
+import { Gamepad, Keyboard, Pointer } from "@typeonce/ritmo-browser"
 import { Session } from "@typeonce/ritmo-devtools"
 import { describe, expect, it } from "vitest"
 
 import { createTopDownSimulation } from "../simulation.ts"
 import { PlayerPositionQuery } from "../queries.ts"
-import { CollectedCount, Game } from "../schema.ts"
-import { inputBindings, padBindings } from "../types.ts"
+import { CollectedCount, Game, InputState } from "../schema.ts"
+import { inputBindings, mouseBindings, padBindings } from "../types.ts"
 
-const start = (timeline: Keyboard.Timeline<typeof inputBindings>, pad: Gamepad.Timeline = []) => {
+const start = (timeline: Keyboard.Timeline<typeof inputBindings>, pad: Gamepad.Timeline = [], clicks: Pointer.Timeline = []) => {
   const simulation = createTopDownSimulation({
     keyboard: Keyboard.scripted(inputBindings, timeline),
+    mouse: Pointer.scripted(mouseBindings, clicks),
     gamepad: Gamepad.scripted(padBindings, pad),
     hostClock: HostClock.test()
   })
@@ -23,6 +24,9 @@ const start = (timeline: Keyboard.Timeline<typeof inputBindings>, pad: Gamepad.T
 const PlayerPosition = Game.Inspector("Test/PlayerPosition", { queries: { player: PlayerPositionQuery } }, ({ queries }) =>
   queries.player.each().map((match) => match.data.position.get()))
 
+const LastUsed = Game.Inspector("Test/LastUsed", { resources: { input: Game.System.readResource(InputState) } }, ({ resources }) =>
+  resources.input.get().lastUsed)
+
 const Collected = Game.Inspector("Test/Collected", { resources: { count: Game.System.readResource(CollectedCount) } }, ({ resources }) =>
   resources.count.get())
 
@@ -64,6 +68,21 @@ describe("top-down simulation", () => {
     expect(runtime.inspect(Collected)).toBe(1)
   })
 
+  it("collects a pickup with a click while walking, keyboard and mouse as one device", () => {
+    const { runtime, update } = start(
+      [{ frame: 0, press: ["left"] }, { frame: 85, release: ["left"] }],
+      [],
+      [{ frame: 40, move: { x: 100, y: 100 } }, { frame: 88, press: ["primary"], release: ["primary"] }]
+    )
+    const lastUsed = new Set<string | null>()
+    for (let frame = 0; frame < 120; frame++) {
+      expect(runtime.tick(update).ok).toBe(true)
+      lastUsed.add(runtime.inspect(LastUsed))
+    }
+    expect(runtime.inspect(Collected)).toBe(1)
+    expect([...lastUsed]).toEqual(["desktop"])
+  })
+
   it("describes the simulation schedules without lint warnings", () => {
     const { runtime } = start([])
     const description = runtime.debug.describe()
@@ -80,7 +99,7 @@ describe("top-down debug session", () => {
       { frame: 88, press: ["interact"], release: ["interact"] }
     ])
     const hostClock = HostClock.test()
-    const simulation = createTopDownSimulation({ keyboard, gamepad: Gamepad.scripted(padBindings, []), hostClock })
+    const simulation = createTopDownSimulation({ keyboard, mouse: Pointer.scripted(mouseBindings, []), gamepad: Gamepad.scripted(padBindings, []), hostClock })
     if (!simulation.ok) throw new Error(simulation.error.message)
     const { runtime, setup, update } = simulation.value
     const session = Session.make(runtime, { hostClock, schedules: { setup, update } })

integration/top-down/types.ts

@@ -1,5 +1,5 @@
 import { Controls } from "@typeonce/ritmo-browser"
-import type { Gamepad, Keyboard } from "@typeonce/ritmo-browser"
+import type { Gamepad, Keyboard, Pointer } from "@typeonce/ritmo-browser"
 import type * as Vector2Value from "@typeonce/ritmo-math/Vector2"
 
 export type Vector2 = Vector2Value.Vector2
@@ -22,8 +22,12 @@ export const padBindings = {
   interact: ["south"]
 } as const
 export type GamepadInput = Gamepad.Actions<typeof padBindings>
-/** The one input source: both devices merged, built the same way in the browser and in tests. */
-export const makeControls = (devices: { readonly keyboard: KeyboardInput; readonly gamepad: GamepadInput }) => Actions.merge(devices)
+/** A click interacts too; the mouse is grouped with the keyboard, so using both is one device in use. */
+export const mouseBindings = { interact: ["primary"] } as const
+export type MouseInput = Pointer.Actions<typeof mouseBindings>
+/** The one input source: keyboard and mouse as one group, and the gamepad, built the same way in the browser and in tests. */
+export const makeControls = (devices: { readonly keyboard: KeyboardInput; readonly mouse: MouseInput; readonly gamepad: GamepadInput }) =>
+  Actions.merge({ desktop: Actions.group({ keyboard: devices.keyboard, mouse: devices.mouse }), gamepad: devices.gamepad })
 export type ControlsValue = ReturnType<typeof makeControls>
 export type InputStateValue = ReturnType<ControlsValue["snapshot"]>
 /** What one fixed update needs from the host: the step length. */

packages/browser/README.md

@@ -1,6 +1,6 @@
 # `@typeonce/ritmo-browser`
 
-Browser integrations for ritmo: animation frames as a tick source for the core fixed-step loop (`BrowserTickSource`), keyboard actions with press/release edges, bound by character or by physical key (`Keyboard`), gamepad actions by standard-mapping button names plus the sticks and triggers (`Gamepad`), both rebindable at runtime for a controls settings screen (capture the key or button to bind, validate saved bindings with `parseRebinding`), mouse and pen position and button edges over an element (`Pointer`), on-screen touch controls with a floating stick and drag-to-aim buttons (`Touch`), scripted and recorded keyboard, pointer, and gamepad timelines for tests and replays, one input source merged from several devices, with the actions declared once (`Controls`), and a system that copies input into a resource (`InputCapture`).
+Browser integrations for ritmo: animation frames as a tick source for the core fixed-step loop (`BrowserTickSource`), keyboard actions with press/release edges, bound by character or by physical key (`Keyboard`), gamepad actions by standard-mapping button names plus the sticks and triggers (`Gamepad`), both rebindable at runtime for a controls settings screen (capture the key or button to bind, validate saved bindings with `parseRebinding`), mouse and pen actions (rebindable too), raw buttons, and position over an element (`Pointer`), on-screen touch controls with a floating stick and drag-to-aim buttons (`Touch`), scripted and recorded keyboard, pointer, and gamepad timelines for tests and replays, one input source merged from several devices, with the actions declared once and devices used together (keyboard and mouse) grouped as one (`Controls`), and a system that copies input into a resource (`InputCapture`).
 
 ```sh
 pnpm add @typeonce/ritmo-browser @typeonce/ritmo