Skip to content

feat(ts)!: rs.ArraySlice for slices of arrays, replacing the as_chunks_mut proxies - #608

Draft
kixelated wants to merge 18 commits into
mainfrom
quest/m3/ts-slice-object
Draft

kixelated wants to merge 18 commits into
mainfrom
quest/m3/ts-slice-object

Conversation

@kixelated

@kixelated kixelated commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Part 3 of quest/m3/ts-slice-object.md, after moq-dev/rust2#576 and moq-dev/rust2#584.

Problem

A slice of arrays had each array as a JS array of its own, so the arrays as_chunks_mut and as_rchunks_mut lend from a flat slice had no JS array to be. They were a Proxy (Runs, ByteRuns) whose elements were arrayView proxies or subarray views of each array, made and cached on first read, with detaching copies before a whole write. In a micro-bench of the codec bench's packets workload, that cost 10-20x reading each array where the flat buffer holds it.

Approach

  • A new runtime type, rs.ArraySlice<S>, is every &[[T; N]], &mut [[T; N]], &[[T; N]; M], and Box<[[T; N]]>. Element i is the slice of array i: an rs.Slice<T>, an rs.ByteSlice for bytes, or an rs.ArraySlice for arrays of arrays. It reads and writes array i where it is, whether each array is a JS array of its own (items and the of that lends one, as a Vec<[T; N]> or [[T; N]; M] holds them) or n elements of one flat slice (as as_chunks splits it). No Proxy.
  • std's methods that lend arrays or parts of the slice (iter/iter_mut, split_at_mut, chunks_mut, windows, split*, first_chunk*, swap, ...) map to the runtime's arraySlice* functions (ts/src/lower/array_slice.rs's native). Any other std function given a shared slice of arrays reads rs.arraySliceAsSlice: the arrays themselves, so a callback's ptr::eq sees the array it was lent, or copies of those as_chunks split from one flat slice, which have no JS array of their own. A &mut one reads rs.arraySliceBy's copies, written back once it returns, as Rust moves their elements. A function that would return a borrow of those copies is refused with a span.
  • as_chunks/as_rchunks (shared and mutable) and slice::from_mut/array::from_mut of a &mut [T; N] lend an rs.ArraySlice of the same storage. Cell::from_mut of an array of arrays is rs.cellOfArraySlice. mem::replace/mem::swap of an array, clone_from of arrays of arrays and of a Box<[[T; N]]>, raw pointers to arrays of arrays, and Debug of their iterators all go through it.
  • An array copied out and written back (rs.arraySliceBy, rs.arraySliceReplace, rs.arraySliceSwap, rs.cellOfArraySlice) copies its elements by the element type (copy::elem_copy): the type's copy shim where JS keeps a Copy value as an object, rs.objectCopy where a write lands inside the object, or none. So the copy shares no object with the arrays it was read from, and a &Cell that does share one is untouched.
  • Every other form a slice of arrays takes is rs.ArraySlice too: constants and Default, Option::as_slice/as_mut_slice of an array payload (rs.optionAsArraySlice), Cow<[[T; N]]> (rs.Cow<T, rs.ArraySlice<S>>, borrowed by rs.cowArraySlice), assume_init_ref/assume_init_mut of MaybeUninit arrays and Box::assume_init (rs.arraySliceOfSlice), generic structs at an array instance, and concat/join parts and separators. join lends an array separator as it is, since it clones it anyway, so each element's Clone reads the separator's own.
  • The Runs/ByteRuns proxies and View.detach are gone. arrayView remains only where a &mut [T; N] into part of an array is taken whole as a JS array; that remainder is now the whole quest.

Impact

  • Breaking for hand-written callers: a slice of arrays is an rs.ArraySlice rather than an rs.Slice<T[]> / rs.Slice<Uint8Array>. Pass one as rs.arraySliceFrom(arrays, rs.sliceFrom) (rs.byteSliceFrom for bytes, rs.arraySliceOf(rs.sliceFrom) for arrays of arrays) and read one as rs.arraySliceItems(slice). ts/README.md documents it.
  • Runtime exports: added ArraySlice, ArrayRef, LentArray, arraySlice* (arraySliceFrom, arraySliceOf, arraySliceAt, arraySliceItems, arraySliceAsSlice, arraySliceBy, arraySliceSet, arraySliceReplace, arraySliceSwap, arraySliceIter, arraySliceSplit*, arraySliceChunks*, arraySliceWindows, ...), cellOfArraySlice, arraySliceBoxCloneFrom, arraySliceBoxCloneFromFn, arraySliceCopy, ElemCopy, optionAsArraySlice, arraySliceOfSlice, and cowArraySlice. rs.Cow takes a second type parameter, its borrowed form, rs.Slice<T> by default. arraySliceSwap, arraySliceBy, and arraySliceReplace take the element copy as an argument. Removed byteSliceAsChunks, byteSliceAsChunksMut, byteSliceAsRchunks, byteSliceAsRchunksMut, sliceAsChunksMut, and sliceAsRchunksMut. sliceAsChunks/sliceAsRchunks now return rs.ArraySlice.
  • No wire impact.

Decisions

Decided with the maintainer on 2026-11-06:

  1. How a &mut [[T; N]] holds its arrays: ✅ (a) a runtime slice-of-arrays type for every slice of arrays, nested or flat, which replaces the Runs/ByteRuns proxies (rejected: (b) eager subarray views for bytes only, keeping the Proxy for other elements; (c) keep the proxies and drop the item). The quest narrows to it and is sized [L].
  2. slice::from_mut/array::from_mut of a &mut u8: ✅ kept, deferred to its own quest at the end of m3, quest/m3/ts-byte-slice-place.md (rejected: drop it until a consumer appears).
  3. Comparing *const [T] of values JS copies: ✅ moved into quest/m3/ts-runtime-throws.md with the form_urlencoded 1.2.2 debug_assert!(ptr::eq(..)) finding (rejected: wait on rs.String's UTF-8 route).

Decided with the maintainer on 2026-11-07, at the round-3 design checkpoint (two medium findings, both conversions that copied arrays where Rust borrows them):

  1. ✅ Continue, with local fixes (rejected: merge as is; rethink to rs.ArraySlice only for what as_chunks splits, keeping rs.Slice<T[]> for other slices of arrays; drop). Where each array is a JS array of its own, rs.arraySliceItems slices the list of them instead of copying each, so arrays keep their identity for std callbacks such as binary_search_by and for ptr::eq. join reads its separator without copying it. The arrays as_chunks splits from one flat buffer have none of their own, so a callback still gets a copy; that ptr::eq gap is recorded in quest/m3/ts-runtime-throws.md.
  2. ✅ Rename rs.Runs to rs.ArraySlice, matching rs.ByteSlice and rs.RawSlice, with its helpers rs.arraySlice* (rejected: rs.Slices, rs.Chunks, keep rs.Runs).

The quest file keeps only the arrayView remainder, sized [M].

Choices made without asking:

  • array_slice::give converts a slice of arrays for every std function that isn't a trait's method, not just [T]'s own methods, so Vec::extend_from_slice(&[[T; N]]) and the like get the slice they read.
  • A copy of an array copies its elements by the element type in the translator, not by inspecting the storage at run time: a storage-directed copy in rs.sliceToArray (an earlier head) also copied arrays a &Cell shares, which Rust doesn't.
  • rs.sliceItems keeps one signature for rs.Slice, and the translator emits rs.arraySliceItems for an array slice. An overload made rs.sliceItems lose its type when passed as a function, as Option::copied passes it.
  • With the rename, the type an rs.ArraySlice lends each array as is rs.ArrayRef (was Run), the array it lends rs.LentArray (was RunArray; rs.ArrayOf was taken), the element copy rs.ElemCopy (was RunCopy), and the old runsArray/runsItems are arraySliceItems/arraySliceAsSlice, after sliceItems and as_slice. The translator's module is lower/array_slice.rs.

Bench

Rerunning at the new head; the numbers below are from 2c7ae6f.

bench/main.ts at 2c7ae6f, main routed to the merge base 324a7b7's translation (.tmp/bench.sh under nix develop .#bench). Nanoseconds per packet, median ± half the IQR:

runtime packets/ts/number main → PR packets/ts/number-copy main → PR A/A at the median workload
Node 24.21 568 ±11% → 70.4 ±23% 1000 ±10% → 331 ±16% 4.5%
Bun 1.4.2 474 ±26% → 194 ±19% 562 ±11% → 285 ±27% 7.7%
Chromium 156 515 ±4.4% → 119 ±10% 1140 ±9.9% → 690 ±7.3% 3.2%
Firefox 159 431 ±30% → 107 ±24% 783 ±27% → 471 ±19% 4.2%

Validation

  • just ci at b2a9375: passes (fixtures 89/89, including the lang fixtures' differential tests against native Rust). slices::array_forms covers each form above; slices::array_identity pins a callback's ptr::eq on a lent array (of bytes, words, and arrays, of all of a slice and of part of it) and join's separator Clone reading its own address. Each fails against native Rust with its fix reverted (74 and 79 differences).

Alternatives

  • Bytes only, with eager subarray views (decision 1b): 3-4x the flat form, and it keeps the Proxy for other element types.
  • One rs.Slice per array made eagerly: allocates every array's slice up front; rs.ArraySlice makes one only when it is read.

Follow-ups

Models

Size: L
Implementer: Claude Code, claude-opus-5-5, unknown effort
Implementer: Claude Code, claude-opus-5-5, high effort

Review

Rewritten by quest review after each round. A new session starts after the round-3 design checkpoint.

🤖 Generated with Claude Code

@pullfrog

pullfrog Bot commented Nov 6, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

@kixelated kixelated changed the title rs.Slice rs.Slice: runs of as_chunks_mut, from_mut of a byte, *const [T] comparison Nov 6, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 4 commits November 6, 2026 16:38
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… in, and pass as slices to std's other functions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title rs.Slice: runs of as_chunks_mut, from_mut of a byte, *const [T] comparison feat(ts)!: rs.Runs for slices of arrays, replacing the as_chunks_mut proxies Nov 7, 2026
kixelated and others added 3 commits November 6, 2026 19:00
…eir elements' type

Constants, defaults, Option::as_slice, a Cow's borrow, and the erased
SliceOf and Ptr are runs too; to_vec reads runs as they are; a wrapped
array is an element, not runs; and runs copied out and written back copy
an element only as its type says: a Copy object's copy, or objectCopy of
one a write lands in, never a reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nitialized MaybeUninits, and joined slices of arrays

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 5 commits November 6, 2026 20:30
…they are

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tive Rust

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Matches rs.ByteSlice and rs.RawSlice, as the maintainer decided at the
round-3 design checkpoint.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title feat(ts)!: rs.Runs for slices of arrays, replacing the as_chunks_mut proxies feat(ts)!: rs.ArraySlice for slices of arrays, replacing the as_chunks_mut proxies Nov 7, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: after the round-3 checkpoint, the maintainer chose to continue with local fixes and to rename the type.

  • Where each array of an array slice is a JS array of its own, rs.arraySliceItems now slices the list of arrays instead of copying each, so arrays keep their identity for std callbacks (binary_search_by) and ptr::eq. join lends its array separator without copying it, since it clones it anyway. The arrays as_chunks splits from one flat buffer have none of their own and still reach a callback as copies; that ptr::eq gap is recorded in quest/m3/ts-runtime-throws.md.
  • rs.Runs is now rs.ArraySlice, matching rs.ByteSlice and rs.RawSlice, and its helpers are rs.arraySlice* (runsArray → arraySliceItems, runsItems → arraySliceAsSlice).
  • A new differential case, slices::array_identity, pins both identity cases against native Rust.

Rejected: merge as is; rs.ArraySlice only for as_chunks output; drop; the names rs.Slices, rs.Chunks, and keeping rs.Runs. The next review starts a new session with quest review --new.

(Written by Claude Opus 5.5 with high effort)

kixelated and others added 4 commits November 6, 2026 22:13
…s of arrays

VecDeque's as_slices, as_mut_slices, and make_contiguous and BinaryHeap's
as_slice of arrays lend an rs.ArraySlice of their own arrays, and a raw
pointer to a slice of arrays is an rs.RawArraySlice of it, so Box::into_raw
and from_raw give back the same storage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	ts/runtime/rs/collections.ts
…d a borrowed Cow, and split arrays lent lazily

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant