Skip to content

feat(ts): translate writes into a Vec's spare capacity, committed by set_len, for any T - #623

Merged
kixelated merged 11 commits into
mainfrom
quest/m3/ts-unsafe/vec-spare-capacity
Nov 7, 2026
Merged

kixelated merged 11 commits into
mainfrom
quest/m3/ts-unsafe/vec-spare-capacity

Conversation

@kixelated

@kixelated kixelated commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m3/ts-unsafe/vec-spare-capacity.md.

Problem

rust2-ts refused Vec::<T>::set_len and spare_capacity_mut for any T but a byte, and a write through as_mut_ptr().add(len) threw a read or write through a pointer past the array JS keeps it in where Rust defines it. An rs.Vec's items are a JS array exactly as long as the Vec, so a write past the end would lengthen it, and nothing held the room past the length. Throw only where Rust panics makes that throw a gap.

Approach

  • The runtime keeps a Vec's room past its length apart from its items, in an rs.Room (a Map by index), found from the items array through a WeakMap. Vec::as_ptr and as_mut_ptr are now rs.vecAsPtr(v, size), which registers the room; AtIndex reads and writes, ptr::copy, slice::from_raw_parts, and slice::from_mut past the length reach it, up to the capacity. A pointer past an array that is no Vec's items, or into items a step since replaced, still throws, as before.
  • spare_capacity_mut is rs.vecSpare(v, size, uninit): a slice whose array is a proxy view (a View subclass, as as_chunks_mut's runs are) of the items and then the room. A slot of a type JS keeps as an object is made by uninit on first read, so slot.write(v) copies into it as for Box::new_uninit_slice.
  • set_len is rs.vecSetLen(v, n, size, uninit): it panics past the capacity with std's precondition message, as byteVecSetLen does, fixes the capacity, moves what it gives up into the room and takes slots in from it. So std's Drain and retain patterns (shorten by set_len, read past the length through the pointer, lengthen again) are exact.
  • A slot never written is what an unwritten MaybeUninit is (undefined for a value JS copies), which a copy keeps.
  • Proven by the new spare::spares case in the lang/elements fixture against native wasm32: u32, String, and Option<u16> through pointers (by element, copy_nonoverlapping, a slice of the pointers, an insert that moves the tail into the room, a drain and a retain through set_len), and those plus a drop-counting struct, a struct, and a tuple through spare_capacity_mut slots (write, whole assignment, write_clone_of_slice, fill_with, read before set_len), and unwritten slots copied, including a Vec<MaybeUninit<u16>> lengthened over them. The hooked panics fixture's set_len past the capacity now covers a Vec<u32> too, and tests/pointers.ts the room's bounds.

Impact

  • Public API: the translator's output. Vec::<T>::spare_capacity_mut and set_len translate for every T, and Vec::as_ptr / as_mut_ptr lower to rs.vecAsPtr instead of rs.arrayAsPtr(v.items) (snapshots of elements, slices, and the hooked panics change accordingly).
  • Runtime: new exports vecAsPtr, vecSpare, vecSetLen, roomPast, and the Room class (runtime-internal, used by collections.ts).
  • The reject fixture loses buffers::spare, which translates now; the census drops its Vec::spare_capacity_mut verbatim entry.
  • Quests: deletes this quest and its links from Vec::drain from std's MIR and BinaryHeap from std's MIR, which are now unblocked, the epic, and Throw only where Rust panics.

Decisions

  • Decided 2026-11-07 (maintainer, /quest-complete): a step other than set_len that shortens a non-byte Vec (pop, truncate, clear, remove, drain, retain, ...) leaves the room as it was. ✅ land as is, with no follow-up quest (rejected: throw on a set_len that regrows over such a slot; every shortening step saves what it gives up; rework rs.Vec to keep its length apart from capacity-long storage). std's Vec::set_len requires old_len..new_len to be initialized, and the Vec docs' Guarantees section says it "will not specifically overwrite any data that is removed from it, but also won't specifically preserve it. Its uninitialized memory is scratch space". So a slot a shortening step gave up is uninitialized, and lengthening over it breaks the contract (for a type with a destructor, it drops twice). Native's [1, 2] for let mut v = vec![1u32, 2]; v.pop(); unsafe { v.set_len(2) } is an accident of today's std, and [1, undefined] is a valid result. ts/README.md now cites the contract instead of calling this a gap, and no test covers the pattern.

Choices made without asking

  • Pointers into a Vec of objects stay refused. as_mut_ptr of a Vec<T> whose T JS keeps as an object (a struct, a tuple, or a type with Drop) is refused with a span, as the epic's 2026-11-01 decision keeps arithmetic on pointers to objects. Those Vecs get their spare capacity through spare_capacity_mut and set_len, which translate for every T. The quest's "for any T" holds for that path, and for the pointer path wherever the pointer model allows it.
  • The room is a Map beside the items, not a separate length field on rs.Vec. Giving rs.Vec a length apart from items.length would touch every Vec operation and the hot paths the capacity work kept free. A room costs a WeakMap lookup per as_ptr / as_mut_ptr call and nothing elsewhere. No moq-net workload calls Vec::as_ptr (the model fixture's snapshot is unchanged), so I took no bench numbers.

Alternatives

  • A separate JS array for the spare slots, which spare_capacity_mut would slice directly with no proxy: its pointers would then be in another array than as_mut_ptr's, so offset_from between them and arithmetic across the length would break.
  • A symbol property on the items array that points at its Vec, instead of the WeakMap: it saves a lookup, but it changes the hidden class of every array that had a pointer taken.

Follow-ups

None.

Caused by

No single PR: the run-time throw was the decided behavior of moq-dev/rust2#400 until moq-dev/rust2#597's rule made it a gap.

Validation

  • just ci origin/main: passes on the head (ts lint, 89 fixture tests, 10 runtime tests, quest check).
  • Review fixes: mem::take and mem::swap of a Vec with a room, and a raw slice read through a pointer from before the move. Each is covered by spare::spares and fails the probe without its fix.
  • A probe of 3,000 random inputs for each of the 10 spare::spares kinds against the native build: no differences.
  • Head 7ebdd7a merges main after review round 3 on 58e9b0b; the next commit rewords the README's line on shortening steps per the decision above.
  • Head a21768c merges main twice after review round 4 (main landed moq-dev/rust2#574's strings among others): conflicts in ts/README.md, collections.ts's imports, and the reject fixture's sources resolved by hand, keeping both sides; the reject census, expected errors, the hooked panics snapshot, and the elements snapshot (a String is now rs.String) regenerated with UPDATE_EXPECT=1. just ci origin/main passes (89 fixture tests, 10 runtime tests, quest check); review round 5 is clean.

Models

Size: M
Implementer: Claude Code, claude-opus-5-5, high effort

(Written by Claude Opus 5.5 with high effort)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, high effort
  • Session: 01a1b3c6-4f0e-7922-bd37-79f7bf42269a, round 5
  • Head: a21768c
  • Base: origin/main at f56859c
  • Score: 10/10, No defects found in the specified branch diff; runtime type-checking, pointer tests, targeted move probes, and 8,137 existing differential cases pass.
  • Findings: none

@pullfrog

pullfrog Bot commented Nov 7, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

kixelated and others added 4 commits November 6, 2026 17:43
…set_len, for any T

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…apacity

# Conflicts:
#	ts/fixtures/reject/census.txt
#	ts/fixtures/reject/expected.txt
#	ts/fixtures/reject/ts.toml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title A Vec's spare capacity feat(ts): translate writes into a Vec's spare capacity, committed by set_len, for any T Nov 7, 2026
kixelated and others added 3 commits November 6, 2026 18:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…apacity

# Conflicts:
#	ts/fixtures/lang/panics/hook/expected/hooked.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 7, 2026 04:28
@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 36cb151b-e830-43b2-ab27-3b228d1c18ee
Commits

Reviewing files that changed from the base of the PR and between f56859c and a21768c.

Files selected for processing: 29

  • quest/m3/ts-runtime-throws.md
  • quest/m3/ts-unsafe/README.md
  • quest/m3/ts-unsafe/binary-heap-mir.md
  • quest/m3/ts-unsafe/vec-drain-mir.md
  • quest/m3/ts-unsafe/vec-spare-capacity.md
  • ts/README.md
  • ts/fixtures/lang/elements/expected/core/bool.ts
  • ts/fixtures/lang/elements/expected/core/option.ts
  • ts/fixtures/lang/elements/expected/elements.ts
  • ts/fixtures/lang/elements/expected/lib.ts
  • ts/fixtures/lang/elements/expected/spare.Noted.ts
  • ts/fixtures/lang/elements/expected/spare.Ops.ts
  • ts/fixtures/lang/elements/expected/spare.Pair.ts
  • ts/fixtures/lang/elements/expected/spare.ts
  • ts/fixtures/lang/elements/src/lib.rs
  • ts/fixtures/lang/elements/src/spare.rs
  • ts/fixtures/lang/panics/hook/expected/hooked.ts
  • ts/fixtures/lang/panics/src/hooked.rs
  • ts/fixtures/lang/slices/expected/slices.ts
  • ts/fixtures/reject/census.txt
  • ts/fixtures/reject/expected.txt
  • ts/fixtures/reject/src/buffers.rs
  • ts/fixtures/reject/ts.toml
  • ts/runtime/rs.ts
  • ts/runtime/rs/collections.ts
  • ts/src/census.rs
  • ts/src/lower/library.rs
  • ts/src/lower/pointer.rs
  • ts/tests/pointers.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 0fbefa8 against base 324a7b7.

No actionable findings in the reviewed spare-capacity implementation. Room storage is keyed by the allocation's items array, and the raw-slice view keeps that array rather than following the original Vec wrapper after mem::take or swap. Pointer copies snapshot their source before writing, including slots past the logical length. set_len fixes capacity before changing length, saves the shortened tail without dropping it, and takes initialized slots back into items. The fixture exercises pointer writes/copies, raw slices, modeled drain/retain patterns, object-backed spare slots, moves, drop counts, and copying uninitialized MaybeUninit slots before writing them.

Direction: keep allocation identity and logical length separate as future Drain/BinaryHeap work builds on this. An explicit offset_from case between a spare-capacity pointer and the Vec's original pointer, plus a spare-slice array-view case, would further pin the View normalization used here. These are coverage suggestions, not identified defects. I did not treat regrowing removed elements after pop/truncate as a required preservation guarantee; that documented decision is distinct from the explicitly supported set_len-shortening path.

Limits: GitHub-only static review of the runtime, lowering, source fixtures, pointer tests, surrounding View/pointer/capacity machinery and documentation. No build, differential tests, random probes, benchmark, quest context, or CI executed, and the PR's reported validation was not independently reproduced. No claim of exhaustive coverage of every element type or unsafe program. This COMMENT is not merge approval.

…c-spare-capacity

# Conflicts:
#	ts/README.md
#	ts/fixtures/lang/panics/hook/expected/hooked.ts
#	ts/fixtures/reject/census.txt
#	ts/fixtures/reject/expected.txt
#	ts/fixtures/reject/src/buffers.rs
#	ts/fixtures/reject/ts.toml
#	ts/runtime/rs/collections.ts
@kixelated
kixelated added this pull request to the merge queue Nov 7, 2026
Merged via the queue into main with commit da768d1 Nov 7, 2026
4 checks passed
@kixelated
kixelated deleted the quest/m3/ts-unsafe/vec-spare-capacity branch November 7, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant