Skip to content

feat(ts): translate Vec::from_raw_parts of memory from alloc or a Vec's - #635

Draft
kixelated wants to merge 6 commits into
mainfrom
quest/m3/ts-unsafe/vec-from-raw-parts
Draft

kixelated wants to merge 6 commits into
mainfrom
quest/m3/ts-unsafe/vec-from-raw-parts

Conversation

@kixelated

@kixelated kixelated commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m3/ts-unsafe/vec-from-raw-parts.md.

Problem

rust2-ts refused Vec::from_raw_parts as an unsupported call. So a Vec could not adopt memory that alloc gave for a Layout::array::<T>(n). Typed allocation is a JS array of Ts since the epic's 2026-11-01 decision, and an rs.Vec holds std's capacity since moq-dev/rust2#555. smallvec's Drop and into_vec hand their spilled heap back this way.

Approach

  • Vec::<T>::from_raw_parts(p, length, capacity) lowers to rs.vecFromRawParts(p, length, capacity, size). p points at the start of a JS array: memory from alloc (an rs.Block's array), or a Vec's own items given up by ManuallyDrop or mem::forget. The new rs.Vec holds that same array with cap = capacity. Everything past length moves into the array's rs.Room (moq-dev/rust2#623), so a set_len back over slots written before the adoption reads them, as std's does. A room the array already had, from an earlier Vec, carries over to the new one. A slot alloc never wrote stays unwritten.
  • The step that cuts and lengthens the items through the room is now one helper, relength, shared with vecSetLen.
  • Vec<u8> lowers to rs.byteVecFromRawParts, which takes the same Uint8Array as its storage. Byte allocation from alloc stays refused, as the epic decided, so this covers a byte Vec's own buffer.
  • A dangling pointer gives an empty Vec with the given capacity (Vec::from_raw_parts(NonNull::dangling().as_ptr(), 0, 0)).
  • A pointer to a value in no array, such as Box::into_raw's, gives a Vec of an array of that one value. A runtime WeakMap (HOMES, through homeOf) records the storage each such place was given, and the place is redefined to read and write that storage's slot 0. A write through either side shows through the other, and adopting the same value again reuses the storage, so a pointer from the first Vec sees writes through the second. Any later route that turns a lone value into array storage can reuse homeOf; only from_raw_parts pays for it now. Bytes from alloc that no cast typed (an rs.Block) give a byte Vec new storage of the capacity, since a write to them is refused.
  • realloc of the memory under a Vec now carries over what the Vec's room holds past its length (rs/alloc.ts). Before, it copied only the items array, so it lost what a pointer or set_len had put past the length.
  • A length past the capacity panics with std's ub_checks message. That panic can't unwind, so it aborts, located at from_raw_parts_in's check (new vec_from_raw_parts site in caller.rs).
  • A pointer that isn't to an array's start, and byte storage whose length isn't the capacity, are UB in Rust. Both throw a rust2-ts: error (tests/pointers.ts).
  • Refused with a span: elements JS keeps as objects (a pointer to one knows no array it is in, as for as_mut_ptr) and zero-sized elements.
  • Proven against native wasm32 by the new adopt module in the lang/heap fixture:
    • adopt::allocated: u32s, Arc<u8>s (strong counts show each drop and leak), and Option<(u8, u16)>s. They are adopted from alloc with Layout::array or an equal from_size_align layout, grown by realloc or not, with slots written past the length. Each Vec then pushes past the capacity, takes in the written slots by set_len, truncates, or shrinks.
    • adopt::readopted: Vec<u32>, Vec<Arc<u8>>, and Vec<u8> round-tripped through ManuallyDrop, with room written through the pointer. Also: an empty Vec's spare-capacity pointer; a Vec grown by realloc and adopted again; Vec<u32> and Vec<u8> of a Box's value, written through both the Vec and the pointer; the same Box adopted twice, for u32 and u8, written through the second Vec, the first Vec's pointer, and the Box's pointer; Vec<u8> of bytes from alloc; and the dangling empty Vec.
    • The hooked panics fixture's new arm 74 covers the abort.

Impact

  • Public API: the translator's output. Vec::from_raw_parts translates where it was refused.
  • Runtime: new exports vecFromRawParts and byteVecFromRawParts. realloc keeps a Vec's room.
  • Census: Vec::from_raw_parts is verbatim in constructs. The model census at the moq pin (ffcb7cd) lists no Vec::from_raw_parts refusal (50 refusals in all). Before this PR, smallvec's Drop for its two instances (Weak<Waker>, NonNull<Segment>) reached it, per the quest.
  • Quests: deletes this quest and its links from the unsafe epic.

Choices made without asking

  • A Vec's own buffer is adopted too, not only memory from alloc. It is the same runtime path: the pointer is as_mut_ptr's rs.AtIndex into the items, whose room the new Vec inherits. It covers the common ManuallyDrop round trip, and leaving it out would be a refusal with no simpler result.
  • Capacity is trusted, not checked against the allocation. Rust requires capacity * size_of::<T>() to equal the allocation's size, and breaking that is UB, so the runtime takes the capacity it is given. Only the byte path checks it, because a ByteVec's capacity is its storage's length.
  • Zero-sized elements are refused rather than modeled. alloc already refuses them, and the JS value of a zero-sized type is not always undefined.
  • The adopted memory's rs.Block stays registered. A later cast of the Vec's pointer back to bytes for dealloc finds it either way. Unregistering would change nothing a defined program sees.

Decisions

  • Storage of a value in no array that Vec::from_raw_parts adopts (maintainer, 2026-11-07, at the round-3 design checkpoint): ✅ a runtime WeakMap from the lone value's place to the array storage it was given, so adopting it again reuses that storage, built as a general mechanism any later route turning a lone value into array storage can reuse, and paid for only by from_raw_parts (rejected: make every Box::into_raw pointer an array of one from the start; drop Box adoption and throw; merge as is).

Alternatives

  • Copy the array into a fresh rs.Vec: simpler, but a pointer taken before the adoption (p.add(i)) would no longer reach the Vec's items or its room.

Follow-ups

  • String::from_raw_parts (and String::from_utf8_unchecked of an adopted Vec<u8>). Drop: no consumer at the moq pin, and byte allocation from alloc stays refused.

Caused by

No single PR: Vec::from_raw_parts was never supported.

Validation

On head 3342dd820 (origin/main fd62edf merged in):

  • just ci origin/main (in nix develop): lint and quest check pass, and so do the 8 unit tests, all 89 fixture tests (lang::dyns too, since moq-dev/rust2#633 landed), and the 10 runtime tests.
  • The twice-adopted Box case against the round-3 runtime (no WeakMap): 164 differences in 841 lang/heap cases; with it, 0.

Earlier, on head b11502c:

  • just ci origin/main (in nix develop): lint and quest check pass, and so do the 8 unit tests and 88 of 89 fixture tests. The one failure is lang::dyns (ids.ts TypeId hashes), which depends on the checkout's path. Per the coordinator's known-issues list it reproduces on bare origin/main (I did not rerun it there), and moq-dev/rust2#633 is fixing it. just test --test runtime, which cargo skipped after that failure, passes on its own (10 tests).
  • Probe of 3,000 random inputs per lang/heap case against the native build: 15,086 cases, 0 differences. Each runtime fix was checked by removing it in the copy under test, with the new cases in place:
    • without relength's room write: 149 differences in 1,586 cases;
    • without realloc's room copy: 313 differences in 1,586 cases;
    • without rehome or lone-place adoption: 164 differences in 841 cases.
  • The hooked panics fixture at 1,500 inputs per case: 0 differences. With the length check moved past the capacity, it shows 14 differences.
  • The model census at the moq pin, ffcb7cd: 50 refusals, none of them Vec::from_raw_parts.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, high effort

(Written by Claude Opus 5.5 with high effort)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, medium effort
  • Session: 01a1b61d-72dd-78b1-a322-97d31ffe645c, round 3
  • Head: b11502c, named in full once a round reports no findings
  • Base: origin/main at efa6300
  • Score: 8/10, The fix preserves aliases during the first adoption, but repeated adoption still splits one Rust allocation into separate storage.
  • Findings, open until the next round settles them:
    • Medium, ts/runtime/rs.ts:2651: Reuse the boxed pointer's storage when adopting it again
  • The last round's findings, at 84e1381:
    • Open: medium, ts/runtime/rs.ts:2651: Preserve the original pointer's storage when adopting a boxed value; The rehome calls fix the initial write-through scenario, but ts/runtime/rs.ts:2651 still copies storage on repeated adoption, leaving pointers from the first vector disconnected.

@pullfrog

pullfrog Bot commented Nov 7, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

kixelated and others added 3 commits November 7, 2026 00:26
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acity pointer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Vec::from_raw_parts of a typed allocation feat(ts): translate Vec::from_raw_parts of memory from alloc or a Vec's Nov 7, 2026
kixelated and others added 2 commits November 7, 2026 05:10
…through realloc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: the maintainer picked a runtime WeakMap at the round-3 checkpoint. It maps a lone value's place (such as Box::into_raw's) to the array storage Vec::from_raw_parts gave it, so adopting the same value again reuses that storage, and a pointer from the first Vec sees writes through the second. It is one private helper (homeOf in ts/runtime/rs.ts) that any later route turning a lone value into array storage can reuse; only from_raw_parts calls it now. Rejected: an array of one for every Box::into_raw pointer, refusing Box adoption, and merging as is. adopt::readopted now adopts the same Box twice, for u32 and u8, and writes through both sides. Review restarts with quest review --new.

(Written by Claude Opus 5.5 with high effort)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant