Skip to content

feat(ts): translate str::as_ptr as a read-only pointer into its UTF-8 - #634

Draft
kixelated wants to merge 4 commits into
mainfrom
quest/m2/ts-model/str-as-ptr
Draft

kixelated wants to merge 4 commits into
mainfrom
quest/m2/ts-model/str-as-ptr

Conversation

@kixelated

@kixelated kixelated commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Closes the quest quest/m2/ts-model/str-as-ptr.md.

Waiting

  • Maintainer: the design checkpoint after review round 3. Codex holds two findings against my refutations. Decide whether a read through a str's pointer outside that str's own bytes may fail at run time, or must be refused with a span. Check: a PR comment starting Design change: or a reply on this PR.

    The two cases are reads beyond the substring but inside its text's allocation:

    • s[..1].as_ptr().add(1), read forward;
    • s[1..].as_ptr().wrapping_sub(1), read back.

    Both are undefined under Stacked Borrows, Miri's default model, because the pointer comes from the substring's reference. Tree Borrows allows them. Here they throw a rust2-ts: error. The static refusal of a constant sub, from round 1, is kept.

    Options:

    1. Merge as is (recommended). They fail loudly, never misread, and are UB under Miri's default model. That matches how the translator already treats Array::Copy and Array::Lone, which also go unchecked for wrapping moves and unknown lengths.
    2. Track str lengths and wrapping offsets through to reads in mir/origin.rs. This is a new read-tracking analysis for every pointer kind. It would still miss pointers the body doesn't show, so the runtime failure stays as the backstop.
    3. Let a substring share its text's encoding. A &str would become a view (text, offset), reversing the representation moq-dev/rust2#574 settled and paying its map-key costs.
    4. Refuse str::as_ptr unless the body proves every read stays within [0, len). This is likely too narrow, and could refuse arrayvec again.

Problem

str::as_ptr was an unsupported call, so arrayvec's ArrayString::try_push_str, which copies a &str into its buffer through the pointer, refused translation. At the moq pin that is 1 refusal in the model's census and 3 in moq-net whole (arrayvec's, plus two in zlib-rs). It was the last child of quest/m2/ts-model/README.md that this repo owns. moq-dev/rust2#574 settled that a &str stays a JS string, so the bytes behind the pointer are an encoding of the text.

Approach

  • rs.strAsPtr(s) encodes the text's UTF-8 and returns a pointer to its first byte. The pointer is a TextAt, a private AtIndex subclass. Reads work as they do for any byte pointer: *p, read, arithmetic, offset_from within one encoding, ptr::copy/copy_nonoverlapping from it, and slice::from_raw_parts.
  • A write fails loudly. Rust forbids writing through a &str's pointer. Here such a write would land in a copy the text never reads, so it fails at run time instead. That covers *p = x, a copy into it, and a mutable slice (from_raw_parts_mut, or derefBytes(false) of a raw slice). Arithmetic keeps the pointer read-only: AtIndex gains an at(index) method that ptrAdd and ptrWrappingAdd call, and TextAt overrides it.
  • Origins. Each str::as_ptr call is its own origin (Array::Text) in mir/origin.rs. So offset_from between two calls' pointers is refused with a span wherever the body shows both. A typical case is a substring's pointer against its text's, which Rust allows but JS can't match. A pointer starts at index 0 of its own encoding, so a constant add/sub/offset the body shows moving it back before its first byte is refused with a span, as one off an Array::Copy is. That covers a substring's pointer moved back into its text. Anything the body doesn't show is left to the runtime, which fails on a read outside the encoding.

Validation

  • Differential, lang/strs' new strs::pointers: text of every UTF-8 width and a substring from each char boundary, against native Rust. It copies into a Vec's room and between bytes, reads every tail through from_raw_parts, walks forward and back by *, NonNull, and offset_from/offset_from_unsigned, and pushes into a [MaybeUninit<u8>; 12] as ArrayString does.
  • Differential, stock's new fixed::strings: the real arrayvec::ArrayString<12> (0.7.8, the version moq's lock has). It covers try_push, try_push_str with its errors, push_str, pop, truncate, remove, from, and clear over chars of each width.
  • Runtime, tests/pointers.ts: reads, arithmetic, and views pass. Writes, copies in, and mutable slices fail with a rust2-ts: error, and so does offset_from across two encodings.
  • Reject census: text_apart's two offset_froms and text_before's sub(1) are refused with spans.
  • Census at the moq pin, this branch against the same branch with the new mapping removed: model 52 → 51 and moq-net 356 → 353. The only lines that change are the three str::as_ptr refusals, and no new refusal appears.
  • just ci at 27f3871e7 passes except lang::dyns (ids.ts) and lang::panics (hook/lib.ts). Those are the TypeId-hash snapshots that depend on the checkout's path: they fail on bare origin/main too, CI passes them, and moq-dev/rust2#633 is fixing them.

Impact

Public API (translator output):

  • str::as_ptr translates instead of refusing.
  • New runtime export: rs.strAsPtr.
  • New AtIndex.at(index) method.
  • The census quotes str and str::as_ptr verbatim in its constructs.
  • Fewer refusals: arrayvec's ArrayString::try_push_str, and zlib-rs' two str::as_ptrs.

Quests:

  • Deletes quest/m2/ts-model/str-as-ptr.md and its link.
  • quest/m2/ts-model/README.md loses its last child. It is sized [L] for its remaining end-to-end work, and its census paragraph is refreshed to 51.

Choices made without asking

  • Runtime write checks, not a translation-time refusal. A write needs cast_mut or as *mut of the pointer, and the result can flow anywhere, so only the runtime sees every write. That is the same split the README documents for pointers from elsewhere. The checks cost one instanceof per copy or mutable slice and nothing on other pointers' writes, because TextAt is its own class.
  • Two UB-only routes stay unchecked, since each needs a second forbidden step and checking them would cost every byte pointer:
    • a write through a *mut [u8; N] cast from a raw slice of the text;
    • a write through the as_ptr of a shared slice that from_raw_parts made of the text.
  • A new encoding per call. No cache, since JS strings can't be weak keys. That makes two calls' pointers different arrays, which is why they are refused or fail when measured against each other. Comparing *const u8s was already refused.
  • str::as_mut_ptr stays unsupported. Writes through it are defined in Rust and must reach the String, so it needs the bytes to be the string's own storage. Nothing in moq calls it, and it is outside this quest's Goal.
  • arrayvec added to the stock fixture's dependencies, at moq's locked version, so the real consumer crate is tested against native and not only a copy of its code.

Alternatives

  • Caching one encoding per string, so that ptr::eq and offset_from agree across calls. A strong Map of strings leaks, and a substring is a new JS string anyway, so its offset into its text still could not match.
  • Making the bytes read-only through an immutable ArrayBuffer (transferToImmutable). Node, Bun, Chrome, and Firefox don't all ship it, and writes to it fail silently outside strict mode.

Follow-ups

None.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, high effort

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, medium effort
  • Session: 01a1b52a-0b86-7ac1-8df0-8726554271f2, round 3
  • Head: 27f3871, named in full once a round reports no findings
  • Base: origin/main at efa6300
  • Score: 8/10, Both valid Rust inputs still produce runtime failures, and the cited implementation limits do not override the base branch's requirement for source-spanned compile errors.
  • Findings, open until the next round settles them:
    • Medium, ts/src/lower/mir/origin.rs:40: [P2] Track the encoded substring's upper bound
    • Medium, ts/src/lower/mir/origin.rs:246: [P2] Check substring reads reached through wrapping arithmetic
  • The last round's findings, at 27f3871:
    • Open: medium, ts/src/lower/mir/origin.rs:40: [P2] Track the encoded substring's upper bound; ts/src/lower/mir/origin.rs:40 still discards text lengths, and the reproduced forward-read failure violates the base rules regardless of the cited analysis scope or README documentation.
    • Open: medium, ts/src/lower/mir/origin.rs:246: [P2] Check substring reads reached through wrapping arithmetic; The arithmetic tracking in ts/src/lower/mir/origin.rs:239 still excludes wrapping operations, and the reproduced valid backward read throws; existing omissions for other origins do not authorize this semantic mismatch.

@pullfrog

pullfrog Bot commented Nov 7, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

kixelated and others added 2 commits November 7, 2026 00:24
A `&str` stays a JS string, so `rs.strAsPtr` encodes its UTF-8 and points
at the first byte. Reads, arithmetic, copies from it, and `from_raw_parts`
read what Rust reads; a write, a copy into it, or a mutable slice of it,
which Rust forbids, fails at run time. `offset_from` between two calls'
pointers is refused where the body shows both.

arrayvec's `ArrayString` now translates: the stock fixture runs it against
native Rust.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title A string's bytes through its pointer feat(ts): translate str::as_ptr as a read-only pointer into its UTF-8 Nov 7, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant