Skip to content

Utf8Stream writes duplicate bytes when a short write splits a multi-byte character #67679

Open

Description

@jakecastelli

Version

v28.0.0-pre (main), v26.13.1, v24.22.3

Platform

Darwin 24.6.0 arm64

Subsystem

fs

What steps will reproduce the bug?

When fs.write() commits only partial of a string and the cut lands inside a multi byte UTF-8 character, fs.Utf8Stream leaves the already written lead bytes on the fd and then writes the whole character again.

See repro below forces one short write through the fs option. The short write itself is real: 4 of the 9 bytes of 'abc中def' reach the fd, which cuts 中 (e4 b8 ad) after its first byte.

'use strict';
const fs = require('node:fs');
const os = require('node:os');
const path = require('node:path');

const file = path.join(os.tmpdir(), `utf8stream-${process.pid}.txt`);
const fd = fs.openSync(file, 'w');

let shortWriteDone = false;
const stream = new fs.Utf8Stream({
  fd,
  fs: {
    // Simulate a short write: the first call commits only 4 of 9 bytes,
    // which cuts '中' (e4 b8 ad) after its first byte.
    write(fd, data, ...rest) {
      if (shortWriteDone) return fs.write(fd, data, ...rest);
      shortWriteDone = true;
      fs.write(fd, Buffer.from(data).subarray(0, 4), rest.at(-1));
    },
  },
});

stream.on('ready', () => {
  stream.write('abc中def');
  stream.end();
});

stream.on('close', () => {
  const actual = fs.readFileSync(file);
  fs.unlinkSync(file);
  console.log('expected:', Buffer.from('abc中def').toString('hex'));
  console.log('actual:  ', actual.toString('hex'), JSON.stringify(actual.toString()));
});

How often does it reproduce? Is there a required condition?

Always

What is the expected behavior? Why is that the expected behavior?

expected: 616263e4b8ad646566

What do you see instead?

actual:   616263e4e4b8ad646566 "abc�中def"

Additional information

No response

Activity

  1. added
    fsIssues and PRs related to file-system APIs and the fs module.
    on Nov 21, 2026
  2. jakecastelli commented on Nov 21, 2026

    @jakecastelli
    MemberAuthor

    Same happens with 2-byte characters, so accented Latin text is also affected.

    Using the repro above with café!

    expected: 636166c3a921
    actual:   636166c3c3a921 "caf�é!"
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    fsIssues and PRs related to file-system APIs and the fs module.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions