Repository navigation
stream: run a replaced write() in the flowing pipe - #66596
nigrosimone wants to merge 2 commits into
Conversation
pipe() calls dest.write(). The fast path of a flowing pipe hands the chunks of a source that pushes from _read() to dest._write() directly, so a write() replaced on the instance or overridden in a subclass no longer runs. A response that is a Writable, wrapped by compression middleware, sent a gzip body with uncompressed chunks in it. Take the fast path only when write() is Writable.prototype.write. Refs: nodejs#66182 Signed-off-by: Nigro Simone <nigro.simone@gmail.com>
|
Review requested:
|
|
@nigrosimone this seems good, why it's draft? |
|
@mcollina my local build is still running, I want to see the test pass there first. Ready in about an hour, or I can mark it ready now and let the CI run, if you prefer. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #66596 +/- ##
==========================================
+ Coverage 90.44% 90.71% +0.27%
==========================================
Files 791 801 +10
Lines 276562 280799 +4237
Branches 53126 54199 +1073
==========================================
+ Hits 250130 254729 +4599
+ Misses 16846 16251 -595
- Partials 9586 9819 +233
🚀 New features to boost your workflow:
|
…eased Signed-off-by: Nigro Simone <nigro.simone@gmail.com>
|
@mcollina it is ready now, the test passes also in my local build. Could you take a look again? |
| const kWriteFlowBlock = kObjectMode | kDestroyed | kErrored | kSync | | ||
| kEnding | kFinished | kWriting | kCorked | kBuffered | kEnded | | ||
| kNeedDrain | kWriteCb | kExpectWriteCb | kBufferProcessing | | ||
| kFinalCalled | kPrefinished | kOnFinished | kErrorEmitted; | ||
|
|
||
| function writeKnownBuffer(stream, chunk) { | ||
| const state = stream._writableState; | ||
| if (state == null || state.length !== 0 || !(chunk instanceof Buffer)) | ||
| if (state == null || state.length !== 0 || !(chunk instanceof Buffer) || | ||
| stream.write !== Writable.prototype.write) |
There was a problem hiding this comment.
One concern: the check compares against Writable.prototype.write as it is at call time, so patching the prototype itself is still bypassed:
const orig = Writable.prototype.write;
// This is how AMP tools usually hook streams
Writable.prototype.write = function(...args) {
// instrumentation
return orig.apply(this, args);
};Could we compare against the original function in module scope instead?
function write(chunk, encoding, cb) { ... }
Writable.prototype.write = write;
stream.write !== writeotherwise, good fix!
Signed-off-by: Nigro Simone <nigro.simone@gmail.com>
Commit Queue failedThe pull request was removed from the Commit Queue and labeled
commit-queue-failed
Full Commit Queue output |
pipe()callsdest.write(), but since #66182 a flowing pipe whose source pushes from_read()hands each chunk todest._write()directly. Awrite()replaced on the destination, or overridden in a subclass, no longer runs.Found on 26.11.1 with a framework whose response is a
stream.Writable: compression middleware replacesres.write(), and a file piped to the response came out as a gzip body with uncompressed chunks in it (Z_DATA_ERRORon the client). 26.10.0 is fine.Fix: take the fast path only when
write()isWritable.prototype.write.Test:
test/parallel/test-stream-pipe-replaced-write.js, onewrite()replaced on the instance and one in a subclass. It fails on 26.11.1 (1 call towrite()out of 64) and passes with the fix.cc @anonrig
Disclosure: I used Opus 5.5 (Max) as coding assistant