Repository navigation
Conversation
|
Review requested:
|
|
I'm not clear what the etiquette is for feedback on things like this from random Node.js users, so if this isn't the place for it please let me know / delete/edit this as appropriate. This seems really neat, but from reading the comparisons it got me wondering: How many of the benefits are inherent to what green-threads can do, and how many of them are things that could conceivably be added to the existing workers implementation? |
|
@jasnell does each green thread still have its own JS environment, like workers and worker threads? |
|
The benefits realized here are because they are not implemented the way existing workers are. Specifically, a green thread is not bound to a single physical thread the way a worker thread is. A green thread does not have an entire Node.js |
|
Is it possible to directly share objects between green threads? |
|
Share-ish. You can postMessage values to it that are structured cloned. There's also an rpc mechanism for accessing objects across the boundary. Otherwise, however, in this impl, no truly shared objects. |
|
OK - last question, I think :-) the RPC mechanism; does that allow two threads to mutate the same object at the same time? |
mcollina
left a comment
There was a problem hiding this comment.
rlgtm, this is a good first step and addition to Node
What I meant is: is it feasible to allow choosing to subtract those features from workers piecemeal? ie. Could the RPC thing be made available to workers? Could workers have a flag to get a limited environment? or the CPU/memory limits etc etc I appreciate the M:N aspect is specific to green threads, but naively the other features appear to be choices made while implementing this. |
Not without essentially reproducing what I have here already. It would end up being about the same but with more effort required to ensure we don't break anything. |
It's still serialized safely in the receiving side. There won't be thread safety issues |
|
This is really neat. I would be interested to see a comparison to building a pool of n worker threads and having them use a vm api to manage the isolates. |
|
@devsnek .... Here's the comparision for a pool of 8 workerthreads (the same size as the green thread pool), with tenants spread across them as tl;dr As one should expect, the pool of vm contexts wins on raw performance but loses the isolation between them; in particular, since they all share an Isolate, the DetailsSetup
Startup and memory (1000 tenants)Release build
Update: The memory measurement here in for green threads is off (too high) by about 1.3mb each. It's only about 3.86 MB per.... essentially the cost of an Isolate. Pointer-compression build
The Throughput (1000 tenants, 50k requests, 64 at a time)
Without When one of 100 tenants misbehavesThe
SummaryIt's a density versus isolation trade-off:
For many trusted, well-behaved tenants, a |
|
Just more on the approach here... looking at the isolation/sandboxing characterstics here... Assuming the V8 isolate boundary holds, so no V8 bug lets code escape its isolate... (most of this AI written with tweaks) Details
Workers aren't a sandboxA worker runs with the whole process's authority. The permission model applies to the whole process, so it can't restrict one worker. With
|
6f21b2c to
4b6fe8b
Compare
|
I pushed a few more commits here to optimize performance and memory further. Details are an AI generated summary... (slightly edited to remove slop) tl;dr -- Compared with one worker per tenant, green threads use about 4.5× less memory (1.36 MB vs 6.17 MB PSS each), start about 4× faster cold (~3,700/s vs ~980/s), and reach 65,000–100,000/s from a warm pool. Message round trips are the same, and so is CPU-bound throughput at equal parallelism (with DetailsUpdated comparison: green threads vs workers vs a pool of workers running
|
| starts/s | PSS per instance | PSS total | OS threads | |
|---|---|---|---|---|
| green thread | 3,739 | 1.36 MB | 1,364 MB | 8 (shared pool) |
| green thread, from warm pool | 80,778 | 1.38 MB | 1,377 MB | 8 |
| worker | 977 | 6.17 MB | 6,171 MB | 1,000 |
Pointer compression build:
| starts/s | PSS per instance | |
|---|---|---|
| green thread, own isolate group (default) | 2,556 | 1.35 MB |
| green thread, V8's default group | 3,926 | 0.95 MB |
green thread, one shared IsolateGroup |
4,170 | 0.95 MB |
| green thread, from warm pool | 101,977 | 1.36 MB |
| worker | 960 | 4.85 MB |
1000 isolated tenants, 50,000 requests
| all started | requests/s | PSS total | |
|---|---|---|---|
| green threads | 271 ms | 201,834 | 1,415 MB |
| workers | 1,025 ms | 157,475 | 6,216 MB |
Message round trips, one instance: the same, 49,187/s vs 49,221/s (20.3 µs each).
CPU-bound tasks, 2000 × fib(27)
| tasks/s | |
|---|---|
| 8 green threads | 5,023 |
| 8 workers | 5,052 |
88 green threads, NODE_GREEN_THREADS_POOL_SIZE=88 |
21,516 |
| 88 workers | 22,230 |
| one green thread per task (2000), pool of 8 | 4,151 |
At equal parallelism, CPU-bound throughput is the same. The pool size caps how
much JavaScript runs in parallel. It defaults to availableParallelism(), up
to 8, and NODE_GREEN_THREADS_POOL_SIZE raises it.
One of 100 tenants loops forever
| stopped by | stopped after | others served in 200 ms | |
|---|---|---|---|
| green threads | maxTaskCpuTimeMs: 50 |
52 ms | 46,332 |
| workers | terminate() after a 50 ms timer |
65 ms | 36,135 |
Green threads vs a pool of workers running vm contexts
(@devsnek's suggestion: N workers, tenants spread over them, one vm context
per tenant, requests run with script.runInContext().)
Startup and memory, 1000 tenants
| tenants/s | PSS per tenant | prepared beforehand | |
|---|---|---|---|
| green threads | 3,781 | 1.36 MB | – |
| green threads, warm pool | 64,951 | 1.39 MB | 394 ms |
| vm pool, 8 workers | 15,360 | 0.25 MB | – |
| vm pool, 8 workers, contexts created beforehand | 121,545 | 0.25 MB | 57 ms |
Pointer compression build: green threads 2,439/s at 1.36 MB, in a shared
IsolateGroup 3,851/s at 0.96 MB, warm pool 69,346/s; vm pool 16,242/s at
0.16 MB.
A vm context is much cheaper than an isolate. It shares the worker's heap,
GC and V8 per-isolate structures, so cold starts are ~4× faster and memory per
tenant is ~5–8× lower. That's the price of the isolation below. About 360 KB of
each green thread (Release) is V8's per-isolate stub caches alone, which a
vm context doesn't pay.
1000 tenants, 50,000 requests, 64 at a time
| requests/s | p50 | p99 | |
|---|---|---|---|
| green threads | 186,481 | 0.32 ms | 1.27 ms |
| vm pool, 8 workers | 190,804 | 0.33 ms | 0.72 ms |
| vm pool, 88 workers | 169,638 | 0.36 ms | 0.82 ms |
vm pool, 8 workers, with timeout |
80,415 | 0.63 ms | 2.59 ms |
vm pool, 88 workers, with timeout |
58,469 | 1.00 ms | 2.90 ms |
Without a timeout, throughput is the same. But a host has to pass
runInContext()'s timeout to stop tenants that loop, and that more than
halves the pool's throughput (it starts a watchdog per call). Green threads
enforce CPU limits without a per-call cost.
One of 100 tenants runs a synchronous infinite loop
| stopped after | tenants lost | others served in 300 ms | slowest other | |
|---|---|---|---|---|
| green threads | 52 ms | 1 (the bad one) | 59,499 | 3 ms |
vm pool, 8 workers, timeout: 50 |
51 ms | 0 | 18,414 | 52 ms |
One of 100 tenants runs an infinite microtask loop
| stopped after | tenants lost | others served in 300 ms | slowest other | |
|---|---|---|---|---|
| green threads | 54 ms | 1 (the bad one) | 61,776 | 3 ms |
vm pool, timeout: 50 |
101 ms (worker terminated) | 13 | 13,659 | 16 ms |
vm pool, timeout: 50, microtaskMode: 'afterEvaluate' |
51 ms | 0 | 18,216 | 52 ms |
timeout doesn't cover microtasks unless the context uses
microtaskMode: 'afterEvaluate'. Without it, the only way out is terminating
the worker, which takes every tenant on it along.
One of 100 tenants fills its heap
| stopped after | tenants lost | others still answering | |
|---|---|---|---|
| green threads | 312 ms | 1 (the bad one) | 99 |
| vm pool, 8 workers | 284 ms | 13 (whole worker) | 87 |
In a vm pool, a tenant that loops blocks the other tenants on its worker until
it's stopped (slowest other request: 52 ms vs 3 ms). A tenant that exhausts the
heap takes down every tenant on its worker, since they share one heap.
Which benefits are inherent to green threads (@glenjamin)
Inherent, from one isolate per tenant on a shared pool:
- A heap and GC per tenant: memory limits and out-of-memory only affect that tenant. A
vmcontext can't have its own heap limit. - CPU limits per tenant without a per-call cost: they cover microtasks, and other tenants keep running on other pool threads.
- One tenant can be terminated without affecting the others. In a vm pool, that requires terminating the worker.
- No Node.js API surface by default: a green thread only has what the host passes in.
vmcontexts are not a security boundary (escapable without V8 bugs), and workers have full Node.js. - Cheaper than workers: about 4.5× less memory (1.36 vs 6.17 MB) and 4× faster cold starts, because there's no OS thread, event loop or Node.js environment per instance.
Not inherent, or where the alternatives win:
- Memory and cold start per tenant: a
vmcontext is cheaper, because it shares its worker's isolate. Most of a green thread's remaining 1–1.4 MB is V8's per-isolate overhead, which only V8 changes could reduce. - Raw request throughput: the same as a vm pool without timeouts, and the same as workers for CPU-bound work at equal parallelism.
- Warm pools: they hide cold-start cost for green threads (65,000–100,000 starts/s), much like pre-creating
vmcontexts does for a pool.
This comment was marked as outdated.
This comment was marked as outdated.
The time origin of events was set when the runtime was bootstrapped. Green threads that start from a snapshot, or from the warm pool, got the time the snapshot was made, or the isolate was initialized, so the timeStamp of their events was too large. Set it when the green thread starts instead, before anything else runs in it. This also keeps the time out of snapshots, which must be reproducible. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads in isolate groups that only green isolates use started from a snapshot created at runtime, which took about 10 ms the first time a process needed it. Green threads in V8's default isolate group, the only kind in builds without pointer compression, like the official ones, ran the bootstrap of their runtime every time. Create the snapshot for green-only groups in node_mksnapshot, and embed it in the binary, in builds that support isolate groups. Add a context in which the runtime is bootstrapped to Node.js' own snapshot, for green threads in the default group. User-built snapshots don't have it, so green threads started from them still run the bootstrap. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads in V8's default isolate group, the only kind in builds without pointer compression, started from Node.js' snapshot, with a bootstrapped context added to it. Deserializing its isolate snapshot, Node.js' startup heap, which green threads don't use, took about 75% of the time it took to start them, and most of their memory. Isolates in a group must only share the read-only snapshot. So create a snapshot from V8's for them in node_mksnapshot, while the isolate that created Node.js' snapshot still exists, so that the read-only heap of the group is still Node.js'. The read-only snapshots then only differ in a counter that V8 doesn't use when the read-only heap already exists. Give the new snapshot Node.js' read-only snapshot, so that V8's checks pass, and only use it if the main isolate uses Node.js' snapshot. Remove the green context from Node.js' snapshot. In a release build, this makes starting a green thread take 1.1 ms rather than 3.6, and a green thread use 1.4 MB of memory rather than 3.7. Node.js' snapshot is back to its size before green threads. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
The pool that green threads run on has as many threads as the available parallelism, but at most 8. That limits how many green threads can run JavaScript at the same time: 256 green threads that each run for 20 ms per request handle 400 requests per second on a machine with 88 CPUs, and 4,205 with 88 pool threads. Add an environment variable that sets the number of pool threads, up to 1024, like UV_THREADPOOL_SIZE does for libuv's threadpool. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
SetIsolateCreateParamsForNode() queried the total and the constrained memory, which reads files on Linux, even when the isolate's heap already had a limit, and the result wasn't used. Workers call it twice, so every worker read them at least once for nothing. Only query them when the heap has no limit, and expose the query as GetMemoryForHeapDefaults(). Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads without a limit for the old generation got V8's defaults for the physical memory from NewIsolate(), which replaced the limit for the young generation, so maxYoungGenerationSizeMb on its own had no effect. Creating them also read /proc/meminfo and cgroup files each time, about 2% of the time it took to start them concurrently. Configure the defaults before setting the young generation limit, the way workers do, with the memory queried once per process. Report the heap spaces in __greenThreadInfo(), for tests. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Only green threads without memory limits, in the default isolate group, could use the warm pool, since those are fixed when an isolate is created. Hosts that run untrusted code usually set limits, so their green threads always started cold. Let setWarmPoolSize() take the isolateGroup and resourceLimits options of new GreenThread(), and keep a pool for each isolate group and set of memory limits it was called with. Green threads only use the pool for theirs, so they get exactly the limits they would get without it. CPU time limits are applied when green threads start, so they don't matter. With maxMemoryMb set, 1000 green threads, 64 at a time, start at about 60,000/s from a full pool rather than 4,100/s in a release build. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
V8 serializes external strings as ordinary strings unless their resources are external references. The snapshots of green threads only had the runtime's callbacks as external references, so they contained copies of the sources of its builtins, about 65 KB, which every green thread deserialized into its heap. Add Node.js' external references, which include the sources of all builtins, so that the snapshots refer to the sources in the binary. This makes each green thread use 60 KB less memory (1.34 MB rather than 1.40 in a release build) and the startup snapshots 65 KB smaller. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Format the code with clang-format, and declare the V8 names that src/node_green_thread.cc uses with using-declarations at the top of the file rather than qualifying them with v8::. Remove an unused declaration and keep all of the file's using-declarations sorted, as tools/checkimports.py requires. Regenerate doc/node.1, which was missing the updated description of NODE_GREEN_THREADS_POOL_SIZE. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
SnapshotBuilder::CreateSnapshot() built the startup snapshot of green threads in the default isolate group for every snapshot without a builder script, including the ones that embedders create through CommonEnvironmentSetup::CreateSnapshot(). Embedders have platforms of their own, so per_process::v8_platform has none and creating the snapshot crashed. Build it only for GenerateAsSource(), which is its only user. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads could only send messages to one receiver at a time. Add BroadcastChannel to green threads, for the names in the new broadcastScopes option of new GreenThread(). The channels are Node.js' own, process-wide, so messages reach every BroadcastChannel with the same name, in the main thread, in workers and in other green threads. Other names throw ERR_GREEN_THREAD_BROADCAST_SCOPE, so green threads can only use the channels that their host allows. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads that ignored termination and were abandoned kept their ports and BroadcastChannels open. The ports they were entangled with were never closed, and messages for them queued up until the process exited. Disentangle the ports of green threads when they are abandoned, and drop their messages. Since the green thread may still be running, the ports are only entangled, disentangled and used to post messages while a lock is held, which is never held while JavaScript runs. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
Green threads could only be terminated, which interrupts whatever they are doing. Add greenThread.close([options]), which calls the green thread's onclose function, and makes it exit with exit code 0 once nothing that it waits for is pending: timers, RPC calls in either direction, evaluations, its main module, and tasks queued before. Messages posted after close() are dropped, and evaluate() rejects with the new ERR_GREEN_THREAD_CLOSING. If the green thread hasn't exited within `timeoutMs`, 5000 by default, it is terminated. Assisted-by: Claude Opus 5.5 Signed-off-by: James M Snell <jasnell@gmail.com>
2b4c77c to
9b5284c
Compare
Functions, RpcTargets and AbortSignals in an RPC message were exported while the message was serialized. If a later value in it couldn't be cloned, such as a Symbol, the message wasn't sent, but its exports were kept, along with the listeners of its signals, so the objects stayed alive until the green thread exited. Serialize the message first, keeping only the indices of its RPC objects, and then make their descriptors with exportHostObjects(), which doesn't change any state. The objects are exported once nothing can fail anymore. Assisted-by: Claude Opus 5.5
The side that receives an AbortSignal through RPC only held it weakly, and Node.js doesn't keep signals alive for their listeners. A signal that only its listeners referred to could be collected before its abort arrived, which then went to a signal that no longer existed, so listeners waiting for it never ran. Keep imported signals until they abort, or the green thread exits. Signals that never abort are now kept until then, along with their originals. Assisted-by: Claude Opus 5.5
With the permission model, green threads could be started without any permission, while workers need --allow-worker. Add a permission of their own, rather than reusing --allow-worker: workers run a full Node.js environment, which is why --allow-worker can undo the permission model, but green threads only have what their host passes them. Without --allow-green-threads, new GreenThread() and setWarmPoolSize() throw ERR_ACCESS_DENIED. The scope is 'green-thread', for process.permission.has() and drop(), and its audit channel is node:permission-model:green-thread. Unlike --allow-worker, it emits no SecurityWarning. Assisted-by: Claude Opus 5.5
Green threads that spin until another green thread does something, for example on a SharedArrayBuffer, stall if they keep all pool threads busy, as the green thread that they wait for never runs. A green thread that keeps a pool thread busy also delays the timers of other green threads on that thread. setStallPolicy() makes the watchdog thread detect stalls: work that waits for a pool thread while each pool thread that could run it has run a single task for at least `timeoutMs`. It first adds up to `extraThreads` pool threads, if the work can run on any of them, and otherwise reports the green thread whose task has run the longest, to the 'stall' listeners of its GreenThread or to the policy's `onstall`, which can terminate it or give it more time. Stalls are reported one at a time, and `fallbackAction` applies if the host doesn't decide within the timeout, for example because its event loop is blocked. Green threads that are terminated for stalling emit ERR_GREEN_THREAD_STALLED. Refs: nodejs#67288 Assisted-by: Claude Opus 5.5
Recommend waiting without blocking, for example with a lock built on Atomics.waitAsync(), which can't stall the pool, and describe which deadlocks setStallPolicy() detects. Deadlocks that more pool threads wouldn't help with, such as two green threads that each hold a lock that the other one waits for, are not detected, and are stopped by maxTaskCpuTimeMs instead. Assisted-by: Claude Opus 5.5
|
Ok, I've updated to address the deadlock/stall issue and updated docs to match.
Currently, no. Native addons are intentionally not enabled.
These currently are not supported, intentionally. New capabilities will be added carefully and case-by-case with extremely careful benchmarking and testing. |
|
This PR is already quite large. Since it's experimental, any additional work will be pushed to follow ups. |
| ## RPC | ||
|
|
||
| A green thread can only use what its host passes it. Besides messages, the | ||
| host can give it an `env` object whose functions the green thread can call, |
There was a problem hiding this comment.
I would not call this env to avoid confusion with process.env or env in worker threads
There was a problem hiding this comment.
Yep... I just have no idea what alternative it should be. Please suggest colors for the bikeshed.
| put: (key, value) => { store.set(key, value); }, | ||
| }, | ||
| newCounter: () => new Counter(), | ||
| }, |
There was a problem hiding this comment.
these seems sync on this side but async on the other. Can you clarify?
There was a problem hiding this comment.
Yep... with the RPC mechanism, on the calling side everything is async. The call needs to be wait for the lock to be held.
| filename: 'file:///app/main.js', | ||
| modules: { | ||
| 'file:///app/scale.js': 'export const scale = (x) => x / 2;', | ||
| }, |
There was a problem hiding this comment.
This API is very awkward, and would result in differences in module resolution.
There was a problem hiding this comment.
Yep, needs to be designed better. Open to suggestions. Not 100% happy with this as is. I'd like the principle to remain the same tho.
| * Modules are taken from `modules`, by URL, and otherwise from the `load` | ||
| hook. | ||
|
|
||
| The hooks run on the host's thread, and may be asynchronous. They are called |
There was a problem hiding this comment.
will do, but will likely do so in a follow up PR. this one is massive already and I'm hesitant to add more to it.
| // Event, EventTarget, AbortSignal and AbortController for green threads. | ||
| // There is no DOM tree, so events are only ever dispatched to their target. | ||
| // Node.js' own implementation depends on too much of Node.js to run in a | ||
| // green thread. |
There was a problem hiding this comment.
I would really prefer to not have to duplicate this. Also: what about WPTs?
There was a problem hiding this comment.
As would I but the current implementation is rather... heavyweight. It brings in a lot more than what is strictly needed and bloats the implementation. This is precisely why this PR does not bring in any other core APIs like URL or TextEncoder. We need to rethink a bit how those are defined to provide an absolute minimal runtime for these.
There was a problem hiding this comment.
This could create a lot of edge cases between the two implementations of EventTarget (what happens if you dispatch an event from one in the other) and I suspect is a big footgun especially for llms.
It would be preferable to decompose the implementation (is it bloated to be compliant or just mistakes we made?)
| [`--experimental-green-threads`][] flag is used. | ||
|
|
||
| ```mjs | ||
| import { GreenThread } from 'node:green_threads'; |
There was a problem hiding this comment.
Why a new module? Why not
| import { GreenThread } from 'node:green_threads'; | |
| import { GreenThread } from 'node:worker_threads'; |
There was a problem hiding this comment.
That's one of the things to discuss I think. I intentionally didn't make it worker_threads yet because I wanted to avoid the potential confusion. These aren't worker threads so wasn't sure if it made sense there.
There was a problem hiding this comment.
Right, worth discussing indeed, I feel we should resist having two top-level modules which that much overlap, at least so the documentation ends up on the same page. Other alternatives to consider:
- add
node:worker_threads/green - move/alias
worker_threadstonode:threads/worker, and addnode:threads/green
There was a problem hiding this comment.
👆
As in #67157, the standard library should move toward namespacing common concepts under umbrella terms. node:threads/<thing> is the right approach.
There was a problem hiding this comment.
My concern there is that node:worker_threads is extremely entrenched at this point.
There was a problem hiding this comment.
And we still find uses of that all over ;-) ... I'm not against the re-org, we just need to accept that it's won't be super clean.
There was a problem hiding this comment.
So node:worker_threads/green?
benjamingr
left a comment
There was a problem hiding this comment.
RS as experimental let's ship this and get feedback!
| added: REPLACEME | ||
| --> | ||
|
|
||
| > Stability: 1 - Experimental |
There was a problem hiding this comment.
| > Stability: 1 - Experimental | |
| > Stability: 1.0 - Early development |
We should use the new experimental status:
|
This pull request has conflicts with its base branch, removing the |
This is an experiment... opening for feedback.This is ready for review.This implements green threads for Node.js. A green thread is a lightweight kind of worker. It has its own V8 isolate with no Node.js APIs. They run on a small pool of physical threads that they share, instead of each getting an OS thread and event loop like a worker thread. Each one gets work from the host through messages, RPC calls, or its module's exports, and every piece of work runs in an invocation context of its own. Each green thread can have limits on CPU time and memory, and the host can terminate it. This makes them cheap to start and to keep around in large numbers, and suited to running many small, untrusted pieces of code, such as plugins, per-tenant handlers, or user-supplied scripts.
For later, the plan is to add the following APIs (but not in this PR)
TextEncoder/TextDecoderURL/URLSearchParamsstructuredClone()Details
Compared with worker threads
Green threads have real advantages over workers, in specific areas:
isolate-group setting. A thousand tenants take about 1 GB instead of 6.7 GB.
1000, and serve about 15–20% more requests per second.
limits.
They don't win on the speed of a single message, and they can't spread heavy
computation across all cores, since the pool is capped at 8 threads.
Setup for this performance comparison
support isolate groups, so every green thread is in V8's default group.
thread is in an isolate group of its own.
kMaxPoolThreads).Each measurement runs in a fresh child process, so that memory numbers don't
include what earlier measurements left behind.
Kinds of green threads measured:
greengreen-defaultisolateGroup: 'default', V8's default isolate groupgreen-sharedIsolateGroupshared by all instancesgreen-warmMemory and startup
1000 idle instances, started at once, measured once each has sent a message.
isolateGroup: 'default')IsolateGroupworker carries a whole Node.js environment, an event loop and an OS thread
with its stack. A green thread is a V8 isolate with a small runtime.
are.
setWarmPoolSize(n)prepares isolates in advance, sostarting a green thread takes about 20 µs. Workers have no equivalent.
shared isolate group.
Speed
Message round trips
Sequential round trips with one instance.
Workers are about 5% faster per message.
CPU-bound tasks
2000 tasks of
fib(27), each executor taking the next task once it's done.Release build.
At equal parallelism, CPU-bound code runs at the same speed. Green threads
can't use more than 8 cores, though, so for heavy parallel computation
workers win clearly. The cap is a choice in this implementation, not
inherent, and could be made configurable.
Many tenants
1000 tenants, each with code and state of its own, serving 50,000 requests
spread over all of them, 64 at a time.
IsolateGroupGreen threads serve about 15–20% more requests, since 8 pool threads don't
compete for the CPU the way 1000 OS threads do.
Runaway code
One of 100 tenants runs into an endless loop, while the others keep serving.
ERR_GREEN_THREAD_CPU_LIMIT(maxTaskCpuTimeMs: 50)terminate()Both stop it. Green threads do it with a limit on CPU time, which the runtime
enforces. Workers need a timer by wall time on the host, which also stops
code that's only waiting.
What's easier to build: a plugin host
The same plugin host, built with each. It runs five plugins, which are ES
modules exporting
transform(text). Plugins can import a shared module, andcall back into the host for configuration and logging. Three of them
misbehave.
upper,reverse(well-behaved)spin(endless loop)ERR_GREEN_THREAD_CPU_LIMITafter 100 ms of CPU timehog(allocatesArrayBuffers)maxMemoryMblimitresourceLimitsonly cover the JS heap. Reached 91 MB within the 100 ms timeoutfail(throws with a code)code: 'ERR_PLUGIN'The green thread version is 43 lines:
The worker version is 155 lines, since it has to build what green threads
provide:
plugins are written to a temporary directory. Green threads take a
modulesbundle andresolve()/load()hooks.calls, and error forwarding, roughly what Comlink does. Green threads get
exportsandenvas RPC stubs.ArrayBufferlimit.Green threads have both.
require('fs'). The permission model applies to the whole process, so itcan't restrict a single worker. Green threads have no Node.js APIs at all:
they can only do what the host passes them. This is the biggest
qualitative difference.
When to use which
Use green threads for:
limits.
memory per instance matter.
Use workers for:
fs,net,child_process), or npmpackages that use them.
is capped at 8 threads.
Atomics.wait()and synchronous waits are disabled in greenthreads, since a blocked isolate would hold up a shared pool thread.
Caveats
f14fd2d7548,which doesn't have the ES modules and errors commits. These scenarios don't
use either.
run than the rest of its table. The first run used a setting that bypassed
the pool.
machines, the gap is likely larger.
a watchdog that abandons such isolates. Workers are affected by the same
bug when terminated.