Skip to content

fix: restart the pipeline worker after an idle connection closes - #551

Merged
Mzack9999 merged 1 commit into
projectdiscovery:mainfrom
kamilikamil:fix-pipeline-idle-restart
Oct 30, 2026
Merged

Mzack9999 merged 1 commit into
projectdiscovery:mainfrom
kamilikamil:fix-pipeline-idle-restart

Conversation

@kamilikamil

@kamilikamil kamilikamil commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #552

Problem

A PipelineClient request sent after the connection has been idle for MaxIdleConnDuration (10s by default) never returns.

When the writer stops on idle, worker() returns nil and the restart loop in init() checks whether the queues are empty:

  • Empty: the loop exits but leaves chR/chW set. init() only starts a worker when chR == nil, so the next Do queues its request with no worker to send it and blocks on <-w.done forever.
  • Not empty: the loop sets both channels to nil and keeps running. The queued requests are stranded in the old channels and their callers block forever.

The condition is inverted. There is also a race: Do called init() and then enqueued without holding chLock, so it could push into a channel whose worker had just decided to stop.

Fix

  • On stop, clear the channels so the next Do starts a new worker. If work is still queued, keep the channels and let the loop restart the worker.
  • Do and PendingRequests call init() and enqueue under chLock, the same lock the stop check holds, so a request can't land in a channel whose worker has exited.

Proof

TestPipelineClientReconnectsAfterIdle sends a request, waits past a 50 ms MaxIdleConnDuration, then sends another. Before the fix the second request hangs; after it, both succeed. go test -race ./... passes.

Downstream, this hangs nuclei pipeline: true templates whenever two sends are more than 10s apart.

I used an AI assistant while working on this.

@Mzack9999
Mzack9999 merged commit 27f44ee into projectdiscovery:main Oct 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipelined request hangs after the idle connection closes

2 participants