Skip to content

test/read-before-exit: wait for deferred ring frees on -ENOMEM - #1663

Closed
7z0423arsenova wants to merge 1 commit into
axboe:masterfrom
7z0423arsenova:read-before-exit-enomem
Closed

7z0423arsenova wants to merge 1 commit into
axboe:masterfrom
7z0423arsenova:read-before-exit-enomem

Conversation

@7z0423arsenova

Copy link
Copy Markdown
Contributor

Run as a regular user, read-before-exit.t fails every time on 7.3:

queue_init: Cannot allocate memory
Test IOPOLL failed loop 1

Without CAP_IPC_LOCK, ring memory is accounted against RLIMIT_MEMLOCK (io_create_region() → __io_account_mem()). Every ring in this test exits with two timerfd reads still pending, so it is freed asynchronously by io_ring_exit_work(). The test creates over 2000 rings back to back, faster than those frees complete. With the default 8 MiB limit and about 8 KiB per ring, setup starts failing around ring 1000. The test passes as root because root rings are not accounted, which is presumably why CI doesn't see this.

Measurements with a small reproducer (create a ring, leave a read pending, exit it, repeat):

case result
no pending requests 20000 rings fine
pending reads, back to back -ENOMEM at ring ~991-996
pending reads, 1 ms sleep per ring 1200 rings fine
lower ulimit -l fails earlier
run again right after exit fine, nothing is leaked

This change makes the test sleep and retry for up to a second when ring setup fails with -ENOMEM as a regular user. Root and other errors go through t_create_ring_params() exactly as before. It also prints the loop index instead of the return value when the IOPOLL loop fails; the old message's "loop 1" was really ret.

Tested on 7.3.0-9-generic as a regular user: 3 out of 3 runs pass with the default limit (0.6 to 0.9 s each), and the test also passes with ulimit -l 512 (2.3 s).

This is likely also behind the read-before-exit.t failure in #1611. The ring setup failed: -12 reported there for read-inc-* and read-mshot may be the same accounting backlog spilling over from earlier tests. I kept this change scoped to one test; retrying in t_create_ring_params() itself would cover those too, if that's preferred.

Without CAP_IPC_LOCK, ring memory is accounted against RLIMIT_MEMLOCK.
Every ring in this test exits with two timerfd reads still pending, so
the kernel frees it asynchronously from the exit work. The test creates
over 2000 rings back to back, faster than those frees complete, and as
a regular user it runs into the limit: with the default 8 MiB limit,
ring setup fails with -ENOMEM after about 1000 rings:

  queue_init: Cannot allocate memory
  Test IOPOLL failed loop 1

The test passes when run as root, since root rings are not accounted.

When ring setup fails with -ENOMEM as a regular user, sleep and retry
for up to a second to let the deferred frees catch up. Also print the
loop index rather than the return value when the IOPOLL loop fails.

Link: axboe#1611
Signed-off-by: Arsenova <7z0423arsenova@proton.me>
@7z0423arsenova

Copy link
Copy Markdown
Contributor Author

Update: the root cause is a kernel regression, not expected behavior.
Since v6.14 ("io_uring: use region api for SQ"/"... for CQ"), the
SQ/CQ rings are charged to RLIMIT_MEMLOCK, which they had been exempt
from before. This is already fixed in for-next:

b59cabf9b4ac ("io_uring: do not charge the SQ/CQ rings to RLIMIT_MEMLOCK")

which is tagged for stable. So this test is doing its job by failing on
6.14 through 7.3 without that fix, and the retry here would hide it.

Two options:

  • drop this PR and let the test keep catching kernels that lack the fix
  • keep only the loop index fix in the IOPOLL error message

Happy to respin with just the message fix, or close this, whichever
you prefer. The same regression likely explains the read-before-exit
and "ring setup failed: -12" failures in #1611.

@axboe

axboe commented Oct 24, 2026

Copy link
Copy Markdown
Owner

drop this PR and let the test keep catching kernels that lack the fix

This, the whole point of the test is to catch a regression, and it did. We should change the test case to make it pass.

@7z0423arsenova

Copy link
Copy Markdown
Contributor Author

Thanks for the review, closing this PR!

@7z0423arsenova
7z0423arsenova deleted the read-before-exit-enomem branch October 25, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants