[pull] main from python:main - #1520
Merged
Merged
Conversation
* gh-151518: Avoid STW starvation of attaching threads Free-threaded stop-the-world pauses can otherwise starve a thread trying to reattach after it was suspended while detached. A tight manual gc.collect() loop can release and immediately request the next stop-the-world pause, repeatedly parking the detached thread before it can attach and make progress. Add a distinct _Py_THREAD_SUSPENDED_DETACHED state for tstates parked from DETACHED. tstate_wait_attach() marks an attach waiter only after observing that detached-origin suspended state, and park_detached_threads() skips only those active waiters on later stop-the-world passes. The ordinary successful tstate_try_attach() path remains the baseline CAS-only path. Teach the related stop-the-world paths about both suspended states, including start_the_world() and tstate_delete_common(). Keep the new wait flag after the existing hot free-threaded _PyThreadStateImpl fields so their offsets do not move. Add a free-threaded GC regression test that runs a subprocess with a tight gc.collect() worker and verifies the main thread can reattach after sleeping and stop the worker. * gh-151518: Track attach waiters in the thread state Represent active attach waiters with suspended-waiting and detached-waiting states. Preserve waiter registration when the world resumes, and keep passive detached threads immediately parkable. Restore the thread-state padding and retain the single-CAS uncontended attach path. * gh-151518: Simplify the GC fairness regression Use explicit warmup imports and a joined non-daemon collector. Run the child through script_helper with a faulthandler watchdog so a stalled attachment still fails with a traceback. Describe the bug as a fairness issue in the NEWS entry. * gh-151518: Exercise STW fairness directly * gh-151518: Preserve attach waiters when resuming BRC suspensions Share the waiter-aware resume transition between stop-the-world pauses and biased reference count merging so a concurrently registering waiter keeps its opportunity to attach. Restore the support import needed by the new upstream GC regression and align the thread-state constants. * gh-151518: Strengthen the STW attach fairness regression Hold each test pause for 10 ms and repeat pauses in C to reduce the opportunities for a waiting thread to run between pause requests. Clarify that parking rechecks the thread state before sleeping. * gh-151518: Restrict STW test pauses to free-threaded builds STW is a no-op in GIL builds. Avoid waiting in the helper there, since WASI cannot perform the blocking futex operation. * gh-151518: Wake only registered attach waiters Avoid a parking-lot lookup when resuming a suspended thread that has not tried to reattach. Use the old state from the successful CAS so concurrent waiter registration still receives a wakeup.
* gh-151464: exclude '<>' token from tokenize output Was: ``` $ echo '1 <> 2' | python -m tokenize 1,0-1,1: NUMBER '1' 1,2-1,4: OP '<>' 1,5-1,6: NUMBER '2' 1,6-1,7: NEWLINE '\n' 2,0-2,0: ENDMARKER '' ``` Now (regardless on ``__future__.barry_as_FLUFL`` import): ``` $ echo '1 <> 2' | ./python -m tokenize 1,0-1,1: NUMBER '1' 1,2-1,3: OP '<' 1,3-1,4: OP '>' 1,5-1,6: NUMBER '2' 1,6-1,7: NEWLINE '\n' 2,0-2,0: ENDMARKER '' ``` in accordance with the Grammar: https://docs.python.org/3.14/reference/lexical_analysis.html#operators-and-delimiters Also adds a custom error message for ``<>`` ("not equal" in Pascal and Python 2). * +1 * address review: lowercase and move invalid rule * address review: news * address review: move test_guido_as_bdfl_ineq_tokens() * address review: revert _PyTokenizer_From* changes * + revert unrelated change * address review: remove whatsnew entry
* test_cext now uses a single virtual environment for all tests. It makes the test way faster since creating a virtual environment is quite slow. * Add Py_TARGET_ABI3T test on C++. * Move macro tests from module exec function to test_macros(). * Call Py_BEGIN_CRITICAL_SECTION() on a fresh dictionary object, rather than on the module. * Rename some environment variables to always use "CPYTHON_TEST_" prefix. * Replace support.MS_WINDOWS with MSVC to be more explicit. * Move macro tests to test_macros(). * On macOS, use C++11 by default.
Py_Version type is "unsigned long", not "int". The bug is visible on big endian machines such as s390x.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )