Crash report
What happened?
In Counter.update, _collections__count_elements_impl (called through _count_elements then _collections__count_elements) has a fast path for plain dicts that uses _PyDict_GetItem_KnownHash to obtain a borrowed reference to the current value. The current value is then incremented using PyNumber_Add (but after the dict lock is released).
|
oldval = _PyDict_GetItem_KnownHash(mapping, key, hash); |
|
if (oldval == NULL) { |
|
if (PyErr_Occurred()) |
|
goto done; |
|
if (_PyDict_SetItem_KnownHash(mapping, key, one, hash) < 0) |
|
goto done; |
|
} else { |
|
/* oldval is a borrowed reference. Keep it alive across |
|
PyNumber_Add(), which can execute arbitrary user code and |
|
mutate (or even clear) the underlying dict. */ |
|
Py_INCREF(oldval); |
|
newval = PyNumber_Add(oldval, one); |
|
Py_DECREF(oldval); |
|
if (newval == NULL) |
|
goto done; |
|
if (_PyDict_SetItem_KnownHash(mapping, key, newval, hash) < 0) |
|
goto done; |
When multiple threads call update on the same Counter concurrently, this can cause a user after free.
_PyDict_GetItem_KnownHash acquires the per-dict lock, looks up oldval, then releases the lock before returning the borrowed reference. Between that return and the next Py_INCREF(oldval), a second thread can:
- call
_PyDict_GetItem_KnownHash on the same key and get the same oldval
- do
Py_INCREF(oldval) / PyNumber_Add(oldval, one) / _PyDict_SetItem_KnownHash
- then cause the dict to decref and free
oldval (if the dict held the last reference).
- The first thread then executes
Py_INCREF(oldval) and PyNumber_Add(oldval, one) on an oldval that has been freed.
The following reproducer run under TSan shows many reports, but simply running it on a free-threaded build produces a hard crash.
import threading
from collections import Counter
def do() -> None:
c: Counter[str] = Counter()
data = ["x"] * 5000
def worker() -> None:
c.update(data)
threads = [threading.Thread(target=worker) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
for _ in range(50):
do()
Under free-threaded Python
$ ./python.exe ./new_bug_counter.py
zsh: segmentation fault ../cpython-nogil/python.exe ./new_bug_counter.py
Under TSAN (CPython main, free-threaded build):
Details
==================
WARNING: ThreadSanitizer: data race (pid=69202)
Read of size 8 at 0x0001120401e0 by thread T4:
#0 _PyFreeList_PopNoStats pycore_freelist.h:80 (python.exe:arm64+0x1000f2318)
#1 _PyFreeList_Pop pycore_freelist.h:89 (python.exe:arm64+0x1000f2318)
#2 _PyLong_FromMedium longobject.c:256 (python.exe:arm64+0x1000f2318)
#3 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
#4 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
#5 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
#6 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
#7 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
#8 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
#9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
#10 cfunction_vectorcall_FASTCALL methodobject.c:449 (python.exe:arm64+0x1001379dc)
#11 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
#12 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
#13 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
#14 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
#15 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#16 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#17 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#19 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#20 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#21 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
#22 context_run context.c:728 (python.exe:arm64+0x1002dce38)
#23 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
#24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
#25 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
#26 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
#27 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
#28 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#29 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#30 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#31 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#32 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#33 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#34 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
#35 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
#36 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
#37 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
#38 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
Previous atomic write of size 8 at 0x0001120401e0 by thread T2:
#0 _Py_atomic_store_ptr_relaxed pyatomic_gcc.h:509 (python.exe:arm64+0x1000ffcb0)
#1 _PyFreeList_Push pycore_freelist.h:56 (python.exe:arm64+0x1000ffcb0)
#2 _PyFreeList_Free pycore_freelist.h:69 (python.exe:arm64+0x1000ffcb0)
#3 long_dealloc longobject.c:3664 (python.exe:arm64+0x1000ffcb0)
#4 _Py_Dealloc object.c:3312 (python.exe:arm64+0x10013fd5c)
#5 _Py_DecRefSharedDebug object.c:426 (python.exe:arm64+0x10013ff34)
#6 _Py_DecRefShared object.c:433 (python.exe:arm64+0x10013ff34)
#7 Py_DECREF refcount.h:385 (python.exe:arm64+0x1003f5228)
#8 _collections__count_elements_impl _collectionsmodule.c:2603 (python.exe:arm64+0x1003f5228)
#9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f5228)
#10 cfunction_vectorcall_FASTCALL methodobject.c:449 (python.exe:arm64+0x1001379dc)
#11 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
#12 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
#13 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
#14 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
#15 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#16 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#17 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#19 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#20 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#21 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
#22 context_run context.c:728 (python.exe:arm64+0x1002dce38)
#23 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
#24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
#25 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
#26 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
#27 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
#28 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#29 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#30 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#31 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#32 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#33 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#34 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
#35 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
#36 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
#37 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
#38 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
Thread T4 (tid=18691961, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
#1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
#2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
#3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
#4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
#5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
#6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
#7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
#8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
#9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
#10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
#14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
#15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
#16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
#17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
#18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
#19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
#20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
#21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
#22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
#23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
#24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
#25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
#26 main python.c:15 (python.exe:arm64+0x100000a78)
Thread T2 (tid=18691959, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
#1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
#2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
#3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
#4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
#5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
#6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
#7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
#8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
#9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
#10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
#14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
#15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
#16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
#17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
#18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
#19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
#20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
#21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
#22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
#23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
#24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
#25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
#26 main python.c:15 (python.exe:arm64+0x100000a78)
SUMMARY: ThreadSanitizer: data race pycore_freelist.h:80 in _PyFreeList_PopNoStats
==================
==================
WARNING: ThreadSanitizer: data race (pid=69202)
Read of size 8 at 0x000116040540 by thread T8:
#0 _PyFreeList_PopNoStats pycore_freelist.h:80 (python.exe:arm64+0x1000f2318)
#1 _PyFreeList_Pop pycore_freelist.h:89 (python.exe:arm64+0x1000f2318)
#2 _PyLong_FromMedium longobject.c:256 (python.exe:arm64+0x1000f2318)
#3 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
#4 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
#5 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
#6 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
#7 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
#8 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
#9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
#10 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:2420 (python.exe:arm64+0x1002976c0)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#14 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#15 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#16 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#17 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
#19 context_run context.c:728 (python.exe:arm64+0x1002dce38)
#20 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
#21 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
#22 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#23 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#24 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#25 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#26 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#27 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#28 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
#29 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
#30 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
#31 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
#32 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
Previous atomic write of size 8 at 0x000116040540 by thread T7:
#0 _Py_atomic_store_ptr_relaxed pyatomic_gcc.h:509 (python.exe:arm64+0x1000ffcb0)
#1 _PyFreeList_Push pycore_freelist.h:56 (python.exe:arm64+0x1000ffcb0)
#2 _PyFreeList_Free pycore_freelist.h:69 (python.exe:arm64+0x1000ffcb0)
#3 long_dealloc longobject.c:3664 (python.exe:arm64+0x1000ffcb0)
#4 _Py_Dealloc object.c:3312 (python.exe:arm64+0x10013fd5c)
#5 _Py_DecRefSharedDebug object.c:426 (python.exe:arm64+0x10013ff34)
#6 _Py_DecRefShared object.c:433 (python.exe:arm64+0x10013ff34)
#7 Py_DECREF refcount.h:385 (python.exe:arm64+0x1003f5228)
#8 _collections__count_elements_impl _collectionsmodule.c:2603 (python.exe:arm64+0x1003f5228)
#9 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f5228)
#10 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:2420 (python.exe:arm64+0x1002976c0)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#14 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#15 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#16 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#17 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
#19 context_run context.c:728 (python.exe:arm64+0x1002dce38)
#20 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
#21 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
#22 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#23 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#24 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#25 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#26 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#27 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#28 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
#29 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
#30 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
#31 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
#32 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
Thread T8 (tid=18691990, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
#1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
#2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
#3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
#4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
#5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
#6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
#7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
#8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
#9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
#10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
#14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
#15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
#16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
#17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
#18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
#19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
#20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
#21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
#22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
#23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
#24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
#25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
#26 main python.c:15 (python.exe:arm64+0x100000a78)
Thread T7 (tid=18691989, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
#1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
#2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
#3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
#4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
#5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
#6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
#7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
#8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
#9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
#10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
#14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
#15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
#16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
#17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
#18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
#19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
#20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
#21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
#22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
#23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
#24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
#25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
#26 main python.c:15 (python.exe:arm64+0x100000a78)
SUMMARY: ThreadSanitizer: data race pycore_freelist.h:80 in _PyFreeList_PopNoStats
==================
ThreadSanitizer:DEADLYSIGNAL
==================
WARNING: ThreadSanitizer: data race (pid=69202)
Write of size 8 at 0x00011e04af40 by thread T5:
#0 _PyLong_SetSignAndDigitCount pycore_long.h:307 (python.exe:arm64+0x1000f242c)
#1 _PyLong_FromMedium longobject.c:267 (python.exe:arm64+0x1000f242c)
#2 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
#3 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
#4 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
#5 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
#6 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
#7 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
#8 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
#9 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
#10 _PyEval_EvalFrameDefault generated_cases.c.h:2420 (python.exe:arm64+0x1002976c0)
#11 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#12 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#13 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#14 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#15 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#16 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#17 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
#18 context_run context.c:728 (python.exe:arm64+0x1002dce38)
#19 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
#20 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
#21 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#22 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#23 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#25 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#26 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#27 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
#28 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
#29 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
#30 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
#31 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
Previous write of size 8 at 0x00011e04af40 by thread T6:
#0 _PyLong_SetSignAndDigitCount pycore_long.h:307 (python.exe:arm64+0x1000f242c)
#1 _PyLong_FromMedium longobject.c:267 (python.exe:arm64+0x1000f242c)
#2 _PyLong_FromSTwoDigits longobject.c:320 (python.exe:arm64+0x1000ffb7c)
#3 long_add longobject.c:3838 (python.exe:arm64+0x1000ffb7c)
#4 long_add_method longobject.c:3878 (python.exe:arm64+0x10010277c)
#5 binary_op1 abstract.c:974 (python.exe:arm64+0x10005edb4)
#6 PyNumber_Add abstract.c:1140 (python.exe:arm64+0x10005eb10)
#7 _collections__count_elements_impl _collectionsmodule.c:2602 (python.exe:arm64+0x1003f51c8)
#8 _collections__count_elements _collectionsmodule.c.h:593 (python.exe:arm64+0x1003f51c8)
#9 _Py_BuiltinCallFast_StackRef ceval.c:815 (python.exe:arm64+0x1002976c0)
#10 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x1002976c0)
#11 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#12 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#13 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#14 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#15 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#16 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#17 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
#18 context_run context.c:728 (python.exe:arm64+0x1002dce38)
#19 _PyCallMethodDescriptorFastWithKeywords_StackRef ceval.c:881 (python.exe:arm64+0x10029ac0c)
#20 _PyEval_EvalFrameDefault generated_cases.c.h:4022 (python.exe:arm64+0x10029ac0c)
#21 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
#22 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
#23 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
#24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
#25 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
#26 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
#27 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
#28 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
#29 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
#30 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
#31 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
Thread T5 (tid=18691987, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
#1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
#2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
#3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
#4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
#5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
#6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
#7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
#8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
#9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
#10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
#14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
#15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
#16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
#17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
#18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
#19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
#20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
#21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
#22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
#23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
#24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
#25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
#26 main python.c:15 (python.exe:arm64+0x100000a78)
Thread T6 (tid=18691988, running) created by main thread at:
#0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
#1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
#2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
#3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
#4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
#5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
#6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
#7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
#8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
#9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
#10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
#11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
#12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
#13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
#14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
#15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
#16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
#17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
#18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
#19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
#20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
#21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
#22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
#23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
#24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
#25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
#26 main python.c:15 (python.exe:arm64+0x100000a78)
SUMMARY: ThreadSanitizer: data race pycore_long.h:307 in _PyLong_SetSignAndDigitCount
==================
ThreadSanitizer:DEADLYSIGNAL
ThreadSanitizer: nested bug in the same thread, aborting.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
No response
Linked PRs
Crash report
What happened?
In
Counter.update,_collections__count_elements_impl(called through_count_elementsthen_collections__count_elements) has a fast path for plain dicts that uses_PyDict_GetItem_KnownHashto obtain a borrowed reference to the current value. The current value is then incremented usingPyNumber_Add(but after the dict lock is released).cpython/Modules/_collectionsmodule.c
Lines 2598 to 2614 in d986124
When multiple threads call
updateon the sameCounterconcurrently, this can cause a user after free._PyDict_GetItem_KnownHashacquires the per-dict lock, looks upoldval, then releases the lock before returning the borrowed reference. Between that return and the nextPy_INCREF(oldval), a second thread can:_PyDict_GetItem_KnownHashon the same key and get the sameoldvalPy_INCREF(oldval)/PyNumber_Add(oldval, one)/_PyDict_SetItem_KnownHasholdval(if the dict held the last reference).Py_INCREF(oldval)andPyNumber_Add(oldval, one)on anoldvalthat has been freed.The following reproducer run under TSan shows many reports, but simply running it on a free-threaded build produces a hard crash.
Under free-threaded Python
Under TSAN (CPython main, free-threaded build):
Details
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
No response
Linked PRs
Counter.update#151634