Skip to content

Commit 9c218d7

Browse files
committed
gh-125053: Document that ob_mutex must only be used via critical section API
Add a warning in the free-threading extensions howto explaining that PyObject.ob_mutex is reserved for the critical section API and must not be locked directly with PyMutex_Lock, as this can cause deadlocks. Extension authors who need their own lock should add a separate PyMutex field to their object struct. Also add an ob_mutex member entry under PyObject in the C API reference (Doc/c-api/structures.rst) with a cross-reference to the howto.
1 parent d736349 commit 9c218d7

File tree

3 files changed

+136
-0
lines changed

3 files changed

+136
-0
lines changed

Doc/c-api/structures.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,22 @@ under :ref:`reference counting <countingrefs>`.
4848
Do not use this field directly; use :c:macro:`Py_TYPE` and
4949
:c:func:`Py_SET_TYPE` instead.
5050

51+
.. c:member:: PyMutex ob_mutex
52+
53+
A per-object lock, present only in the :term:`free-threaded <free threading>`
54+
build (when :c:macro:`Py_GIL_DISABLED` is defined).
55+
56+
This field is **reserved for use by the critical section API**
57+
(:c:macro:`Py_BEGIN_CRITICAL_SECTION` / :c:macro:`Py_END_CRITICAL_SECTION`).
58+
Do **not** lock it directly with ``PyMutex_Lock``; doing so can cause
59+
deadlocks. If you need your own lock, add a separate :c:type:`PyMutex`
60+
field to your object struct.
61+
62+
See :ref:`per-object-locks` in the free-threading extension guide for
63+
details.
64+
65+
.. versionadded:: 3.13
66+
5167

5268
.. c:type:: PyVarObject
5369

Doc/howto/free-threading-extensions.rst

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -384,6 +384,55 @@ Important Considerations
384384
internal extension state, standard mutexes or other synchronization
385385
primitives might be more appropriate.
386386

387+
.. _per-object-locks:
388+
389+
Per-Object Locks (``ob_mutex``)
390+
...............................
391+
392+
In the free-threaded build, each Python object contains a :c:member:`~PyObject.ob_mutex`
393+
field of type :c:type:`PyMutex`. This mutex is **reserved for use by the
394+
critical section API** (:c:macro:`Py_BEGIN_CRITICAL_SECTION` /
395+
:c:macro:`Py_END_CRITICAL_SECTION`).
396+
397+
.. warning::
398+
399+
Do **not** lock ``ob_mutex`` directly with ``PyMutex_Lock(&obj->ob_mutex)``.
400+
Mixing direct ``PyMutex_Lock`` calls with the critical section API on the
401+
same mutex can cause deadlocks, because the critical section implementation
402+
may suspend and release its locks when contention is detected.
403+
404+
Even if your own code never uses critical sections on a particular object type,
405+
**CPython internals may use the critical section API on any Python object**.
406+
For example, the garbage collector or other interpreter internals may enter a
407+
critical section on your object. If your code holds ``ob_mutex`` directly at
408+
that point, a deadlock can occur.
409+
410+
If your extension type needs its own lock, add a separate :c:type:`PyMutex`
411+
field (or another synchronization primitive) to your object struct.
412+
:c:type:`PyMutex` is very lightweight — it is only one byte — so there is
413+
negligible cost to having an additional one::
414+
415+
/* WRONG — do not lock ob_mutex directly */
416+
PyMutex_Lock(&obj->ob_mutex);
417+
...
418+
PyMutex_Unlock(&obj->ob_mutex);
419+
420+
/* RIGHT — use critical sections for ob_mutex */
421+
Py_BEGIN_CRITICAL_SECTION(obj);
422+
...
423+
Py_END_CRITICAL_SECTION();
424+
425+
/* RIGHT — use your own mutex for your own state */
426+
typedef struct {
427+
PyObject_HEAD
428+
PyMutex my_mutex; /* separate lock for extension state */
429+
int my_data;
430+
} MyObject;
431+
432+
PyMutex_Lock(&self->my_mutex);
433+
self->my_data++;
434+
PyMutex_Unlock(&self->my_mutex);
435+
387436

388437
Building Extensions for the Free-Threaded Build
389438
===============================================
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
# gh-125053: Document `ob_mutex` usage warning
2+
3+
**Issue:** https://github.com/python/cpython/issues/125053
4+
**Branch:** `doc-ob-mutex-warning`
5+
6+
## Summary
7+
8+
Added documentation warning that `PyObject.ob_mutex` must only be used via the
9+
critical section API (`Py_BEGIN_CRITICAL_SECTION` / `Py_END_CRITICAL_SECTION`).
10+
Directly locking it with `PyMutex_Lock(&obj->ob_mutex)` can cause deadlocks
11+
because the critical section implementation may also lock the same mutex on any
12+
Python object. Extension authors who need their own lock should create a
13+
separate `PyMutex` field.
14+
15+
## Files changed
16+
17+
### 1. `Doc/howto/free-threading-extensions.rst`
18+
19+
**Location:** New subsection "Per-Object Locks (`ob_mutex`)" added after
20+
"Important Considerations" (after line 385), within the existing "Critical
21+
Sections" section.
22+
23+
**What was added (49 lines):**
24+
25+
- RST label `.. _per-object-locks:` for cross-referencing
26+
- Explanation that `ob_mutex` is a per-object `PyMutex` present in the
27+
free-threaded build, reserved for the critical section API
28+
- A `.. warning::` admonition against locking `ob_mutex` directly with
29+
`PyMutex_Lock`, explaining the deadlock risk
30+
- Note that CPython internals may use the critical section API on *any* Python
31+
object (e.g., garbage collector), so even types that don't use critical
32+
sections themselves are affected
33+
- Recommendation to add a separate `PyMutex` field (only 1 byte) for
34+
extension-specific locking needs
35+
- Code examples showing:
36+
- **WRONG:** direct `PyMutex_Lock(&obj->ob_mutex)`
37+
- **RIGHT:** `Py_BEGIN_CRITICAL_SECTION(obj)` / `Py_END_CRITICAL_SECTION()`
38+
- **RIGHT:** defining a custom `PyMutex my_mutex` field on your type struct
39+
40+
### 2. `Doc/c-api/structures.rst`
41+
42+
**Location:** New `.. c:member:: PyMutex ob_mutex` entry under `PyObject`,
43+
after the `ob_type` member (after line 49).
44+
45+
**What was added (16 lines):**
46+
47+
- Member documentation stating `ob_mutex` is a per-object lock present only in
48+
the free-threaded build (`Py_GIL_DISABLED`)
49+
- Warning that it is reserved for the critical section API and must not be
50+
locked directly
51+
- Cross-reference to `:ref:`per-object-locks`` in the free-threading howto
52+
- `.. versionadded:: 3.13` directive
53+
54+
## Verification against source code
55+
56+
All claims were verified against the actual CPython C headers:
57+
58+
| Claim | Verified at |
59+
|---|---|
60+
| `ob_mutex` is a `PyMutex` field on `PyObject` | `Include/object.h:162` |
61+
| Only present in free-threaded build (`Py_GIL_DISABLED`) | `Include/object.h:126,150` — two separate `_object` struct defs gated on `#elif !defined(Py_GIL_DISABLED)` / `#else` |
62+
| Critical sections lock `ob_mutex` | `Include/internal/pycore_critical_section.h:113``_PyCriticalSection_BeginMutex(tstate, c, &op->ob_mutex)` |
63+
| `PyMutex` is 1 byte | `Include/cpython/pylock.h:33-35``typedef struct PyMutex { uint8_t _bits; } PyMutex;` |
64+
| `ob_mutex` introduced in 3.13 | Commit `6dfb8fe0236` first appears in tag `v3.13.0a2` |
65+
| Deadlock risk from mixing direct lock with critical sections | Critical section design suspends/releases its locks on contention (`Include/cpython/critical_section.h:22-25`), but a direct `PyMutex_Lock` holder does not participate in this protocol |
66+
67+
## Build verification
68+
69+
- `make html` in `Doc/` completed with **no warnings or errors**
70+
- `ob_mutex` appears in both generated HTML pages
71+
- Cross-reference from `structures.html` to `free-threading-extensions.html#per-object-locks` renders correctly

0 commit comments

Comments
 (0)