Merge branch 'main' into disable-perf-old-macos

Author: ned-deily

Doc/_static/profiling-sampling-visualization.css

@@ -33,6 +33,7 @@
     'issue_role',
     'lexers',
     'misc_news',
+    'profiling_trace',
     'pydoc_topics',
     'pyspecific',
     'sphinx.ext.coverage',

Doc/_static/profiling-sampling-visualization.js

@@ -297,8 +297,9 @@ clocks to track time.
    are called is undefined.
 
    The optional positional *args* will be passed to the callback when
-   it is called. If you want the callback to be called with keyword
-   arguments use :func:`functools.partial`.
+   it is called. Use :func:`functools.partial`
+   :ref:`to pass keyword arguments <asyncio-pass-keywords>` to
+   *callback*.
 
    An optional keyword-only *context* argument allows specifying a
    custom :class:`contextvars.Context` for the *callback* to run in.
@@ -1034,8 +1035,8 @@ Watching file descriptors
 .. method:: loop.add_writer(fd, callback, *args)
 
    Start monitoring the *fd* file descriptor for write availability and
-   invoke *callback* with the specified arguments once *fd* is available for
-   writing.
+   invoke *callback* with the specified arguments *args* once *fd* is
+   available for writing.
 
    Any preexisting callback registered for *fd* is cancelled and replaced by
    *callback*.
@@ -1308,7 +1309,8 @@ Unix signals
 
 .. method:: loop.add_signal_handler(signum, callback, *args)
 
-   Set *callback* as the handler for the *signum* signal.
+   Set *callback* as the handler for the *signum* signal,
+   passing *args* as positional arguments.
 
    The callback will be invoked by *loop*, along with other queued callbacks
    and runnable coroutines of that event loop. Unlike signal handlers
@@ -1343,7 +1345,8 @@ Executing code in thread or process pools
 
 .. awaitablemethod:: loop.run_in_executor(executor, func, *args)
 
-   Arrange for *func* to be called in the specified executor.
+   Arrange for *func* to be called in the specified executor
+   passing *args* as positional arguments.
 
    The *executor* argument should be an :class:`concurrent.futures.Executor`
    instance. The default executor is used if *executor* is ``None``.

Doc/conf.py

@@ -0,0 +1 @@
+<div id="sampling-profiler-viz" class="sampling-profiler-viz"></div>

Doc/library/asyncio-eventloop.rst

@@ -63,7 +63,7 @@ Choosing a profiler
 
 For most performance analysis, use the statistical profiler
 (:mod:`profiling.sampling`). It has minimal overhead, works for both development
-and production, and provides rich visualization options including flamegraphs,
+and production, and provides rich visualization options including flame graphs,
 heatmaps, GIL analysis, and more.
 
 Use the deterministic profiler (:mod:`profiling.tracing`) when you need **exact
@@ -81,7 +81,7 @@ The following table summarizes the key differences:
 +--------------------+------------------------------+------------------------------+
 | **Accuracy**       | Statistical estimate         | Exact call counts            |
 +--------------------+------------------------------+------------------------------+
-| **Output formats** | pstats, flamegraph, heatmap, | pstats                       |
+| **Output formats** | pstats, flame graph, heatmap,| pstats                       |
 |                    | gecko, collapsed             |                              |
 +--------------------+------------------------------+------------------------------+
 | **Profiling modes**| Wall-clock, CPU, GIL         | Wall-clock                   |
@@ -103,7 +103,7 @@ performance analysis tasks. Use it the same way you would use
 
 One of the main strengths of the sampling profiler is its variety of output
 formats. Beyond traditional pstats tables, it can generate interactive
-flamegraphs that visualize call hierarchies, line-level source heatmaps that
+flame graphs that visualize call hierarchies, line-level source heatmaps that
 show exactly where time is spent in your code, and Firefox Profiler output for
 timeline-based analysis.
 
@@ -157,7 +157,7 @@ command::
    python -m profiling.sampling run -m mypackage.module
 
 This runs the script under the profiler and prints a summary of where time was
-spent. For an interactive flamegraph::
+spent. For an interactive flame graph::
 
    python -m profiling.sampling run --flamegraph script.py
 
@@ -197,7 +197,7 @@ Understanding profile output
 
 Both profilers collect function-level statistics, though they present them in
 different formats. The sampling profiler offers multiple visualizations
-(flamegraphs, heatmaps, Firefox Profiler, pstats tables), while the
+(flame graphs, heatmaps, Firefox Profiler, pstats tables), while the
 deterministic profiler produces pstats-compatible output. Regardless of format,
 the underlying concepts are the same.
 
@@ -226,7 +226,7 @@ Key profiling concepts:
 
 **Caller/Callee relationships**
    Which functions called a given function (callers) and which functions it
-   called (callees). Flamegraphs visualize this as nested rectangles; pstats
+   called (callees). Flame graphs visualize this as nested rectangles; pstats
    can display it via the :meth:`~pstats.Stats.print_callers` and
    :meth:`~pstats.Stats.print_callees` methods.
 
@@ -248,7 +248,7 @@ continue to work without modification in all future Python versions.
 .. seealso::
 
    :mod:`profiling.sampling`
-      Statistical sampling profiler with flamegraphs, heatmaps, and GIL analysis.
+      Statistical sampling profiler with flame graphs, heatmaps, and GIL analysis.
       Recommended for most users.
 
    :mod:`profiling.tracing`

Doc/library/profiling-sampling-visualization.html

@@ -44,6 +44,23 @@ of samples over a profiling session, Tachyon constructs an accurate statistical
 estimate of where time is spent. The more samples collected, the
 more precise this estimate becomes.
 
+.. only:: html
+
+   The following interactive visualization demonstrates how sampling profiling
+   works. Press **Play** to watch a Python program execute, and observe how the
+   profiler periodically captures snapshots of the call stack. Adjust the
+   **sample interval** to see how sampling frequency affects the results.
+
+   .. raw:: html
+      :file: profiling-sampling-visualization.html
+
+.. only:: not html
+
+   .. note::
+
+      An interactive visualization of sampling profiling is available in the
+      HTML version of this documentation.
+
 
 How time is estimated
 ---------------------
@@ -354,7 +371,7 @@ Together, these determine how many samples will be collected during a profiling
 session.
 
 The :option:`--sampling-rate` option (:option:`-r`) sets how frequently samples
-are collected. The default is 1 kHz (10,000 samples per second)::
+are collected. The default is 1 kHz (1,000 samples per second)::
 
    python -m profiling.sampling run -r 20khz script.py
 

Doc/library/profiling.rst

@@ -2870,6 +2870,14 @@ expression support in the :mod:`re` module).
    You can use :meth:`str.maketrans` to create a translation map from
    character-to-character mappings in different formats.
 
+   The following example uses a mapping to replace ``'a'`` with ``'X'``,
+   ``'b'`` with ``'Y'``, and delete ``'c'``:
+
+   .. doctest::
+
+      >>> 'abc123'.translate({ord('a'): 'X', ord('b'): 'Y', ord('c'): None})
+      'XY123'
+
    See also the :mod:`codecs` module for a more flexible approach to custom
    character mappings.
 

Doc/library/profiling.sampling.rst

@@ -0,0 +1,166 @@
+"""
+Sphinx extension to generate profiler trace data during docs build.
+
+This extension executes a demo Python program with sys.settrace() to capture
+the execution trace and injects it into the profiling visualization JS file.
+"""
+
+import json
+import re
+import sys
+from io import StringIO
+from pathlib import Path
+
+from sphinx.errors import ExtensionError
+
+DEMO_SOURCE = """\
+def add(a, b):
+    return a + b
+
+def multiply(x, y):
+    result = 0
+    for i in range(y):
+        result = add(result, x)
+    return result
+
+def calculate(a, b):
+    sum_val = add(a, b)
+    product = multiply(a, b)
+    return sum_val + product
+
+def main():
+    result = calculate(3, 4)
+    print(f"Result: {result}")
+
+main()
+"""
+
+PLACEHOLDER = "/* PROFILING_TRACE_DATA */null"
+
+
+def generate_trace(source: str) -> list[dict]:
+    """
+    Execute the source code with tracing enabled and capture execution events.
+    """
+    trace_events = []
+    timestamp = [0]
+    timestamp_step = 50
+    tracing_active = [False]
+    pending_line = [None]
+
+    def tracer(frame, event, arg):
+        if frame.f_code.co_filename != '<demo>':
+            return tracer
+
+        func_name = frame.f_code.co_name
+        lineno = frame.f_lineno
+
+        if event == 'line' and not tracing_active[0]:
+            pending_line[0] = {'type': 'line', 'line': lineno}
+            return tracer
+
+        # Start tracing only once main() is called
+        if event == 'call' and func_name == 'main':
+            tracing_active[0] = True
+            # Emit the buffered line event (the main() call line) at ts=0
+            if pending_line[0]:
+                pending_line[0]['ts'] = 0
+                trace_events.append(pending_line[0])
+                pending_line[0] = None
+                timestamp[0] = timestamp_step
+
+        # Skip events until we've entered main()
+        if not tracing_active[0]:
+            return tracer
+
+        if event == 'call':
+            trace_events.append({
+                'type': 'call',
+                'func': func_name,
+                'line': lineno,
+                'ts': timestamp[0],
+            })
+        elif event == 'line':
+            trace_events.append({
+                'type': 'line',
+                'line': lineno,
+                'ts': timestamp[0],
+            })
+        elif event == 'return':
+            try:
+                value = arg if arg is None else repr(arg)
+            except Exception:
+                value = '<unprintable>'
+            trace_events.append({
+                'type': 'return',
+                'func': func_name,
+                'ts': timestamp[0],
+                'value': value,
+            })
+
+            if func_name == 'main':
+                tracing_active[0] = False
+
+        timestamp[0] += timestamp_step
+        return tracer
+
+    # Suppress print output during tracing
+    old_stdout = sys.stdout
+    sys.stdout = StringIO()
+
+    old_trace = sys.gettrace()
+    sys.settrace(tracer)
+    try:
+        code = compile(source, '<demo>', 'exec')
+        exec(code, {'__name__': '__main__'})
+    finally:
+        sys.settrace(old_trace)
+        sys.stdout = old_stdout
+
+    return trace_events
+
+
+def inject_trace(app, exception):
+    if exception:
+        return
+
+    js_path = (
+        Path(app.outdir) / '_static' / 'profiling-sampling-visualization.js'
+    )
+
+    if not js_path.exists():
+        return
+
+    trace = generate_trace(DEMO_SOURCE)
+
+    demo_data = {'source': DEMO_SOURCE.rstrip(), 'trace': trace, 'samples': []}
+
+    demo_json = json.dumps(demo_data, indent=2)
+    content = js_path.read_text(encoding='utf-8')
+
+    pattern = r"(const DEMO_SIMPLE\s*=\s*/\* PROFILING_TRACE_DATA \*/)[^;]+;"
+
+    if re.search(pattern, content):
+        content = re.sub(
+            pattern, lambda m: f"{m.group(1)} {demo_json};", content
+        )
+        js_path.write_text(content, encoding='utf-8')
+        print(
+            f"profiling_trace: Injected {len(trace)} trace events into {js_path.name}"
+        )
+    else:
+        raise ExtensionError(
+            f"profiling_trace: Placeholder pattern not found in {js_path.name}"
+        )
+
+
+def setup(app):
+    app.connect('build-finished', inject_trace)
+    app.add_js_file('profiling-sampling-visualization.js')
+    app.add_css_file('profiling-sampling-visualization.css')
+
+    return {
+        'version': '1.0',
+        'parallel_read_safe': True,
+        'parallel_write_safe': True,
+    }