gh-143543: Fix re-entrant use-after-free in itertools.groupby

[VanshAgarwal24036]

Fix a use-after-free in itertools.groupby when a user-defined eq
re-enters next(groupby) during key comparison.

groupby_next() compared borrowed references to currkey and tgtkey using
PyObject_RichCompareBool(). A re-entrant eq could advance the iterator,
replace and decref those keys, leaving the outer comparison accessing
freed memory.

The fix temporarily INCREFs both keys for the duration of the comparison,
preventing re-entrancy from invalidating them. A regression test is added.

• Issue: Use-after-free in itertools.groupby via re-entrant key comparison through __eq__ #143543

[VanshAgarwal24036]

Skip News

[picnixz]

Skip News

No. This is a bug fix that is impacting end users. In the future, please let triagers decide which labels to use when the bot doesn't do it automatically.

[VanshAgarwal24036]

Thanks for the clarification — I’ve added a NEWS entry documenting the user-visible crash fix.

[skirpichev]

@VanshAgarwal24036, please see https://devguide.python.org/core-team/committing/#how-to-add-a-news-entry on how to properly add news entry.

[skirpichev]

In order to keep the commit history intact, please avoid squashing or amending history and then force-pushing to the PR. Reviewers often want to look at individual commits. When the PR is merged, everything will be squashed into a single commit.

[skirpichev]

@VanshAgarwal24036, please avoid force pushes!

[VanshAgarwal24036]

Thanks for the clarification — understood. I won’t rewrite history or use force-pushes going forward and will only add new commits.

I’m still seeing bedevere/news failing in CI, and I want to make sure I’m following the expected workflow correctly. Would you prefer that I delete the current NEWS entry and re-add it using blurb add, or should I keep the existing file and adjust it further?

Please let me know the preferred approach and I’ll follow that.

[VanshAgarwal24036]

Thanks for the suggestion.
I’ve moved the comment before the declarations and expanded it to clarify the re-entrancy scenario and the need for local snapshots and strong references.

[encukou]

!buildbot Asan

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @encukou for commit b48d388 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F143738%2Fmerge

The command will test the builders whose names match following regular expression: Asan

The builders matched are:

• AMD64 Arch Linux Asan PR
• AMD64 Arch Linux Asan Debug PR
• x86-64 MacOS Intel ASAN NoGIL PR

[miss-islington-app]

Thanks @VanshAgarwal24036 for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[miss-islington-app]

Thanks @VanshAgarwal24036 for the PR, and @encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-144626 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-144627 is a backport of this pull request to the 3.13 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 CentOS9 NoGIL 3.x (tier-1) has failed when building commit a91b5c3.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1609/builds/4821) and take a look at the build logs.
4. Check if the failure is related to this commit (a91b5c3) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1609/builds/4821

Failed tests:

• test_profiling

Failed subtests:

• test_interrupt - test.test_multiprocessing_forkserver.test_processes.WithProcessesTestProcess.test_interrupt

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/multiprocessing/process.py", line 320, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/_test_multiprocessing.py", line 525, in _sleep_some_event
    time.sleep(100)
    ~~~~~~~~~~^^^^^
KeyboardInterrupt
k


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/_test_multiprocessing.py", line 597, in test_interrupt
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/contextlib.py", line 85, in inner
    return func(*args, **kwds)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/_test_multiprocessing.py", line 578, in _kill_process
    self.assertEqual(join(), None)
                     ~~~~^^
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/_test_multiprocessing.py", line 251, in __call__
    return self.func(*args, **kwds)
           ~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/multiprocessing/process.py", line 156, in join
    res = self._popen.wait(timeout)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/multiprocessing/popen_fork.py", line 44, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/multiprocessing/popen_forkserver.py", line 65, in poll
    if not wait([self.sentinel], timeout):
           ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/multiprocessing/connection.py", line 1164, in wait
    ready = selector.select(timeout)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/selectors.py", line 398, in select
    fd_event_list = self._selector.poll(timeout)
  File "/home/buildbot/buildarea/3.x.itamaro-centos-aws.nogil/build/Lib/test/_test_multiprocessing.py", line 574, in handler
    raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-169' pid=2297385 parent=2294209 started daemon>