PYTHON-5615 Use uv python when python toolchain is not available

[blink1073]

PYTHON-5615

Summary

Currently when there is no python toolchain, we have to either use the MongoDB toolchain or the system python, which could be too old. See PYTHON-5604 for example with the ECS tests.

Using the uv provided python allows us to select the version of our choice, that is compatible with the current hardware, and comes with the Python build dependency (Python.h). The only downside to using the uv installed python is that it has patches on top of core CPython to make it relocatable.

Changes in this PR

Streamline the handling of the python binary used by UV, using either an explicit PYTHON_BINARY, the Python toolchain is found and a PYTHON_VERSION is given, a PYTHON_VERSION that is used directly as UV_PYTHON when no toolchain is found, or the default version (3.10).

Testing Plan

Manual Tests

Local Usage
Setup spawn host - rhel
Setup spawn host - windows

GitHub Actions

Verify the lock file change fails as expected - done

Evergreen Tests

https://spruce.mongodb.com/version/68f7ff042bb7760007c92662/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC

Default version

KMS GCP - UV_PYTHON=3.10
KMS Azure - UV_PYTHON=3.10
OIDC GCP - UV_PYTHON=3.10
OIDC Azure - UV_PYTHON=3.10
ECS - UV_PYTHON=3.10
Other Hosts - UV_PYTHON=3.10
Import Time - UV_PYTHON=3.10
Coverage - Using CPython 3.10.18

Explicit version

FIPS Host - UV_PYTHON=/usr/bin/python3.11
Toolchain - UV_PYTHON=C:/python/Python310/python.exe
Win32 Python 3.14t - UV_PYTHON=C:/python/32/Python314/python3.14t.exe
Perf - UV_PYTHON= (--python=3.10.11)
Search Index - UV_PYTHON=/opt/python/3.10/bin/python3
Atlas Connect - UV_PYTHON=/opt/python/3.14/bin/python3

Checklist

Checklist for Author

• Did you update the changelog (if necessary)? N/A
• Is the intention of the code captured in relevant tests? Yes
• If there are new TODOs, has a related JIRA ticket been created? N/A

Checklist for Reviewer {@primary_reviewer}

• Does the title of the PR reference a JIRA Ticket?
• Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
• Have you checked for spelling & grammar errors?
• Is all relevant documentation (README or docstring) updated?

Focus Areas for Reviewer (optional)

[blink1073]

I opened https://jira.mongodb.org/browse/PYTHON-5627 for the link failure

[NoahStapp]

Can we collapse down the different ways of setting a python version? We currently have three in PYTHON_BINARY, PYTHON_VERSION, UV_PYTHON and it's unclear when each is supposed to be used or why they all exist.

[NoahStapp]

Intentional removal of NO_EXT=1?

[blink1073]

Yup, it is an improvement

[blink1073]

Can we collapse down the different ways of setting a python version? We currently have three in PYTHON_BINARY, PYTHON_VERSION, UV_PYTHON and it's unclear when each is supposed to be used or why they all exist.

How about this:

• UV_PYTHON is for when we have an explicit path or we let UV install a Python
• TOOLCHAIN_VERSION is for when we want to use the version in the toolchain

[NoahStapp]

Can we collapse down the different ways of setting a python version? We currently have three in PYTHON_BINARY, PYTHON_VERSION, UV_PYTHON and it's unclear when each is supposed to be used or why they all exist.

How about this:

• UV_PYTHON is for when we have an explicit path or we let UV install a Python
• TOOLCHAIN_VERSION is for when we want to use the version in the toolchain

What's an example of a scenario that needs to use each?

[blink1073]

What's an example of a scenario that needs to use each?

• All of our tasks that use the toolchain would set TOOLCHAIN_VERSION, and we'd use that
• Tests that need a specific version (like PERF) or a known binary path (like FIPS) would use UV_PYTHON
• The fallback would be to use UV_PYTHON=3.10

[NoahStapp]

What's an example of a scenario that needs to use each?

• All of our tasks that use the toolchain would set TOOLCHAIN_VERSION, and we'd use that
• Tests that need a specific version (like PERF) or a known binary path (like FIPS) would use UV_PYTHON
• The fallback would be to use UV_PYTHON=3.10

By fallback do you mean the default? Or as a version of last resort in case the requested versions can't be found?

[blink1073]

The logic would be:

• If TOOLCHAIN_VERSION is set, look for it, and set UV_PYTHON to the path found. Fail if not found.
• If UV_PYTHON is not set, set it to 3.10.

[NoahStapp]

The logic would be:

• If TOOLCHAIN_VERSION is set, look for it, and set UV_PYTHON to the path found. Fail if not found.
• If UV_PYTHON is not set, set it to 3.10.

Got it. I like that approach, it's more explicit.

[blink1073]

Alternatively, we could explicitly set UV_PYTHON from the config generator and do the toolchain lookup there, and then just have a "UV_PYTHON=${UV_PYTHON:-3.10}" logic in the test runner.

[blink1073]

Alternatively, we could explicitly set UV_PYTHON from the config generator and do the toolchain lookup there, and then just have a "UV_PYTHON=${UV_PYTHON:-3.10}" logic in the test runner.

Oh wait, we can't do that because the tasks are platform-agnostic

[blink1073]

Updated patch build: https://spruce.mongodb.com/version/68f8f913739d3e000893a675/tasks?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC

[NoahStapp]

Can we remove PYTHON_BINARY and PYTHON_VERSION here now?

[blink1073]

Done