Introduce autoTestDiscoverOnSavePattern configuration option

[soceanainn]

Closes #24817

What this change does

Introduce autoTestDiscoverOnSavePattern configuration option to control autoTestDiscoverOnSaveEnabled behavior to only attempt test discovery refresh when files matching the specified glob pattern are saved.

Why this change

In a Python project we have with over 40K tests, developers definitely notice issues when pytest discovery is running whenever any file in the workspace is saved, despite all tests matching a very consistent pattern (./tests/**/test_*.py).

Other alternatives I considered

I did consider trying to match only the specific patterns used by unittest/pytest here. Given that would require parsing underlying configuration files / raw args in the test configuration for the workspace for both unittest and pytest (plus any other test runners supported in future) - I don't think that's going to be easy to maintain. Plus the addition / deletion of __init__.py files play a significant part in test discovery despite not being covered by the test configuration pattern - so this solution would be incomplete.

Another alternative would be to accept a parent directory and only include python files from that directory + subdirectories (using workspace directory as default value). This avoids introducing a glob configuration value, but feels very limiting.

[soceanainn]

@microsoft-github-policy-service agree

[eleanorjboyd]

Hi @soceanainn, apologies for the delay! Think this is a great add but will need to discuss with the team about the settings add- I will get back to you soon! Thanks

[soceanainn]

Thanks @eleanorjboyd !

I think it's missing 'skip package*.json' label (and any other relevant labels) which I don't have permissions to add, but otherwise should be good to go from a CI perspective.

It would be good to try to get this in ASAP, as VsCode is borderline unusable on our project with autoTestDiscoverOnSaveEnabled enabled (multiple devs reporting freezing on save when test discovery kicks in). We have something like ~40K tests in the project.

[eleanorjboyd]

yes will merge it today after getting additional approval from a coworker. Will also need to change the glob pattern to **/*.py as default since I think this will work for any python file regardless of if its in a subdirectory etc so Ill make that edit too

[eleanorjboyd]

@soceanainn now that this is merged it will be on the next version of the python extension pre-release (which will come out monday morning). If you are able to give it a try and let me know if it works as intended that would be great!