gh-152548: Add a test.support.isolated() decorator#152551
gh-152548: Add a test.support.isolated() decorator#152551serhiy-storchaka wants to merge 3 commits into
Conversation
Run a test in a fresh interpreter subprocess, so that it does not share global or interpreter state with the rest of the test run. It can decorate a test method (only that method runs in a subprocess) or a TestCase subclass (the whole class runs in one subprocess, with its setUpClass()/setUp()/tearDown()/ tearDownClass() running once there). Failures, errors and skips, including those of individual subtests, are reported for the test and show the original subprocess traceback. The subprocess inherits the parent's resource, memory and verbosity configuration, so that requires_resource(), bigmemtest() and similar behave the same in both processes. The test.support.running_isolated flag is true in the subprocess, so that fixtures can choose what to run there. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
560e882 to
2d5b380
Compare
Fixes the ruff F401 "imported but unused" lint failure for the re-exports. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documentation build overview
6 files changed ·
|
@isolated() always runs the test in a subprocess, so skip it in the parent process on platforms that lack subprocess support, the same way the tests it replaces were guarded by requires_subprocess(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
encukou
left a comment
There was a problem hiding this comment.
Adding some thoughts as I read this.
I'm not convinced that this feature is worth the complexity.
(I'm also not convinced that it's complete -- but for an internal testing helper, that's OK.)
| # Run a test method or class in an isolated subprocess. Implemented in a | ||
| # dedicated module so that its frames carry the __unittest marker and are | ||
| # stripped from reported tracebacks. | ||
| from test.support._isolation import isolated, running_isolated |
There was a problem hiding this comment.
Does this need to be support.isolated? Regrtest has been moving toward more granular helpers; this could be e.g. support.isolation.isolated.
If it needs to stay in this namespace, why are the imports not at the start of the file?
| def _remote(detail): | ||
| # Wrap the subprocess traceback the way concurrent.futures does, so it is | ||
| # clearly delimited when shown as the cause. | ||
| return _RemoteTraceback('\n"""\n%s"""' % detail) |
| if kind == 'skipped': | ||
| test.skipTest(detail) # the detail is the skip reason, not a traceback | ||
| elif kind == 'failure': | ||
| raise test.failureException('test failed in the subprocess') \ |
There was a problem hiding this comment.
Per PEP 8, prefer parentheses over the backslash.
| def _check_subprocess_support(): | ||
| # isolated() always runs the test in a subprocess, so skip (in the parent) | ||
| # on platforms that do not support spawning one. | ||
| import test.support as support | ||
| if not support.has_subprocess_support: | ||
| raise unittest.SkipTest('requires subprocess support') |
There was a problem hiding this comment.
Skipping tests on platforms without subprocsess support looks dangerous to me: it's not obvious at all that @isolated will do that, so I worry people will apply the decorator to tests that don't strictly need it.
Perhaps name it @run_in_subprocess, or require @skipIf(not has_subprocess_support in call sites?
There was a problem hiding this comment.
run_in_subprocess was the initial name. I changed it to reflect the purpose rather than mechanism. It also better fits unittest which already have IsolatedAsyncTestCase and in general do not use snake_case.
The final destination for this feature is unittest. I placed it in test.support first for two reasons. 1) to test and dogfeed it, 2) to simplify backports. But in future it will be moved to unittest.
I can move it to the test.support submodule for now.
There was a problem hiding this comment.
Making it failing instead of skipping if subprocess is not available will just require every use to be decorated with require_resources('subprocess'). This will just add boilerplate, which this feature purposes to eliminate.
Tests which run code in subprocesses already skipped if subprocess is not supported. This decorator will replace asser_python_ok with hand-written scripts in which you cannot even use assertEqual.
| orig_setUpClass = cls.setUpClass.__func__ | ||
| orig_tearDownClass = cls.tearDownClass.__func__ |
There was a problem hiding this comment.
What's the __func__ for? Could this use the bound classmethod?
| The test is skipped on platforms without subprocess support. | ||
| """ | ||
| def decorator(obj): | ||
| if isinstance(obj, type): |
There was a problem hiding this comment.
Should this be unittest.TestCase, as per the docstring?
| outcomes = [_outcome('failure', t, tb) for t, tb in result.failures] | ||
| outcomes += [_outcome('error', t, tb) for t, tb in result.errors] | ||
| outcomes += [_outcome('skipped', t, reason) for t, reason in result.skipped] | ||
|
|
There was a problem hiding this comment.
This will ignore any other outcomes, like expectedFailures.
Also, collectedDurations isn't preserved.
| Fixtures can test :data:`running_isolated` to decide what to run in each | ||
| process. |
There was a problem hiding this comment.
The added docs repeat this 3 times.
| :meth:`~unittest.TestCase.setUp` and :meth:`~unittest.TestCase.tearDown` | ||
| run both in the parent process (as usual) and in the subprocess around the |
There was a problem hiding this comment.
Same for all the other fixtures, right?
Add
test.support.isolated(), a decorator to run a test method or a wholeTestCasesubclass in a fresh interpreter subprocess, isolated from the rest of the test run.TestCasesubclass (the whole class runs in a single subprocess, sosetUpClass()/setUpModule()run once there).subTest()s — are reported for the corresponding test and show the original subprocess traceback. AsetUpClass()/setUpModule()failure or skip is reported for the whole class.-u), memory-limit (-M) and verbosity (-v) configuration, sorequires_resource(),requires(),bigmemtest()and similar behave the same in both processes.test.support.running_isolatedis true while running in the subprocess, so fixtures can choose what to run there.🤖 Generated with Claude Code