gh-143055: Delegate to subiterators when unpacking in generator expressions#152550
Draft
graingert wants to merge 3 commits into
Draft
gh-143055: Delegate to subiterators when unpacking in generator expressions#152550graingert wants to merge 3 commits into
graingert wants to merge 3 commits into
Conversation
… expressions Unpacking a sub-iterable with `*` in a generator expression (PEP 798) now delegates to the sub-iterable using `yield from` semantics, so that values sent with send() and exceptions thrown with throw() are forwarded to the sub-iterator (and the sub-iterator's return value is discarded). This also works in asynchronous generator expressions. Since `*` unpacking is synchronous, the sub-iterable is a sync iterable, but it is delegated to from inside an async generator, so each produced value must be wrapped as an async-generator value -- including values produced in response to asend() and athrow(). A new internal `_PyAsyncGenUnpack` iterator (constructed via the new INTRINSIC_ASYNC_GEN_UNPACK intrinsic) wraps the sync iterable so that asend(), athrow() and aclose() are forwarded to the sub-iterator's send(), throw() and close(). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Documentation build overview
|
- async_gen_unpack_throw: validate arity with _PyArg_CheckPositional so that calling throw() with no arguments on the internal wrapper (reachable via ag_await while suspended at the delegation) raises TypeError instead of crashing the interpreter via gen_set_exception(NULL, ...). - Correct the whatsnew/NEWS/reference wording: an async generator expression's aclose() throws GeneratorExit into the sub-iterator via its throw(), rather than calling close(). - Add a regression test for the throw() arity check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nalyzer Add the new static type to Tools/c-analyzer/cpython/globals-to-fix.tsv alongside the other genobject static types, so the check-c-globals CI step passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unpacking a sub-iterable with
*in a generator expression (PEP 798) now delegates to the sub-iterable usingyield fromsemantics, rather than re-yielding each value with a manual loop.Sync generator expressions
(*sub for sub in subs)now forwardssend()andthrow()to the sub-iterator currently being unpacked, and discards the sub-iterator's return value:Asynchronous generator expressions
*unpacking is synchronous (per PEP 448), so the sub-iterable is always a sync iterable. But it is delegated to from inside an async generator, where each yielded value must be wrapped as an async-generator value — otherwise the async-generator machinery treats it as an awaited value. The throw path is the subtle case:_gen_throwforwards throws to the sub in the runtime and returns the result bypassing any bytecode-level wrap (and that same path is shared withawait/async forpassthrough, which must stay unwrapped).To handle this, a small internal iterator
_PyAsyncGenUnpackwraps the sync iterable so that every value it produces — via__next__,am_send, andthrow— is wrapped, while the return value and exceptions pass through unwrapped. It is constructed via a newINTRINSIC_ASYNC_GEN_UNPACKintrinsic emitted only in the async-generator case. As a resultasend(),athrow()andaclose()are forwarded to the sub-iterator'ssend(),throw()andclose().No
async yield from/ PEP 828 syntax is introduced; this is purely about delegating PEP 798 unpacking.Notes
PYC_MAGIC_NUMBERand the managed static type count are bumped (new helper type + changed codegen).Tests
New
unittest.TestCaseclasses inLib/test/test_unpack_ex.pycover syncsend/throw/closeforwarding and asyncasend/athrow/acloseforwarding. The async tests drive the generators by hand (noIsolatedAsyncioTestCase/asyncio), matching the convention intest_asyncgen.py.🤖 Generated with Claude Code