Skip to content

[css-syntax-3] Remove duplicate token stream creations - #11129

Merged
tabatkins merged 1 commit into
w3c:mainfrom
cdoublev:patch-3
Apr 28, 2026
Merged

[css-syntax-3] Remove duplicate token stream creations#11129
tabatkins merged 1 commit into
w3c:mainfrom
cdoublev:patch-3

Conversation

@cdoublev

@cdoublev cdoublev commented Nov 1, 2024

Copy link
Copy Markdown
Collaborator

tokenize maps a stream of code points into a stream of tokens:

To tokenize a stream of code points into a stream of CSS tokens input, repeatedly consume a token from input until an <EOF-token> is reached, pushing each of the returned tokens into a stream.

But a stream of tokens is also created with its result:

  1. [...], tokenize the result, then create a new token stream with those tokens as its tokens, and return it.
  1. Let tokens be the result of tokenizing [...]. Let input be a new token stream from tokens.

@fantasai
fantasai requested a review from tabatkins April 13, 2026 22:12
@tabatkins
tabatkins merged commit de3eb40 into w3c:main Apr 28, 2026
@cdoublev
cdoublev deleted the patch-3 branch April 29, 2026 02:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants