1. 13.2 Parsing HTML documents
      1. 13.2.1 Overview of the parsing model
      2. 13.2.2 Parse errors
      3. 13.2.3 The input byte stream
        1. 13.2.3.1 Parsing with a known character encoding
        2. 13.2.3.2 Determining the character encoding
        3. 13.2.3.3 Character encodings
        4. 13.2.3.4 Changing the encoding while parsing
        5. 13.2.3.5 Preprocessing the input stream
      4. 13.2.4 Parse state
        1. 13.2.4.1 The insertion mode
        2. 13.2.4.2 The stack of open elements
        3. 13.2.4.3 The list of active formatting elements
        4. 13.2.4.4 The element pointers
        5. 13.2.4.5 Other parsing state flags
      5. 13.2.5 Tokenization
        1. 13.2.5.1 Data state
        2. 13.2.5.2 RCDATA state
        3. 13.2.5.3 RAWTEXT state
        4. 13.2.5.4 Script data state
        5. 13.2.5.5 PLAINTEXT state
        6. 13.2.5.6 Tag open state
        7. 13.2.5.7 End tag open state
        8. 13.2.5.8 Tag name state
        9. 13.2.5.9 RCDATA less-than sign state
        10. 13.2.5.10 RCDATA end tag open state
        11. 13.2.5.11 RCDATA end tag name state
        12. 13.2.5.12 RAWTEXT less-than sign state
        13. 13.2.5.13 RAWTEXT end tag open state
        14. 13.2.5.14 RAWTEXT end tag name state
        15. 13.2.5.15 Script data less-than sign state
        16. 13.2.5.16 Script data end tag open state
        17. 13.2.5.17 Script data end tag name state
        18. 13.2.5.18 Script data escape start state
        19. 13.2.5.19 Script data escape start dash state
        20. 13.2.5.20 Script data escaped state
        21. 13.2.5.21 Script data escaped dash state
        22. 13.2.5.22 Script data escaped dash dash state
        23. 13.2.5.23 Script data escaped less-than sign state
        24. 13.2.5.24 Script data escaped end tag open state
        25. 13.2.5.25 Script data escaped end tag name state
        26. 13.2.5.26 Script data double escape start state
        27. 13.2.5.27 Script data double escaped state
        28. 13.2.5.28 Script data double escaped dash state
        29. 13.2.5.29 Script data double escaped dash dash state
        30. 13.2.5.30 Script data double escaped less-than sign state
        31. 13.2.5.31 Script data double escape end state
        32. 13.2.5.32 Before attribute name state
        33. 13.2.5.33 Attribute name state
        34. 13.2.5.34 After attribute name state
        35. 13.2.5.35 Before attribute value state
        36. 13.2.5.36 Attribute value (double-quoted) state
        37. 13.2.5.37 Attribute value (single-quoted) state
        38. 13.2.5.38 Attribute value (unquoted) state
        39. 13.2.5.39 After attribute value (quoted) state
        40. 13.2.5.40 Self-closing start tag state
        41. 13.2.5.41 Bogus comment state
        42. 13.2.5.42 Markup declaration open state
        43. 13.2.5.43 Comment start state
        44. 13.2.5.44 Comment start dash state
        45. 13.2.5.45 Comment state
        46. 13.2.5.46 Comment less-than sign state
        47. 13.2.5.47 Comment less-than sign bang state
        48. 13.2.5.48 Comment less-than sign bang dash state
        49. 13.2.5.49 Comment less-than sign bang dash dash state
        50. 13.2.5.50 Comment end dash state
        51. 13.2.5.51 Comment end state
        52. 13.2.5.52 Comment end bang state
        53. 13.2.5.53 DOCTYPE state
        54. 13.2.5.54 Before DOCTYPE name state
        55. 13.2.5.55 DOCTYPE name state
        56. 13.2.5.56 After DOCTYPE name state
        57. 13.2.5.57 After DOCTYPE public keyword state
        58. 13.2.5.58 Before DOCTYPE public identifier state
        59. 13.2.5.59 DOCTYPE public identifier (double-quoted) state
        60. 13.2.5.60 DOCTYPE public identifier (single-quoted) state
        61. 13.2.5.61 After DOCTYPE public identifier state
        62. 13.2.5.62 Between DOCTYPE public and system identifiers state
        63. 13.2.5.63 After DOCTYPE system keyword state
        64. 13.2.5.64 Before DOCTYPE system identifier state
        65. 13.2.5.65 DOCTYPE system identifier (double-quoted) state
        66. 13.2.5.66 DOCTYPE system identifier (single-quoted) state
        67. 13.2.5.67 After DOCTYPE system identifier state
        68. 13.2.5.68 Bogus DOCTYPE state
        69. 13.2.5.69 CDATA section state
        70. 13.2.5.70 CDATA section bracket state
        71. 13.2.5.71 CDATA section end state
        72. 13.2.5.72 Character reference state
        73. 13.2.5.73 Named character reference state
        74. 13.2.5.74 Ambiguous ampersand state
        75. 13.2.5.75 Numeric character reference state
        76. 13.2.5.76 Hexadecimal character reference start state
        77. 13.2.5.77 Decimal character reference start state
        78. 13.2.5.78 Hexadecimal character reference state
        79. 13.2.5.79 Decimal character reference state
        80. 13.2.5.80 Numeric character reference end state
      6. 13.2.6 Tree construction
        1. 13.2.6.1 Creating and inserting nodes
        2. 13.2.6.2 Parsing elements that contain only text
        3. 13.2.6.3 Closing elements that have implied end tags
        4. 13.2.6.4 The rules for parsing tokens in HTML content
          1. 13.2.6.4.1 The "initial" insertion mode
          2. 13.2.6.4.2 The "before html" insertion mode
          3. 13.2.6.4.3 The "before head" insertion mode
          4. 13.2.6.4.4 The "in head" insertion mode
          5. 13.2.6.4.5 The "in head noscript" insertion mode
          6. 13.2.6.4.6 The "after head" insertion mode
          7. 13.2.6.4.7 The "in body" insertion mode
          8. 13.2.6.4.8 The "text" insertion mode
          9. 13.2.6.4.9 The "in table" insertion mode
          10. 13.2.6.4.10 The "in table text" insertion mode
          11. 13.2.6.4.11 The "in caption" insertion mode
          12. 13.2.6.4.12 The "in column group" insertion mode
          13. 13.2.6.4.13 The "in table body" insertion mode
          14. 13.2.6.4.14 The "in row" insertion mode
          15. 13.2.6.4.15 The "in cell" insertion mode
          16. 13.2.6.4.16 The "in template" insertion mode
          17. 13.2.6.4.17 The "after body" insertion mode
          18. 13.2.6.4.18 The "in frameset" insertion mode
          19. 13.2.6.4.19 The "after frameset" insertion mode
          20. 13.2.6.4.20 The "after after body" insertion mode
          21. 13.2.6.4.21 The "after after frameset" insertion mode
        5. 13.2.6.5 The rules for parsing tokens in foreign content
      7. 13.2.7 The end
      8. 13.2.8 Speculative HTML parsing
      9. 13.2.9 Coercing an HTML DOM into an infoset
      10. 13.2.10 An introduction to error handling and strange cases in the parser
        1. 13.2.10.1 Misnested tags: <b><i></b></i>
        2. 13.2.10.2 Misnested tags: <b><p></b></p>
        3. 13.2.10.3 Unexpected markup in tables
        4. 13.2.10.4 Scripts that modify the page as it is being parsed
        5. 13.2.10.5 The execution of scripts that are moving across multiple documents
        6. 13.2.10.6 Unclosed formatting elements
    2. 13.3 Serializing HTML fragments
    3. 13.4 Parsing HTML fragments

13.2 Parsing HTML documents

This section only applies to user agents, data mining tools, and conformance checkers.

The rules for parsing XML documents into DOM trees are covered by the next section, entitled "The XML syntax".

User agents must use the parsing rules described in this section to generate the DOM trees from text/html resources. Together, these rules define what is referred to as the HTML parser.

While the HTML syntax described in this specification bears a close resemblance to SGML and XML, it is a separate language with its own parsing rules.

Some earlier versions of HTML (in particular from HTML2 to HTML4) were based on SGML and used SGML parsing rules. However, few (if any) web browsers ever implemented true SGML parsing for HTML documents; the only user agents to strictly handle HTML as an SGML application have historically been validators. The resulting confusion — with validators claiming documents to have one representation while widely deployed web browsers interoperably implemented a different representation — has wasted decades of productivity. This version of HTML thus returns to a non-SGML basis.

For the purposes of conformance checkers, if a resource is determined to be in the HTML syntax, then it is an HTML document.

As stated in the terminology section, references to element types that do not explicitly specify a namespace always refer to elements in the HTML namespace. For example, if the spec talks about "a menu element", then that is an element with the local name "menu", the namespace "http://www.w3.org/1999/xhtml", and the interface HTMLMenuElement. Where possible, references to such elements are hyperlinked to their definition.

13.2.1 Overview of the parsing model

The input to the HTML parsing process consists of a stream of code points, which is passed through a tokenization stage followed by a tree construction stage. The output is a Document object.

Implementations that do not support scripting do not have to actually create a DOM Document object, but the DOM tree in such cases is still used as the model for the rest of the specification.

In the common case, the data handled by the tokenization stage comes from the network, but it can also come from script running in the user agent, e.g. using the document.write() API.

There is only one set of states for the tokenizer stage and the tree construction stage, but the tree construction stage is reentrant, meaning that while the tree construction stage is handling one token, the tokenizer might be resumed, causing further tokens to be emitted and processed before the first token's processing is complete.

In the following example, the tree construction stage will be called upon to handle a "p" start tag token while handling the "script" end tag token:

...
<script>
 document.write('<p>');
</script>
...

To handle these cases, parsers have a script nesting level, which must be initially set to zero, and a parser pause flag, which must be initially set to false.

13.2.2 Parse errors

This specification defines the parsing rules for HTML documents, whether they are syntactically correct or not. Certain points in the parsing algorithm are said to be parse errors. The error handling for parse errors is well-defined (that's the processing rules described throughout this specification), but user agents, while parsing an HTML document, may abort the parser at the first parse error that they encounter for which they do not wish to apply the rules described in this specification.

Conformance checkers must report at least one parse error condition to the user if one or more parse error conditions exist in the document and must not report parse error conditions if none exist in the document. Conformance checkers may report more than one parse error condition if more than one parse error condition exists in the document.

Parse errors are only errors with the syntax of HTML. In addition to checking for parse errors, conformance checkers will also verify that the document obeys all the other conformance requirements described in this specification.

Some parse errors have dedicated codes outlined in the table below that should be used by conformance checkers in reports.

Error descriptions in the table below are non-normative.

Code Description
abrupt-closing-of-empty-comment

This error occurs if the parser encounters an empty comment that is abruptly closed by a U+003E (>) code point (i.e., <!--> or <!--->). The parser behaves as if the comment is closed correctly.

abrupt-doctype-public-identifier

This error occurs if the parser encounters a U+003E (>) code point in the DOCTYPE public identifier (e.g., <!DOCTYPE html PUBLIC "foo>). In such a case, if the DOCTYPE is correctly placed as a document preamble, the parser sets the Document to quirks mode.

abrupt-doctype-system-identifier

This error occurs if the parser encounters a U+003E (>) code point in the DOCTYPE system identifier (e.g., <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "foo>). In such a case, if the DOCTYPE is correctly placed as a document preamble, the parser sets the Document to quirks mode.

absence-of-digits-in-numeric-character-reference

This error occurs if the parser encounters a numeric character reference that doesn't contain any digits (e.g., &#qux;). In this case the parser doesn't resolve the character reference.

cdata-in-html-content

This error occurs if the parser encounters a CDATA section outside of foreign content (SVG or MathML). The parser treats such CDATA sections (including leading "[CDATA[" and trailing "]]" strings) as comments.

character-reference-outside-unicode-range

This error occurs if the parser encounters a numeric character reference that references a code point that is greater than the valid Unicode range. The parser resolves such a character reference to a U+FFFD REPLACEMENT CHARACTER.

control-character-in-input-stream

This error occurs if the input stream contains a control code point that is not ASCII whitespace or U+0000 NULL. Such code points are parsed as-is and usually, where parsing rules don't apply any additional restrictions, make their way into the DOM.

control-character-reference

This error occurs if the parser encounters a numeric character reference that references a control code point that is not ASCII whitespace or is a U+000D CARRIAGE RETURN. The parser resolves such character references as-is except C1 control references that are replaced according to the numeric character reference end state.

duplicate-attribute

This error occurs if the parser encounters an attribute in a tag that already has an attribute with the same name. The parser ignores all such duplicate occurrences of the attribute.

end-tag-with-attributes

This error occurs if the parser encounters an end tag with attributes. Attributes in end tags are ignored and do not make their way into the DOM.

end-tag-with-trailing-solidus

This error occurs if the parser encounters an end tag that has a U+002F (/) code point right before the closing U+003E (>) code point (e.g., </div/>). Such a tag is treated as a regular end tag.

eof-before-tag-name

This error occurs if the parser encounters the end of the input stream where a tag name is expected. In this case the parser treats the beginning of a start tag (i.e., <) or an end tag (i.e., </) as text content.

eof-in-cdata

This error occurs if the parser encounters the end of the input stream in a CDATA section. The parser treats such CDATA sections as if they are closed immediately before the end of the input stream.

eof-in-comment

This error occurs if the parser encounters the end of the input stream in a comment. The parser treats such comments as if they are closed immediately before the end of the input stream.

eof-in-doctype

This error occurs if the parser encounters the end of the input stream in a DOCTYPE. In such a case, if the DOCTYPE is correctly placed as a document preamble, the parser sets the Document to quirks mode.

eof-in-script-html-comment-like-text

This error occurs if the parser encounters the end of the input stream in text that resembles an HTML comment inside script element content (e.g., <script><!-- foo).

Syntactic structures that resemble HTML comments in script elements are parsed as text content. They can be a part of a scripting language-specific syntactic structure or be treated as an HTML-like comment, if the scripting language supports them (e.g., parsing rules for HTML-like comments can be found in Annex B of the JavaScript specification). The common reason for this error is a violation of the restrictions for contents of script elements. [JAVASCRIPT]

eof-in-tag

This error occurs if the parser encounters the end of the input stream in a start tag or an end tag (e.g., <div id=). Such a tag is ignored.

incorrectly-closed-comment

This error occurs if the parser encounters a comment that is closed by the "--!>" code point sequence. The parser treats such comments as if they are correctly closed by the "-->" code point sequence.

incorrectly-opened-comment

This error occurs if the parser encounters the "<!" code point sequence that is not immediately followed by two U+002D (-) code points and that is not the start of a DOCTYPE or a CDATA section. All content that follows the "<!" code point sequence up to a U+003E (>) code point (if present) or to the end of the input stream is treated as a comment.

One possible cause of this error is using an XML markup declaration (e.g., <!ELEMENT br EMPTY>) in HTML.

invalid-character-sequence-after-doctype-name

This error occurs if the parser encounters any code point sequence other than "PUBLIC" and "SYSTEM" keywords after a DOCTYPE name. In such a case, the parser ignores any following public or system identifiers, and if the DOCTYPE is correctly placed as a document preamble, and if the parser cannot change the mode flag is false, sets the Document to quirks mode.

invalid-first-character-of-tag-name

This error occurs if the parser encounters a code point that is not an ASCII alpha where first code point of a start tag name or an end tag name is expected. If a start tag was expected such code point and a preceding U+003C (<) is treated as text content, and all content that follows is treated as markup. Whereas, if an end tag was expected, such code point and all content that follows up to a U+003E (>) code point (if present) or to the end of the input stream is treated as a comment.

For example, consider the following markup:

<42></42>

This will be parsed into:

While the first code point of a tag name is limited to an ASCII alpha, a wide range of code points (including ASCII digits) is allowed in subsequent positions.

missing-attribute-value

This error occurs if the parser encounters a U+003E (>) code point where an attribute value is expected (e.g., <div id=>). The parser treats the attribute as having an empty value.

missing-doctype-name

This error occurs if the parser encounters a DOCTYPE that is missing a name (e.g., <!DOCTYPE>). In such a case, if the DOCTYPE is correctly placed as a document preamble, the parser sets the Document to quirks mode.

missing-doctype-public-identifier

This error occurs if the parser encounters a U+003E (>) code point where start of the DOCTYPE public identifier is expected (e.g., <!DOCTYPE html PUBLIC >). In such a case, if the DOCTYPE is correctly placed as a document preamble, the parser sets the Document to quirks mode.

missing-doctype-system-identifier

This error occurs if the parser encounters a U+003E (>) code point where start of the DOCTYPE system identifier is expected (e.g., <!DOCTYPE html SYSTEM >). In such a case, if the DOCTYPE is correctly placed as a document preamble, the parser sets the Document to quirks mode.

missing-end-tag-name

This error occurs if the parser encounters a U+003E (>) code point where an end tag name is expected, i.e., </>. The parser ignores the whole "</>" code point sequence.

missing-quote-before-doctype-public-identifier

This error occurs if the parser encounters the DOCTYPE public identifier that is not preceded by a quote (e.g., <!DOCTYPE html PUBLIC -//W3C//DTD HTML 4.01//EN">). In such a case, the parser ignores the public identifier, and if the DOCTYPE is correctly placed as a document preamble, sets the Document to quirks mode.

missing-quote-before-doctype-system-identifier

This error occurs if the parser encounters the DOCTYPE system identifier that is not preceded by a quote (e.g., <!DOCTYPE html SYSTEM http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">). In such a case, the parser ignores the system identifier, and if the DOCTYPE is correctly placed as a document preamble, sets the Document to quirks mode.

missing-semicolon-after-character-reference

This error occurs if the parser encounters a character reference that is not terminated by a U+003B (;) code point. The parser behaves the same as if the character reference is terminated by the U+003B (;) code point.

Most named character references require a terminating U+003B (;) code point. Those that don't might get resolved as a longer named character reference in certain ambiguous scenarios.

For example, &notin will be parsed as "¬in", i.e., the same as if the input were &not;in, whereas &notin; will be parsed as "∉".

missing-whitespace-after-doctype-public-keyword

This error occurs if the parser encounters a DOCTYPE whose "PUBLIC" keyword and public identifier are not separated by ASCII whitespace. In this case the parser behaves as if ASCII whitespace is present.

missing-whitespace-after-doctype-system-keyword

This error occurs if the parser encounters a DOCTYPE whose "SYSTEM" keyword and system identifier are not separated by ASCII whitespace. In this case the parser behaves as if ASCII whitespace is present.

missing-whitespace-before-doctype-name

This error occurs if the parser encounters a DOCTYPE whose "DOCTYPE" keyword and name are not separated by ASCII whitespace. In this case the parser behaves as if ASCII whitespace is present.

missing-whitespace-between-attributes

This error occurs if the parser encounters attributes that are not separated by ASCII whitespace (e.g., <div id="foo"class="bar">). In this case the parser behaves as if ASCII whitespace is present.

missing-whitespace-between-doctype-public-and-system-identifiers

This error occurs if the parser encounters a DOCTYPE whose public and system identifiers are not separated by ASCII whitespace. In this case the parser behaves as if ASCII whitespace is present.

nested-comment

This error occurs if the parser encounters a nested comment (e.g., <!-- <!-- nested --> -->). Such a comment will be closed by the first occurring "-->" code point sequence and everything that follows will be treated as markup.

noncharacter-character-reference

This error occurs if the parser encounters a numeric character reference that references a noncharacter. The parser resolves such character references as-is.

noncharacter-in-input-stream

This error occurs if the input stream contains a noncharacter. Such code points are parsed as-is and usually, where parsing rules don't apply any additional restrictions, make their way into the DOM.

non-void-html-element-start-tag-with-trailing-solidus

This error occurs if the parser encounters a start tag for an element that is not in the list of void elements or is not a part of foreign content (i.e., not an SVG or MathML element) that has a U+002F (/) code point right before the closing U+003E (>) code point. The parser behaves as if the U+002F (/) is not present.

For example, consider the following markup:

<div/><span></span><span></span>

This will be parsed into:

The trailing U+002F (/) in a start tag name can be used only in foreign content to specify self-closing tags. (Self-closing tags don't exist in HTML.) It is also allowed for void elements, but doesn't have any effect in this case.

null-character-reference

This error occurs if the parser encounters a numeric character reference that references a U+0000 NULL code point. The parser resolves such character references to a U+FFFD REPLACEMENT CHARACTER.

surrogate-character-reference

This error occurs if the parser encounters a numeric character reference that references a surrogate. The parser resolves such character references to a U+FFFD REPLACEMENT CHARACTER.

surrogate-in-input-stream

This error occurs if the input stream contains a surrogate. Such code points are parsed as-is and usually, where parsing rules don't apply any additional restrictions, make their way into the DOM.

Surrogates can only find their way into the input stream via script APIs such as document.write().

unexpected-character-after-doctype-system-identifier

This error occurs if the parser encounters any code points other than ASCII whitespace or closing U+003E (>) after the DOCTYPE system identifier. The parser ignores these code points.

unexpected-character-in-attribute-name

This error occurs if the parser encounters a U+0022 ("), U+0027 ('), or U+003C (<) code point in an attribute name. The parser includes such code points in the attribute name.

Code points that trigger this error are usually a part of another syntactic construct and can be a sign of a typo around the attribute name.

For example, consider the following markup:

<div foo<div>

Due to a forgotten U+003E (>) code point after foo the parser treats this markup as a single div element with a "foo<div" attribute.

As another example of this error, consider the following markup:

<div id'bar'>

Due to a forgotten U+003D (=) code point between an attribute name and value the parser treats this markup as a div element with the attribute "id'bar'" that has an empty value.

unexpected-character-in-unquoted-attribute-value

This error occurs if the parser encounters a U+0022 ("), U+0027 ('), U+003C (<), U+003D (=), or U+0060 (`) code point in an unquoted attribute value. The parser includes such code points in the attribute value.

Code points that trigger this error are usually a part of another syntactic construct and can be a sign of a typo around the attribute value.

U+0060 (`) is in the list of code points that trigger this error because certain legacy user agents treat it as a quote.

For example, consider the following markup:

<div foo=b'ar'>

Due to a misplaced U+0027 (') code point the parser sets the value of the "foo" attribute to "b'ar'".

unexpected-equals-sign-before-attribute-name

This error occurs if the parser encounters a U+003D (=) code point before an attribute name. In this case the parser treats U+003D (=) as the first code point of the attribute name.

The common reason for this error is a forgotten attribute name.

For example, consider the following markup:

<div foo="bar" ="baz">

Due to a forgotten attribute name the parser treats this markup as a div element with two attributes: a "foo" attribute with a "bar" value and a "="baz"" attribute with an empty value.

unexpected-null-character

This error occurs if the parser encounters a U+0000 NULL code point in the input stream in certain positions. In general, such code points are either ignored or, for security reasons, replaced with a U+FFFD REPLACEMENT CHARACTER.

unexpected-question-mark-instead-of-tag-name

This error occurs if the parser encounters a U+003F (?) code point where first code point of a start tag name is expected. The U+003F (?) and all content that follows up to a U+003E (>) code point (if present) or to the end of the input stream is treated as a comment.

For example, consider the following markup:

<?xml-stylesheet type="text/css" href="style.css"?>

This will be parsed into:

The common reason for this error is an XML processing instruction (e.g., <?xml-stylesheet type="text/css" href="style.css"?>) or an XML declaration (e.g., <?xml version="1.0" encoding="UTF-8"?>) being used in HTML.

unexpected-solidus-in-tag

This error occurs if the parser encounters a U+002F (/) code point that is not a part of a quoted attribute value and not immediately followed by a U+003E (>) code point in a tag (e.g., <div / id="foo">). In this case the parser behaves as if it encountered ASCII whitespace.

unknown-named-character-reference

This error occurs if the parser encounters an ambiguous ampersand. In this case the parser doesn't resolve the character reference.

13.2.3 The input byte stream

The stream of code points that comprises the input to the tokenization stage will be initially seen by the user agent as a stream of bytes (typically coming over the network or from the local file system). The bytes encode the actual characters according to a particular character encoding, which the user agent uses to decode the bytes into characters.

For XML documents, the algorithm user agents are required to use to determine the character encoding is given by XML. This section does not apply to XML documents. [XML]

Usually, the encoding sniffing algorithm defined below is used to determine the character encoding.

Given a character encoding, the bytes in the input byte stream must be converted to characters for the tokenizer's input stream, by passing the input byte stream and character encoding to decode.

A leading Byte Order Mark (BOM) causes the character encoding argument to be ignored and will itself be skipped.

Bytes or sequences of bytes in the original byte stream that did not conform to the Encoding standard (e.g. invalid UTF-8 byte sequences in a UTF-8 input byte stream) are errors that conformance checkers are expected to report. [ENCODING]

The decoder algorithms describe how to handle invalid input; for security reasons, it is imperative that those rules be followed precisely. Differences in how invalid byte sequences are handled can result in, amongst other problems, script injection vulnerabilities ("XSS").

When the HTML parser is decoding an input byte stream, it uses a character encoding and a confidence. The confidence is either tentative, certain, or irrelevant. The encoding used, and whether the confidence in that encoding is tentative or certain, is used during the parsing to determine whether to change the encoding. If no encoding is necessary, e.g. because the parser is operating on a Unicode stream and doesn't have to use a character encoding at all, then the confidence is irrelevant.

Some algorithms feed the parser by directly adding characters to the input stream rather than adding bytes to the input byte stream.

13.2.3.1 Parsing with a known character encoding

When the HTML parser is to operate on an input byte stream that has a known definite encoding, then the character encoding is that encoding and the confidence is certain.

13.2.3.2 Determining the character encoding

In some cases, it might be impractical to unambiguously determine the encoding before parsing the document. Because of this, this specification provides for a two-pass mechanism with an optional pre-scan. Implementations are allowed, as described below, to apply a simplified parsing algorithm to whatever bytes they have available before beginning to parse the document. Then, the real parser is started, using a tentative encoding derived from this pre-parse and other out-of-band metadata. If, while the document is being loaded, the user agent discovers a character encoding declaration that conflicts with this information, then the parser can get reinvoked to perform a parse of the document with the real encoding.

User agents must use the following algorithm, called the encoding sniffing algorithm, to determine the character encoding to use when decoding a document in the first pass. This algorithm takes as input any out-of-band metadata available to the user agent (e.g. the Content-Type metadata of the document) and all the bytes available so far, and returns a character encoding and a confidence that is either tentative or certain.

  1. If the result of BOM sniffing is an encoding, return that encoding with confidence certain.

    Although the decode algorithm will itself change the encoding to use based on the presence of a byte order mark, this algorithm sniffs the BOM as well in order to set the correct document's character encoding and confidence.

  2. If the user has explicitly instructed the user agent to override the document's character encoding with a specific encoding, optionally return that encoding with the confidence certain.

    Typically, user agents remember such user requests across sessions, and in some cases apply them to documents in iframes as well.

  3. The user agent may wait for more bytes of the resource to be available, either in this step or at any later step in this algorithm. For instance, a user agent might wait 500ms or 1024 bytes, whichever came first. In general preparsing the source to find the encoding improves performance, as it reduces the need to throw away the data structures used when parsing upon finding the encoding information. However, if the user agent delays too long to obtain data to determine the encoding, then the cost of the delay could outweigh any performance improvements from the preparse.

    The authoring conformance requirements for character encoding declarations limit them to only appearing in the first 1024 bytes. User agents are therefore encouraged to use the prescan algorithm below (as invoked by these steps) on the first 1024 bytes, but not to stall beyond that.

  4. If the transport layer specifies a character encoding, and it is supported, return that encoding with the confidence certain.

  5. Optionally, prescan the byte stream to determine its encoding, with the end condition being when the user agent decides that scanning further bytes would not be efficient. User agents are encouraged to only prescan the first 1024 bytes. User agents may decide that scanning any bytes is not efficient, in which case these substeps are entirely skipped.

    The aforementioned algorithm returns either a character encoding or failure. If it returns a character encoding, then return the same encoding, with confidence tentative.

  6. If the HTML parser for which this algorithm is being run is associated with a Document d whose container document is non-null, then:

    1. Let parentDocument be d's container document.

    2. If parentDocument's origin is same origin with d's origin and parentDocument's character encoding is not UTF-16BE/LE, then return parentDocument's character encoding, with the confidence tentative.

  7. Otherwise, if the user agent has information on the likely encoding for this page, e.g. based on the encoding of the page when it was last visited, then return that encoding, with the confidence tentative.

  8. The user agent may attempt to autodetect the character encoding from applying frequency analysis or other algorithms to the data stream. Such algorithms may use information about the resource other than the resource's contents, including the address of the resource. If autodetection succeeds in determining a character encoding, and that encoding is a supported encoding, then return that encoding, with the confidence tentative. [UNIVCHARDET]

    User agents are generally discouraged from attempting to autodetect encodings for resources obtained over the network, since doing so involves inherently non-interoperable heuristics. Attempting to detect encodings based on an HTML document's preamble is especially tricky since HTML markup typically uses only ASCII characters, and HTML documents tend to begin with a lot of markup rather than with text content.

    The UTF-8 encoding has a highly detectable bit pattern. Files from the local file system that contain bytes with values greater than 0x7F which match the UTF-8 pattern are very likely to be UTF-8, while documents with byte sequences that do not match it are very likely not. When a user agent can examine the whole file, rather than just the preamble, detecting for UTF-8 specifically can be especially effective. [PPUTF8] [UTF8DET]

  9. Otherwise, return an implementation-defined or user-specified default character encoding, with the confidence tentative.

    In controlled environments or in environments where the encoding of documents can be prescribed (for example, for user agents intended for dedicated use in new networks), the comprehensive UTF-8 encoding is suggested.

    In other environments, the default encoding is typically dependent on the user's locale (an approximation of the languages, and thus often encodings, of the pages that the user is likely to frequent). The following table gives suggested defaults based on the user's locale, for compatibility with legacy content. Locales are identified by BCP 47 language tags. [BCP47] [ENCODING]

    Locale language Suggested default encoding
    ar Arabic windows-1256
    az Azeri windows-1254
    ba Bashkir windows-1251
    be Belarusian windows-1251
    bg Bulgarian windows-1251
    cs Czech windows-1250
    el Greek ISO-8859-7
    et Estonian windows-1257
    fa Persian windows-1256
    he Hebrew windows-1255
    hr Croatian windows-1250
    hu Hungarian ISO-8859-2
    ja Japanese Shift_JIS
    kk Kazakh windows-1251
    ko Korean EUC-KR
    ku Kurdish windows-1254
    ky Kyrgyz windows-1251
    lt Lithuanian windows-1257
    lv Latvian windows-1257
    mk Macedonian windows-1251
    pl Polish ISO-8859-2
    ru Russian windows-1251
    sah Yakut windows-1251
    sk Slovak windows-1250
    sl Slovenian ISO-8859-2
    sr Serbian windows-1251
    tg Tajik windows-1251
    th Thai windows-874
    tr Turkish windows-1254
    tt Tatar windows-1251
    uk Ukrainian windows-1251
    vi Vietnamese windows-1258
    zh-Hans, zh-CN, zh-SG Chinese, Simplified GBK
    zh-Hant, zh-HK, zh-MO, zh-TW Chinese, Traditional Big5
    All other locales windows-1252

    The contents of this table are derived from the intersection of Windows, Chrome, and Firefox defaults.

The document's character encoding must immediately be set to the value returned from this algorithm, at the same time as the user agent uses the returned value to select the decoder to use for the input byte stream.

When an algorithm requires a user agent to prescan a byte stream to determine its encoding, given some defined end condition, then it must run the following steps. If at any point during these steps (including during instances of the get an attribute algorithm invoked by this one) the user agent either runs out of bytes (meaning the position pointer created in the first step below goes beyond the end of the byte stream obtained so far) or reaches its end condition, then abort the prescan a byte stream to determine its encoding algorithm and return the result get an XML encoding applied to the same bytes that the prescan a byte stream to determine its encoding algorithm was applied to. Otherwise, these steps will return a character encoding.

  1. Let position be a pointer to a byte in the input byte stream, initially pointing at the first byte.

  2. Prescan for UTF-16 XML declarations: If position points to:

    A sequence of bytes starting with: 0x3C, 0x0, 0x3F, 0x0, 0x78, 0x0 (case-sensitive UTF-16 little-endian '<?x')

    Return UTF-16LE.

    A sequence of bytes starting with: 0x0, 0x3C, 0x0, 0x3F, 0x0, 0x78 (case-sensitive UTF-16 big-endian '<?x')

    Return UTF-16BE.

    For historical reasons, the prefix is two bytes longer than in Appendix F of XML and the encoding name is not checked.

  3. Loop: If position points to:

    A sequence of bytes starting with: 0x3C 0x21 0x2D 0x2D (`<!--`)

    Advance the position pointer so that it points at the first 0x3E byte which is preceded by two 0x2D bytes (i.e. at the end of an ASCII '-->' sequence) and comes after the 0x3C byte that was found. (The two 0x2D bytes can be the same as those in the '<!--' sequence.)

    A sequence of bytes starting with: 0x3C, 0x4D or 0x6D, 0x45 or 0x65, 0x54 or 0x74, 0x41 or 0x61, and one of 0x09, 0x0A, 0x0C, 0x0D, 0x20, 0x2F (case-insensitive ASCII '<meta' followed by a space or slash)

    1. Advance the position pointer so that it points at the next 0x09, 0x0A, 0x0C, 0x0D, 0x20, or 0x2F byte (the one in sequence of characters matched above).

    2. Let attribute list be an empty list of strings.

    3. Let got pragma be false.

    4. Let need pragma be null.

    5. Let charset be the null value (which, for the purposes of this algorithm, is distinct from an unrecognized encoding or the empty string).

    6. Attributes: Get an attribute and its value. If no attribute was sniffed, then jump to the processing step below.

    7. If the attribute's name is already in attribute list, then return to the step labeled attributes.

    8. Add the attribute's name to attribute list.

    9. Run the appropriate step from the following list, if one applies:

      If the attribute's name is "http-equiv"

      If the attribute's value is "content-type", then set got pragma to true.

      If the attribute's name is "content"

      Apply the algorithm for extracting a character encoding from a meta element, giving the attribute's value as the string to parse. If a character encoding is returned, and if charset is still set to null, let charset be the encoding returned, and set need pragma to true.

      If the attribute's name is "charset"

      Let charset be the result of getting an encoding from the attribute's value, and set need pragma to false.

    10. Return to the step labeled attributes.

    11. Processing: If need pragma is null, then jump to the step below labeled next byte.

    12. If need pragma is true but got pragma is false, then jump to the step below labeled next byte.

    13. If charset is failure, then jump to the step below labeled next byte.

    14. If charset is UTF-16BE/LE, then set charset to UTF-8.

    15. If charset is x-user-defined, then set charset to windows-1252.

    16. Return charset.

    A sequence of bytes starting with a 0x3C byte (<), optionally a 0x2F byte (/), and finally a byte in the range 0x41-0x5A or 0x61-0x7A (A-Z or a-z)

    1. Advance the position pointer so that it points at the next 0x09 (HT), 0x0A (LF), 0x0C (FF), 0x0D (CR), 0x20 (SP), or 0x3E (>) byte.

    2. Repeatedly get an attribute until no further attributes can be found, then jump to the step below labeled next byte.

    A sequence of bytes starting with: 0x3C 0x21 (`<!`)
    A sequence of bytes starting with: 0x3C 0x2F (`</`)
    A sequence of bytes starting with: 0x3C 0x3F (`<?`)

    Advance the position pointer so that it points at the first 0x3E byte (>) that comes after the 0x3C byte that was found.

    Any other byte

    Do nothing with that byte.

  4. Next byte: Move position so it points at the next byte in the input byte stream, and return to the step above labeled loop.

When the prescan a byte stream to determine its encoding algorithm says to get an attribute, it means doing this:

  1. If the byte at position is one of 0x09 (HT), 0x0A (LF), 0x0C (FF), 0x0D (CR), 0x20 (SP), or 0x2F (/), then advance position to the next byte and redo this step.

  2. If the byte at position is 0x3E (>), then abort the get an attribute algorithm. There isn't one.

  3. Otherwise, the byte at position is the start of the attribute name. Let attribute name and attribute value be the empty string.

  4. Process the byte at position as follows:

    If it is 0x3D (=), and the attribute name is longer than the empty string
    Advance position to the next byte and jump to the step below labeled value.
    If it is 0x09 (HT), 0x0A (LF), 0x0C (FF), 0x0D (CR), or 0x20 (SP)
    Jump to the step below labeled spaces.
    If it is 0x2F (/) or 0x3E (>)
    Abort the get an attribute algorithm. The attribute's name is the value of attribute name, its value is the empty string.
    If it is in the range 0x41 (A) to 0x5A (Z)
    Append the code point b+0x20 to attribute name (where b is the value of the byte at position). (This converts the input to lowercase.)
    Anything else
    Append the code point with the same value as the byte at position to attribute name. (It doesn't actually matter how bytes outside the ASCII range are handled here, since only ASCII bytes can contribute to the detection of a character encoding.)

  5. Advance position to the next byte and return to the previous step.

  6. Spaces: If the byte at position is one of 0x09 (HT), 0x0A (LF), 0x0C (FF), 0x0D (CR), or 0x20 (SP), then advance position to the next byte, then, repeat this step.

  7. If the byte at position is not 0x3D (=), abort the get an attribute algorithm. The attribute's name is the value of attribute name, its value is the empty string.

  8. Advance position past the 0x3D (=) byte.

  9. Value: If the byte at position is one of 0x09 (HT), 0x0A (LF), 0x0C (FF), 0x0D (CR), or 0x20 (SP), then advance position to the next byte, then, repeat this step.

  10. Process the byte at position as follows:

    If it is 0x22 (") or 0x27 (')
    1. Let b be the value of the byte at position.
    2. Quote loop: Advance position to the next byte.
    3. If the value of the byte at position is the value of b, then advance position to the next byte and abort the "get an attribute" algorithm. The attribute's name is the value of attribute name, and its value is the value of attribute value.
    4. Otherwise, if the value of the byte at position is in the range 0x41 (A) to 0x5A (Z), then append a code point to attribute value whose value is 0x20 more than the value of the byte at position.
    5. Otherwise, append a code point to attribute value whose value is the same as the value of the byte at position.
    6. Return to the step above labeled quote loop.
    If it is 0x3E (>)
    Abort the get an attribute algorithm. The attribute's name is the value of attribute name, its value is the empty string.
    If it is in the range 0x41 (A) to 0x5A (Z)
    Append a code point b+0x20 to attribute value (where b is the value of the byte at position). Advance position to the next byte.
    Anything else
    Append a code point with the same value as the byte at position to attribute value. Advance position to the next byte.

  11. Process the byte at position as follows:

    If it is 0x09 (HT), 0x0A (LF), 0x0C (FF), 0x0D (CR), 0x20 (SP), or 0x3E (>)
    Abort the get an attribute algorithm. The attribute's name is the value of attribute name and its value is the value of attribute value.
    If it is in the range 0x41 (A) to 0x5A (Z)
    Append a code point b+0x20 to attribute value (where b is the value of the byte at position).
    Anything else
    Append a code point with the same value as the byte at position to attribute value.

  12. Advance position to the next byte and return to the previous step.

When the prescan a byte stream to determine its encoding algorithm is aborted without returning an encoding, get an XML encoding means doing this.

Looking for syntax resembling an XML declaration, even in text/html, is necessary for compatibility with existing content.

  1. Let encodingPosition be a pointer to the start of the stream.

  2. If encodingPosition does not point to the start of a byte sequence 0x3C, 0x3F, 0x78, 0x6D, 0x6C (`<?xml`), then return failure.

  3. Let xmlDeclarationEnd be a pointer to the next byte in the input byte stream which is 0x3E (>). If there is no such byte, then return failure.

  4. Set encodingPosition to the position of the first occurrence of the subsequence of bytes 0x65, 0x6E, 0x63, 0x6F, 0x64, 0x69, 0x6E, 0x67 (`encoding`) at or after the current encodingPosition. If there is no such sequence, then return failure.

  5. Advance encodingPosition past the 0x67 (g) byte.

  6. While the byte at encodingPosition is less than or equal to 0x20 (i.e., it is either an ASCII space or control character), advance encodingPosition to the next byte.

  7. If the byte at encodingPosition is not 0x3D (=), then return failure.

  8. Advance encodingPosition to the next byte.

  9. While the byte at encodingPosition is less than or equal to 0x20 (i.e., it is either an ASCII space or control character), advance encodingPosition to the next byte.

  10. Let quoteMark be the byte at encodingPosition.

  11. If quoteMark is not either 0x22 (") or 0x27 ('), then return failure.

  12. Advance encodingPosition to the next byte.

  13. Let encodingEndPosition be the position of the next occurrence of quoteMark at or after encodingPosition. If quoteMark does not occur again, then return failure.

  14. Let potentialEncoding be the sequence of the bytes between encodingPosition (inclusive) and encodingEndPosition (exclusive).

  15. If potentialEncoding contains one or more bytes whose byte value is 0x20 or below, then return failure.

  16. Let encoding be the result of getting an encoding given potentialEncoding isomorphic decoded.

  17. If the encoding is UTF-16BE/LE, then change it to UTF-8.

  18. Return encoding.

For the sake of interoperability, user agents should not use a pre-scan algorithm that returns different results than the one described above. (But, if you do, please at least let us know, so that we can improve this algorithm and benefit everyone...)

13.2.3.3 Character encodings

User agents must support the encodings defined in Encoding, including, but not limited to, UTF-8, ISO-8859-2, ISO-8859-7, ISO-8859-8, windows-874, windows-1250, windows-1251, windows-1252, windows-1254, windows-1255, windows-1256, windows-1257, windows-1258, GBK, Big5, ISO-2022-JP, Shift_JIS, EUC-KR, UTF-16BE, UTF-16LE, UTF-16BE/LE, and x-user-defined. User agents must not support other encodings.

The above prohibits supporting, for example, CESU-8, UTF-7, BOCU-1, SCSU, EBCDIC, and UTF-32. This specification does not make any attempt to support prohibited encodings in its algorithms; support and use of prohibited encodings would thus lead to unexpected behavior. [CESU8] [UTF7] [BOCU1] [SCSU]

13.2.3.4 Changing the encoding while parsing

When the parser requires the user agent to change the encoding, it must run the following steps. This might happen if the encoding sniffing algorithm described above failed to find a character encoding, or if it found a character encoding that was not the actual encoding of the file.

  1. If the encoding that is already being used to interpret the input stream is UTF-16BE/LE, then set the confidence to certain and return. The new encoding is ignored; if it was anything but the same encoding, then it would be clearly incorrect.

  2. If the new encoding is UTF-16BE/LE, then change it to UTF-8.

  3. If the new encoding is x-user-defined, then change it to windows-1252.

  4. If the new encoding is identical or equivalent to the encoding that is already being used to interpret the input stream, then set the confidence to certain and return. This happens when the encoding information found in the file matches what the encoding sniffing algorithm determined to be the encoding, and in the second pass through the parser if the first pass found that the encoding sniffing algorithm described in the earlier section failed to find the right encoding.

  5. If all the bytes up to the last byte converted by the current decoder have the same Unicode interpretations in both the current encoding and the new encoding, and if the user agent supports changing the converter on the fly, then the user agent may change to the new converter for the encoding on the fly. Set the document's character encoding and the encoding used to convert the input stream to the new encoding, set the confidence to certain, and return.

  6. Otherwise, restart the navigate algorithm, with historyHandling set to "replace" and other inputs kept the same, but this time skip the encoding sniffing algorithm and instead just set the encoding to the new encoding and the confidence to certain. Whenever possible, this should be done without actually contacting the network layer (the bytes should be re-parsed from memory), even if, e.g., the document is marked as not being cacheable. If this is not possible and contacting the network layer would involve repeating a request that uses a method other than `GET`, then instead set the confidence to certain and ignore the new encoding. The resource will be misinterpreted. User agents may notify the user of the situation, to aid in application development.

This algorithm is only invoked when a new encoding is found declared on a meta element.

13.2.3.5 Preprocessing the input stream

The input stream consists of the characters pushed into it as the input byte stream is decoded or from the various APIs that directly manipulate the input stream.

Any occurrences of surrogates are surrogate-in-input-stream parse errors. Any occurrences of noncharacters are noncharacter-in-input-stream parse errors and any occurrences of controls other than ASCII whitespace and U+0000 NULL characters are control-character-in-input-stream parse errors.

The handling of U+0000 NULL characters varies based on where the characters are found and happens at the later stages of the parsing. They are either ignored or, for security reasons, replaced with a U+FFFD REPLACEMENT CHARACTER. This handling is, by necessity, spread across both the tokenization stage and the tree construction stage.

Before the tokenization stage, the input stream must be preprocessed by normalizing newlines. Thus, newlines in HTML DOMs are represented by U+000A LF characters, and there are never any U+000D CR characters in the input to the tokenization stage.

The next input character is the first character in the input stream that has not yet been consumed or explicitly ignored by the requirements in this section. Initially, the next input character is the first character in the input. The current input character is the last character to have been consumed.

The insertion point is the position (just before a character or just before the end of the input stream) where content inserted using document.write() is actually inserted. The insertion point is relative to the position of the character immediately after it, it is not an absolute offset into the input stream. Initially, the insertion point is undefined.

The "EOF" character in the tables below is a conceptual character representing the end of the input stream. If the parser is a script-created parser, then the end of the input stream is reached when an explicit "EOF" character (inserted by the document.close() method) is consumed. Otherwise, the "EOF" character is not a real character in the stream, but rather the lack of any further characters.

13.2.4 Parse state

13.2.4.1 The insertion mode

The insertion mode is a state variable that controls the primary operation of the tree construction stage.

Initially, the insertion mode is "initial". It can change to "before html", "before head", "in head", "in head noscript", "after head", "in body", "text", "in table", "in table text", "in caption", "in column group", "in table body", "in row", "in cell", "in template", "after body", "in frameset", "after frameset", "after after body", and "after after frameset" during the course of the parsing, as described in the tree construction stage. The insertion mode affects how tokens are processed and whether CDATA sections are supported.

Several of these modes, namely "in head", "in body", and "in table", are special, in that the other modes defer to them at various times. When the algorithm below says that the user agent is to do something "using the rules for the m insertion mode", where m is one of these modes, the user agent must use the rules described under the m insertion mode's section, but must leave the insertion mode unchanged unless the rules in m themselves switch the insertion mode to a new value.

When the insertion mode is switched to "text" or "in table text", the original insertion mode is also set. This is the insertion mode to which the tree construction stage will return.

Similarly, to parse nested template elements, a stack of template insertion modes is used. It is initially empty. The current template insertion mode is the insertion mode that was most recently added to the stack of template insertion modes. The algorithms in the sections below will push insertion modes onto this stack, meaning that the specified insertion mode is to be added to the stack, and pop insertion modes from the stack, which means that the most recently added insertion mode must be removed from the stack.

When the steps below require the UA to reset the insertion mode appropriately, it means the UA must follow these steps:

  1. Let last be false.

  2. Let node be the last node in the stack of open elements.

  3. Loop: If node is the first node in the stack of open elements, then set last to true, and, if the parser was created as part of the HTML fragment parsing algorithm (fragment case), set node to the context element passed to that algorithm.

  4. If node is a td or th element and last is false, then switch the insertion mode to "in cell" and return.

  5. If node is a tr element, then switch the insertion mode to "in row" and return.

  6. If node is a tbody, thead, or tfoot element, then switch the insertion mode to "in table body" and return.

  7. If node is a caption element, then switch the insertion mode to "in caption" and return.

  8. If node is a colgroup element, then switch the insertion mode to "in column group" and return.

  9. If node is a table element, then switch the insertion mode to "in table" and return.

  10. If node is a template element, then switch the insertion mode to the current template insertion mode and return.

  11. If node is a head element and last is false, then switch the insertion mode to "in head" and return.

  12. If node is a body element, then switch the insertion mode to "in body" and return.

  13. If node is a frameset element, then switch the insertion mode to "in frameset" and return. (fragment case)

  14. If node is an html element, run these substeps:

    1. If the head element pointer is null, switch the insertion mode to "before head" and return. (fragment case)

    2. Otherwise, the head element pointer is not null, switch the insertion mode to "after head" and return.

  15. If last is true, then switch the insertion mode to "in body" and return. (fragment case)

  16. Let node now be the node before node in the stack of open elements.

  17. Return to the step labeled loop.

13.2.4.2 The stack of open elements

Initially, the stack of open elements is empty. The stack grows downwards; the topmost node on the stack is the first one added to the stack, and the bottommost node of the stack is the most recently added node in the stack (notwithstanding when the stack is manipulated in a random access fashion as part of the handling for misnested tags).

The "before html" insertion mode creates the html document element, which is then added to the stack.

In the fragment case, the stack of open elements is initialized to contain an html element that is created as part of that algorithm. (The fragment case skips the "before html" insertion mode.)

The html node, however it is created, is the topmost node of the stack. It only gets popped off the stack when the parser finishes.

The current node is the bottommost node in this stack of open elements.

The adjusted current node is the context element if the parser was created as part of the HTML fragment parsing algorithm and the stack of open elements has only one element in it (fragment case); otherwise, the adjusted current node is the current node.

When the current node is removed from the stack of open elements, process internal resource links given the current node's node document.

Elements in the stack of open elements fall into the following categories:

Special

The following elements have varying levels of special parsing rules: HTML's address, applet, area, article, aside, base, basefont, bgsound, blockquote, body, br, button, caption, center, col, colgroup, dd, details, dir, div, dl, dt, embed, fieldset, figcaption, figure, footer, form, frame, frameset, h1, h2, h3, h4, h5, h6, head, header, hgroup, hr, html, iframe, img, input, keygen, li, link, listing, main, marquee, menu, meta, nav, noembed, noframes, noscript, object, ol, p, param, plaintext, pre, script, search, section, select, source, style, summary, table, tbody, td, template, textarea, tfoot, th, thead, title, tr, track, ul, wbr, xmp; MathML mi, MathML mo, MathML mn, MathML ms, MathML mtext, and MathML annotation-xml; and SVG foreignObject, SVG desc, and SVG title.

An image start tag token is handled by the tree builder, but it is not in this list because it is not an element; it gets turned into an img element.

Formatting

The following HTML elements are those that end up in the list of active formatting elements: a, b, big, code, em, font, i, nobr, s, small, strike, strong, tt, and u.

Ordinary

All other elements found while parsing an HTML document.

Typically, the special elements have the start and end tag tokens handled specifically, while ordinary elements' tokens fall into "any other start tag" and "any other end tag" clauses, and some parts of the tree builder check if a particular element in the stack of open elements is in the special category. However, some elements (e.g., the option element) have their start or end tag tokens handled specifically, but are still not in the special category, so that they get the ordinary handling elsewhere.

The stack of open elements is said to have an element target node in a specific scope consisting of a list of element types list when the following algorithm terminates in a match state:

  1. Initialize node to be the current node (the bottommost node of the stack).

  2. If node is target node, terminate in a match state.

  3. Otherwise, if node is one of the element types in list, terminate in a failure state.

  4. Otherwise, set node to the previous entry in the stack of open elements and return to step 2. (This will never fail, since the loop will always terminate in the previous step if the top of the stack — an html element — is reached.)

The stack of open elements is said to have a particular element in scope when it has that element in the specific scope consisting of the following element types:

The stack of open elements is said to have a particular element in list item scope when it has that element in the specific scope consisting of the following element types:

The stack of open elements is said to have a particular element in button scope when it has that element in the specific scope consisting of the following element types:

The stack of open elements is said to have a particular element in table scope when it has that element in the specific scope consisting of the following element types:

Nothing happens if at any time any of the elements in the stack of open elements are moved to a new location in, or removed from, the Document tree. In particular, the stack is not changed in this situation. This can cause, amongst other strange effects, content to be appended to nodes that are no longer in the DOM.

In some cases (namely, when closing misnested formatting elements), the stack is manipulated in a random-access fashion.

13.2.4.3 The list of active formatting elements

Initially, the list of active formatting elements is empty. It is used to handle mis-nested formatting element tags.

The list contains elements in the formatting category, and markers. The markers are inserted when entering applet, object, marquee, template, td, th, and caption elements, and are used to prevent formatting from "leaking" into applet, object, marquee, template, td, th, and caption elements.

In addition, each element in the list of active formatting elements is associated with the token for which it was created, so that further elements can be created for that token if necessary.

When the steps below require the UA to push onto the list of active formatting elements an element element, the UA must perform the following steps:

  1. If there are already three elements in the list of active formatting elements after the last marker, if any, or anywhere in the list if there are no markers, that have the same tag name, namespace, and attributes as element, then remove the earliest such element from the list of active formatting elements. For these purposes, the attributes must be compared as they were when the elements were created by the parser; two elements have the same attributes if all their parsed attributes can be paired such that the two attributes in each pair have identical names, namespaces, and values (the order of the attributes does not matter).

    This is the Noah's Ark clause. But with three per family instead of two.

  2. Add element to the list of active formatting elements.

When the steps below require the UA to reconstruct the active formatting elements, the UA must perform the following steps:

  1. If there are no entries in the list of active formatting elements, then there is nothing to reconstruct; stop this algorithm.

  2. If the last (most recently added) entry in the list of active formatting elements is a marker, or if it is an element that is in the stack of open elements, then there is nothing to reconstruct; stop this algorithm.

  3. Let entry be the last (most recently added) element in the list of active formatting elements.

  4. Rewind: If there are no entries before entry in the list of active formatting elements, then jump to the step labeled create.

  5. Let entry be the entry one earlier than entry in the list of active formatting elements.

  6. If entry is neither a marker nor an element that is also in the stack of open elements, go to the step labeled rewind.

  7. Advance: Let entry be the element one later than entry in the list of active formatting elements.

  8. Create: Insert an HTML element for the token for which the element entry was created, to obtain new element.

  9. Replace the entry for entry in the list with an entry for new element.

  10. If the entry for new element in the list of active formatting elements is not the last entry in the list, return to the step labeled advance.

This has the effect of reopening all the formatting elements that were opened in the current body, cell, or caption (whichever is youngest) that haven't been explicitly closed.

The way this specification is written, the list of active formatting elements always consists of elements in chronological order with the least recently added element first and the most recently added element last (except for while steps 7 to 10 of the above algorithm are being executed, of course).

When the steps below require the UA to clear the list of active formatting elements up to the last marker, the UA must perform the following steps:

  1. Let entry be the last (most recently added) entry in the list of active formatting elements.

  2. Remove entry from the list of active formatting elements.

  3. If entry was a marker, then stop the algorithm at this point. The list has been cleared up to the last marker.

  4. Go to step 1.

13.2.4.4 The element pointers

Initially, the head element pointer and the form element pointer are both null.

Once a head element has been parsed (whether implicitly or explicitly) the head element pointer gets set to point to this node.

The form element pointer points to the last form element that was opened and whose end tag has not yet been seen. It is used to make form controls associate with forms in the face of dramatically bad markup, for historical reasons. It is ignored inside template elements.

13.2.4.5 Other parsing state flags

The scripting flag is set to "enabled" if scripting was enabled for the Document with which the parser is associated when the parser was created, and "disabled" otherwise.

The scripting flag can be enabled even when the parser was created as part of the HTML fragment parsing algorithm, even though script elements don't execute in that case.

The frameset-ok flag is set to "ok" when the parser is created. It is set to "not ok" after certain tokens are seen.

13.2.5 Tokenization

Implementations must act as if they used the following state machine to tokenize HTML. The state machine must start in the data state. Most states consume a single character, which may have various side-effects, and either switches the state machine to a new state to reconsume the current input character, or switches it to a new state to consume the next character, or stays in the same state to consume the next character. Some states have more complicated behavior and can consume several characters before switching to another state. In some cases, the tokenizer state is also changed by the tree construction stage.

When a state says to reconsume a matched character in a specified state, that means to switch to that state, but when it attempts to consume the next input character, provide it with the current input character instead.

The exact behavior of certain states depends on the insertion mode and the stack of open elements. Certain states also use a temporary buffer to track progress, and the character reference state uses a return state to return to the state it was invoked from.

The output of the tokenization step is a series of zero or more of the following tokens: DOCTYPE, start tag, end tag, comment, character, end-of-file. DOCTYPE tokens have a name, a public identifier, a system identifier, and a force-quirks flag. When a DOCTYPE token is created, its name, public identifier, and system identifier must be marked as missing (which is a distinct state from the empty string), and the force-quirks flag must be set to off (its other state is on). Start and end tag tokens have a tag name, a self-closing flag, and a list of attributes, each of which has a name and a value. When a start or end tag token is created, its self-closing flag must be unset (its other state is that it be set), and its attributes list must be empty. Comment and character tokens have data.

When a token is emitted, it must immediately be handled by the tree construction stage. The tree construction stage can affect the state of the tokenization stage, and can insert additional characters into the stream. (For example, the script element can result in scripts executing and using the dynamic markup insertion APIs to insert characters into the stream being tokenized.)

Creating a token and emitting it are distinct actions. It is possible for a token to be created but implicitly abandoned (never emitted), e.g. if the file ends unexpectedly while processing the characters that are being parsed into a start tag token.

When a start tag token is emitted with its self-closing flag set, if the flag is not acknowledged when it is processed by the tree construction stage, that is a non-void-html-element-start-tag-with-trailing-solidus parse error.

When an end tag token is emitted with attributes, that is an end-tag-with-attributes parse error.

When an end tag token is emitted with its self-closing flag set, that is an end-tag-with-trailing-solidus parse error.

An appropriate end tag token is an end tag token whose tag name matches the tag name of the last start tag to have been emitted from this tokenizer, if any. If no start tag has been emitted from this tokenizer, then no end tag token is appropriate.

A character reference is said to be consumed as part of an attribute if the return state is either attribute value (double-quoted) state, attribute value (single-quoted) state, or attribute value (unquoted) state.

When a state says to flush code points consumed as a character reference, it means that for each code point in the temporary buffer (in the order they were added to the buffer), the user agent must append the code point from the buffer to the current attribute's value if the character reference was consumed as part of an attribute, or emit the code point as a character token otherwise.

Before each step of the tokenizer, the user agent must first check the parser pause flag. If it is true, then the tokenizer must abort the processing of any nested invocations of the tokenizer, yielding control back to the caller.

The tokenizer state machine consists of the states defined in the following subsections.

13.2.5.1 Data state

Consume the next input character:

U+0026 AMPERSAND (&)
Set the return state to the data state. Switch to the character reference state.
U+003C LESS-THAN SIGN (<)
Switch to the tag open state.
U+0000 NULL
This is an unexpected-null-character parse error. Emit the current input character as a character token.
EOF
Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.2 RCDATA state

Consume the next input character:

U+0026 AMPERSAND (&)
Set the return state to the RCDATA state. Switch to the character reference state.
U+003C LESS-THAN SIGN (<)
Switch to the RCDATA less-than sign state.
U+0000 NULL
This is an unexpected-null-character parse error. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.3 RAWTEXT state

Consume the next input character:

U+003C LESS-THAN SIGN (<)
Switch to the RAWTEXT less-than sign state.
U+0000 NULL
This is an unexpected-null-character parse error. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.4 Script data state

Consume the next input character:

U+003C LESS-THAN SIGN (<)
Switch to the script data less-than sign state.
U+0000 NULL
This is an unexpected-null-character parse error. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.5 PLAINTEXT state

Consume the next input character:

U+0000 NULL
This is an unexpected-null-character parse error. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.6 Tag open state

Consume the next input character:

U+0021 EXCLAMATION MARK (!)
Switch to the markup declaration open state.
U+002F SOLIDUS (/)
Switch to the end tag open state.
ASCII alpha
Create a new start tag token, set its tag name to the empty string. Reconsume in the tag name state.
U+003F QUESTION MARK (?)
This is an unexpected-question-mark-instead-of-tag-name parse error. Create a comment token whose data is the empty string. Reconsume in the bogus comment state.
EOF
This is an eof-before-tag-name parse error. Emit a U+003C LESS-THAN SIGN character token and an end-of-file token.
Anything else
This is an invalid-first-character-of-tag-name parse error. Emit a U+003C LESS-THAN SIGN character token. Reconsume in the data state.
13.2.5.7 End tag open state

Consume the next input character:

ASCII alpha
Create a new end tag token, set its tag name to the empty string. Reconsume in the tag name state.
U+003E GREATER-THAN SIGN (>)
This is a missing-end-tag-name parse error. Switch to the data state.
EOF
This is an eof-before-tag-name parse error. Emit a U+003C LESS-THAN SIGN character token, a U+002F SOLIDUS character token and an end-of-file token.
Anything else
This is an invalid-first-character-of-tag-name parse error. Create a comment token whose data is the empty string. Reconsume in the bogus comment state.
13.2.5.8 Tag name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
Switch to the before attribute name state.
U+002F SOLIDUS (/)
Switch to the self-closing start tag state.
U+003E GREATER-THAN SIGN (>)
Switch to the data state. Emit the current tag token.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the current tag token's tag name.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the current tag token's tag name.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
Append the current input character to the current tag token's tag name.
13.2.5.9 RCDATA less-than sign state

Consume the next input character:

U+002F SOLIDUS (/)
Set the temporary buffer to the empty string. Switch to the RCDATA end tag open state.
Anything else
Emit a U+003C LESS-THAN SIGN character token. Reconsume in the RCDATA state.
13.2.5.10 RCDATA end tag open state

Consume the next input character:

ASCII alpha
Create a new end tag token, set its tag name to the empty string. Reconsume in the RCDATA end tag name state.
Anything else
Emit a U+003C LESS-THAN SIGN character token and a U+002F SOLIDUS character token. Reconsume in the RCDATA state.
13.2.5.11 RCDATA end tag name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
If the current end tag token is an appropriate end tag token, then switch to the before attribute name state. Otherwise, treat it as per the "anything else" entry below.
U+002F SOLIDUS (/)
If the current end tag token is an appropriate end tag token, then switch to the self-closing start tag state. Otherwise, treat it as per the "anything else" entry below.
U+003E GREATER-THAN SIGN (>)
If the current end tag token is an appropriate end tag token, then switch to the data state and emit the current tag token. Otherwise, treat it as per the "anything else" entry below.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the current tag token's tag name. Append the current input character to the temporary buffer.
ASCII lower alpha
Append the current input character to the current tag token's tag name. Append the current input character to the temporary buffer.
Anything else
Emit a U+003C LESS-THAN SIGN character token, a U+002F SOLIDUS character token, and a character token for each of the characters in the temporary buffer (in the order they were added to the buffer). Reconsume in the RCDATA state.
13.2.5.12 RAWTEXT less-than sign state

Consume the next input character:

U+002F SOLIDUS (/)
Set the temporary buffer to the empty string. Switch to the RAWTEXT end tag open state.
Anything else
Emit a U+003C LESS-THAN SIGN character token. Reconsume in the RAWTEXT state.
13.2.5.13 RAWTEXT end tag open state

Consume the next input character:

ASCII alpha
Create a new end tag token, set its tag name to the empty string. Reconsume in the RAWTEXT end tag name state.
Anything else
Emit a U+003C LESS-THAN SIGN character token and a U+002F SOLIDUS character token. Reconsume in the RAWTEXT state.
13.2.5.14 RAWTEXT end tag name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
If the current end tag token is an appropriate end tag token, then switch to the before attribute name state. Otherwise, treat it as per the "anything else" entry below.
U+002F SOLIDUS (/)
If the current end tag token is an appropriate end tag token, then switch to the self-closing start tag state. Otherwise, treat it as per the "anything else" entry below.
U+003E GREATER-THAN SIGN (>)
If the current end tag token is an appropriate end tag token, then switch to the data state and emit the current tag token. Otherwise, treat it as per the "anything else" entry below.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the current tag token's tag name. Append the current input character to the temporary buffer.
ASCII lower alpha
Append the current input character to the current tag token's tag name. Append the current input character to the temporary buffer.
Anything else
Emit a U+003C LESS-THAN SIGN character token, a U+002F SOLIDUS character token, and a character token for each of the characters in the temporary buffer (in the order they were added to the buffer). Reconsume in the RAWTEXT state.
13.2.5.15 Script data less-than sign state

Consume the next input character:

U+002F SOLIDUS (/)
Set the temporary buffer to the empty string. Switch to the script data end tag open state.
U+0021 EXCLAMATION MARK (!)
Switch to the script data escape start state. Emit a U+003C LESS-THAN SIGN character token and a U+0021 EXCLAMATION MARK character token.
Anything else
Emit a U+003C LESS-THAN SIGN character token. Reconsume in the script data state.
13.2.5.16 Script data end tag open state

Consume the next input character:

ASCII alpha
Create a new end tag token, set its tag name to the empty string. Reconsume in the script data end tag name state.
Anything else
Emit a U+003C LESS-THAN SIGN character token and a U+002F SOLIDUS character token. Reconsume in the script data state.
13.2.5.17 Script data end tag name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
If the current end tag token is an appropriate end tag token, then switch to the before attribute name state. Otherwise, treat it as per the "anything else" entry below.
U+002F SOLIDUS (/)
If the current end tag token is an appropriate end tag token, then switch to the self-closing start tag state. Otherwise, treat it as per the "anything else" entry below.
U+003E GREATER-THAN SIGN (>)
If the current end tag token is an appropriate end tag token, then switch to the data state and emit the current tag token. Otherwise, treat it as per the "anything else" entry below.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the current tag token's tag name. Append the current input character to the temporary buffer.
ASCII lower alpha
Append the current input character to the current tag token's tag name. Append the current input character to the temporary buffer.
Anything else
Emit a U+003C LESS-THAN SIGN character token, a U+002F SOLIDUS character token, and a character token for each of the characters in the temporary buffer (in the order they were added to the buffer). Reconsume in the script data state.
13.2.5.18 Script data escape start state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the script data escape start dash state. Emit a U+002D HYPHEN-MINUS character token.
Anything else
Reconsume in the script data state.
13.2.5.19 Script data escape start dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the script data escaped dash dash state. Emit a U+002D HYPHEN-MINUS character token.
Anything else
Reconsume in the script data state.
13.2.5.20 Script data escaped state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the script data escaped dash state. Emit a U+002D HYPHEN-MINUS character token.
U+003C LESS-THAN SIGN (<)
Switch to the script data escaped less-than sign state.
U+0000 NULL
This is an unexpected-null-character parse error. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
This is an eof-in-script-html-comment-like-text parse error. Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.21 Script data escaped dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the script data escaped dash dash state. Emit a U+002D HYPHEN-MINUS character token.
U+003C LESS-THAN SIGN (<)
Switch to the script data escaped less-than sign state.
U+0000 NULL
This is an unexpected-null-character parse error. Switch to the script data escaped state. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
This is an eof-in-script-html-comment-like-text parse error. Emit an end-of-file token.
Anything else
Switch to the script data escaped state. Emit the current input character as a character token.
13.2.5.22 Script data escaped dash dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Emit a U+002D HYPHEN-MINUS character token.
U+003C LESS-THAN SIGN (<)
Switch to the script data escaped less-than sign state.
U+003E GREATER-THAN SIGN (>)
Switch to the script data state. Emit a U+003E GREATER-THAN SIGN character token.
U+0000 NULL
This is an unexpected-null-character parse error. Switch to the script data escaped state. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
This is an eof-in-script-html-comment-like-text parse error. Emit an end-of-file token.
Anything else
Switch to the script data escaped state. Emit the current input character as a character token.
13.2.5.23 Script data escaped less-than sign state

Consume the next input character:

U+002F SOLIDUS (/)
Set the temporary buffer to the empty string. Switch to the script data escaped end tag open state.
ASCII alpha
Set the temporary buffer to the empty string. Emit a U+003C LESS-THAN SIGN character token. Reconsume in the script data double escape start state.
Anything else
Emit a U+003C LESS-THAN SIGN character token. Reconsume in the script data escaped state.
13.2.5.24 Script data escaped end tag open state

Consume the next input character:

ASCII alpha
Create a new end tag token, set its tag name to the empty string. Reconsume in the script data escaped end tag name state.
Anything else
Emit a U+003C LESS-THAN SIGN character token and a U+002F SOLIDUS character token. Reconsume in the script data escaped state.
13.2.5.25 Script data escaped end tag name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
If the current end tag token is an appropriate end tag token, then switch to the before attribute name state. Otherwise, treat it as per the "anything else" entry below.
U+002F SOLIDUS (/)
If the current end tag token is an appropriate end tag token, then switch to the self-closing start tag state. Otherwise, treat it as per the "anything else" entry below.
U+003E GREATER-THAN SIGN (>)
If the current end tag token is an appropriate end tag token, then switch to the data state and emit the current tag token. Otherwise, treat it as per the "anything else" entry below.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the current tag token's tag name. Append the current input character to the temporary buffer.
ASCII lower alpha
Append the current input character to the current tag token's tag name. Append the current input character to the temporary buffer.
Anything else
Emit a U+003C LESS-THAN SIGN character token, a U+002F SOLIDUS character token, and a character token for each of the characters in the temporary buffer (in the order they were added to the buffer). Reconsume in the script data escaped state.
13.2.5.26 Script data double escape start state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
U+002F SOLIDUS (/)
U+003E GREATER-THAN SIGN (>)
If the temporary buffer is the string "script", then switch to the script data double escaped state. Otherwise, switch to the script data escaped state. Emit the current input character as a character token.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the temporary buffer. Emit the current input character as a character token.
ASCII lower alpha
Append the current input character to the temporary buffer. Emit the current input character as a character token.
Anything else
Reconsume in the script data escaped state.
13.2.5.27 Script data double escaped state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the script data double escaped dash state. Emit a U+002D HYPHEN-MINUS character token.
U+003C LESS-THAN SIGN (<)
Switch to the script data double escaped less-than sign state. Emit a U+003C LESS-THAN SIGN character token.
U+0000 NULL
This is an unexpected-null-character parse error. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
This is an eof-in-script-html-comment-like-text parse error. Emit an end-of-file token.
Anything else
Emit the current input character as a character token.
13.2.5.28 Script data double escaped dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the script data double escaped dash dash state. Emit a U+002D HYPHEN-MINUS character token.
U+003C LESS-THAN SIGN (<)
Switch to the script data double escaped less-than sign state. Emit a U+003C LESS-THAN SIGN character token.
U+0000 NULL
This is an unexpected-null-character parse error. Switch to the script data double escaped state. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
This is an eof-in-script-html-comment-like-text parse error. Emit an end-of-file token.
Anything else
Switch to the script data double escaped state. Emit the current input character as a character token.
13.2.5.29 Script data double escaped dash dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Emit a U+002D HYPHEN-MINUS character token.
U+003C LESS-THAN SIGN (<)
Switch to the script data double escaped less-than sign state. Emit a U+003C LESS-THAN SIGN character token.
U+003E GREATER-THAN SIGN (>)
Switch to the script data state. Emit a U+003E GREATER-THAN SIGN character token.
U+0000 NULL
This is an unexpected-null-character parse error. Switch to the script data double escaped state. Emit a U+FFFD REPLACEMENT CHARACTER character token.
EOF
This is an eof-in-script-html-comment-like-text parse error. Emit an end-of-file token.
Anything else
Switch to the script data double escaped state. Emit the current input character as a character token.
13.2.5.30 Script data double escaped less-than sign state

Consume the next input character:

U+002F SOLIDUS (/)
Set the temporary buffer to the empty string. Switch to the script data double escape end state. Emit a U+002F SOLIDUS character token.
Anything else
Reconsume in the script data double escaped state.
13.2.5.31 Script data double escape end state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
U+002F SOLIDUS (/)
U+003E GREATER-THAN SIGN (>)
If the temporary buffer is the string "script", then switch to the script data escaped state. Otherwise, switch to the script data double escaped state. Emit the current input character as a character token.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the temporary buffer. Emit the current input character as a character token.
ASCII lower alpha
Append the current input character to the temporary buffer. Emit the current input character as a character token.
Anything else
Reconsume in the script data double escaped state.
13.2.5.32 Before attribute name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
Ignore the character.
U+002F SOLIDUS (/)
U+003E GREATER-THAN SIGN (>)
EOF
Reconsume in the after attribute name state.
U+003D EQUALS SIGN (=)
This is an unexpected-equals-sign-before-attribute-name parse error. Start a new attribute in the current tag token. Set that attribute's name to the current input character, and its value to the empty string. Switch to the attribute name state.
Anything else
Start a new attribute in the current tag token. Set that attribute name and value to the empty string. Reconsume in the attribute name state.
13.2.5.33 Attribute name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
U+002F SOLIDUS (/)
U+003E GREATER-THAN SIGN (>)
EOF
Reconsume in the after attribute name state.
U+003D EQUALS SIGN (=)
Switch to the before attribute value state.
ASCII upper alpha
Append the lowercase version of the current input character (add 0x0020 to the character's code point) to the current attribute's name.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the current attribute's name.
U+0022 QUOTATION MARK (")
U+0027 APOSTROPHE (')
U+003C LESS-THAN SIGN (<)
This is an unexpected-character-in-attribute-name parse error. Treat it as per the "anything else" entry below.
Anything else
Append the current input character to the current attribute's name.

When the user agent leaves the attribute name state (and before emitting the tag token, if appropriate), the complete attribute's name must be compared to the other attributes on the same token; if there is already an attribute on the token with the exact same name, then this is a duplicate-attribute parse error and the new attribute must be removed from the token.

If an attribute is so removed from a token, it, and the value that gets associated with it, if any, are never subsequently used by the parser, and are therefore effectively discarded. Removing the attribute in this way does not change its status as the "current attribute" for the purposes of the tokenizer, however.

13.2.5.34 After attribute name state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
Ignore the character.
U+002F SOLIDUS (/)
Switch to the self-closing start tag state.
U+003D EQUALS SIGN (=)
Switch to the before attribute value state.
U+003E GREATER-THAN SIGN (>)
Switch to the data state. Emit the current tag token.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
Start a new attribute in the current tag token. Set that attribute name and value to the empty string. Reconsume in the attribute name state.
13.2.5.35 Before attribute value state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
Ignore the character.
U+0022 QUOTATION MARK (")
Switch to the attribute value (double-quoted) state.
U+0027 APOSTROPHE (')
Switch to the attribute value (single-quoted) state.
U+003E GREATER-THAN SIGN (>)
This is a missing-attribute-value parse error. Switch to the data state. Emit the current tag token.
Anything else
Reconsume in the attribute value (unquoted) state.
13.2.5.36 Attribute value (double-quoted) state

Consume the next input character:

U+0022 QUOTATION MARK (")
Switch to the after attribute value (quoted) state.
U+0026 AMPERSAND (&)
Set the return state to the attribute value (double-quoted) state. Switch to the character reference state.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the current attribute's value.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
Append the current input character to the current attribute's value.
13.2.5.37 Attribute value (single-quoted) state

Consume the next input character:

U+0027 APOSTROPHE (')
Switch to the after attribute value (quoted) state.
U+0026 AMPERSAND (&)
Set the return state to the attribute value (single-quoted) state. Switch to the character reference state.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the current attribute's value.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
Append the current input character to the current attribute's value.
13.2.5.38 Attribute value (unquoted) state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
Switch to the before attribute name state.
U+0026 AMPERSAND (&)
Set the return state to the attribute value (unquoted) state. Switch to the character reference state.
U+003E GREATER-THAN SIGN (>)
Switch to the data state. Emit the current tag token.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the current attribute's value.
U+0022 QUOTATION MARK (")
U+0027 APOSTROPHE (')
U+003C LESS-THAN SIGN (<)
U+003D EQUALS SIGN (=)
U+0060 GRAVE ACCENT (`)
This is an unexpected-character-in-unquoted-attribute-value parse error. Treat it as per the "anything else" entry below.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
Append the current input character to the current attribute's value.
13.2.5.39 After attribute value (quoted) state

Consume the next input character:

U+0009 CHARACTER TABULATION (tab)
U+000A LINE FEED (LF)
U+000C FORM FEED (FF)
U+0020 SPACE
Switch to the before attribute name state.
U+002F SOLIDUS (/)
Switch to the self-closing start tag state.
U+003E GREATER-THAN SIGN (>)
Switch to the data state. Emit the current tag token.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
This is a missing-whitespace-between-attributes parse error. Reconsume in the before attribute name state.
13.2.5.40 Self-closing start tag state

Consume the next input character:

U+003E GREATER-THAN SIGN (>)
Set the self-closing flag of the current tag token. Switch to the data state. Emit the current tag token.
EOF
This is an eof-in-tag parse error. Emit an end-of-file token.
Anything else
This is an unexpected-solidus-in-tag parse error. Reconsume in the before attribute name state.
13.2.5.41 Bogus comment state

Consume the next input character:

U+003E GREATER-THAN SIGN (>)
Switch to the data state. Emit the current comment token.
EOF
Emit the comment. Emit an end-of-file token.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the comment token's data.
Anything else
Append the current input character to the comment token's data.
13.2.5.42 Markup declaration open state

If the next few characters are:

Two U+002D HYPHEN-MINUS characters (-)
Consume those two characters, create a comment token whose data is the empty string, and switch to the comment start state.
ASCII case-insensitive match for the word "DOCTYPE"
Consume those characters and switch to the DOCTYPE state.
The string "[CDATA[" (the five uppercase letters "CDATA" with a U+005B LEFT SQUARE BRACKET character before and after)
Consume those characters. If there is an adjusted current node and it is not an element in the HTML namespace, then switch to the CDATA section state. Otherwise, this is a cdata-in-html-content parse error. Create a comment token whose data is the "[CDATA[" string. Switch to the bogus comment state.
Anything else
This is an incorrectly-opened-comment parse error. Create a comment token whose data is the empty string. Switch to the bogus comment state (don't consume anything in the current state).
13.2.5.43 Comment start state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the comment start dash state.
U+003E GREATER-THAN SIGN (>)
This is an abrupt-closing-of-empty-comment parse error. Switch to the data state. Emit the current comment token.
Anything else
Reconsume in the comment state.
13.2.5.44 Comment start dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the comment end state.
U+003E GREATER-THAN SIGN (>)
This is an abrupt-closing-of-empty-comment parse error. Switch to the data state. Emit the current comment token.
EOF
This is an eof-in-comment parse error. Emit the current comment token. Emit an end-of-file token.
Anything else
Append a U+002D HYPHEN-MINUS character (-) to the comment token's data. Reconsume in the comment state.
13.2.5.45 Comment state

Consume the next input character:

U+003C LESS-THAN SIGN (<)
Append the current input character to the comment token's data. Switch to the comment less-than sign state.
U+002D HYPHEN-MINUS (-)
Switch to the comment end dash state.
U+0000 NULL
This is an unexpected-null-character parse error. Append a U+FFFD REPLACEMENT CHARACTER character to the comment token's data.
EOF
This is an eof-in-comment parse error. Emit the current comment token. Emit an end-of-file token.
Anything else
Append the current input character to the comment token's data.
13.2.5.46 Comment less-than sign state

Consume the next input character:

U+0021 EXCLAMATION MARK (!)
Append the current input character to the comment token's data. Switch to the comment less-than sign bang state.
U+003C LESS-THAN SIGN (<)
Append the current input character to the comment token's data.
Anything else
Reconsume in the comment state.
13.2.5.47 Comment less-than sign bang state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the comment less-than sign bang dash state.
Anything else
Reconsume in the comment state.
13.2.5.48 Comment less-than sign bang dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the comment less-than sign bang dash dash state.
Anything else
Reconsume in the comment end dash state.
13.2.5.49 Comment less-than sign bang dash dash state

Consume the next input character:

U+003E GREATER-THAN SIGN (>)
EOF
Reconsume in the comment end state.
Anything else
This is a nested-comment parse error. Reconsume in the comment end state.
13.2.5.50 Comment end dash state

Consume the next input character:

U+002D HYPHEN-MINUS (-)
Switch to the comment end state.
EOF
This is an eof-in-comment parse error. Emit the current comment token. Emit an end-of-file token.
Anything else
Append a U+002D HYPHEN-MINUS character (-) to the comment token's data. Reconsume in the comment state.
13.2.5.51 Comment end state

Consume the next input character:

U+003E GREATER-THAN SIGN (>)
Switch to the