URL

To percent-encode a byte byte, return a string consisting of U+0025 (%), followed by two ASCII upper hex digits representing byte.

To percent-decode a byte sequence input, run these steps:

Using anything but UTF-8 decode without BOM when input contains bytes that are not ASCII bytes might be insecure and is not recommended.

1. Let output be an empty byte sequence.

2. For each byte byte in input:

   1. If byte is not 0x25 (%), then append byte to output.

   2. Otherwise, if byte is 0x25 (%) and the next two bytes after byte in input are not in the ranges 0x30 (0) to 0x39 (9), 0x41 (A) to 0x46 (F), and 0x61 (a) to 0x66 (f), all inclusive, append byte to output.

   3. Otherwise:

      1. Let bytePoint be the two bytes after byte in input, decoded, and then interpreted as a hexadecimal number.

      2. Append a byte whose value is bytePoint to output.

      3. Skip the next two bytes in input.

3. Return output.

To percent-decode a scalar value string input:

1. Let bytes be the UTF-8 encoding of input.

2. Return the percent-decoding of bytes.

In general, percent-encoding results in a string with more U+0025 (%) code points than the input, and percent-decoding results in a byte sequence with less 0x25 (%) bytes than the input.

To percent-encode after encoding, given an encoding encoding, scalar value string input, and a percent-encode set percentEncodeSet:

1. Assert: encoding is UTF-8 or percentEncodeSet is special-query percent-encode set or application/x-www-form-urlencoded percent-encode set.

2. Let spaceAsPlus be true if percentEncodeSet is application/x-www-form-urlencoded percent-encode set; otherwise false.

3. Let encoder be the result of getting an encoder from encoding.

4. Let inputQueue be input converted to an I/O queue.

5. Let output be the empty string.

6. Let potentialError be 0.

   This needs to be a non-null value to initiate the subsequent while loop.

7. While potentialError is non-null:

   1. Let encodeOutput be an empty I/O queue.

   2. Set potentialError to the result of running encode or fail with inputQueue, encoder, and encodeOutput.

   3. For each byte of encodeOutput converted to a byte sequence:

      1. If spaceAsPlus is true and byte is 0x20 (SP), then append U+002B (+) to output and continue.

      2. Let isomorph be a code point whose value is byte’s value.

      3. Assert: percentEncodeSet includes all non-ASCII code points.

      4. If isomorph is not in percentEncodeSet, then append isomorph to output.

      5. Otherwise, percent-encode byte and append the result to output.

   4. If potentialError is non-null, then append "%26%23", followed by the shortest sequence of ASCII digits representing potentialError in base ten, followed by "%3B", to output.

      This can happen when encoding is not UTF-8.

8. Return output.

Of the possible values for the percentEncodeSet argument only two end up encoding U+0025 (%) and thus give “roundtripable data”: component percent-encode set and application/x-www-form-urlencoded percent-encode set. The other values for the percentEncodeSet argument — which happen to be used by the URL parser — leave U+0025 (%) untouched and as such it needs to be percent-encoded first in order to be properly represented.

To UTF-8 percent-encode a scalar value scalarValue using a percentEncodeSet, return the result of running percent-encode after encoding with UTF-8, scalarValue as a string, and percentEncodeSet.

To UTF-8 percent-encode a scalar value string input using a percentEncodeSet, return the result of running percent-encode after encoding with UTF-8, input, and percentEncodeSet.

To obtain the public suffix of a host host, run these steps. They return null or a domain representing a portion of host that is included on the Public Suffix List. [PSL]

1. If host is not a domain, then return null.

2. Let trailingDot be "." if host ends with "."; otherwise the empty string.

3. Let publicSuffix be the public suffix determined by running the Public Suffix List algorithm with host as domain. [PSL]

4. Assert: publicSuffix is an ASCII string that ends with trailingDot.

5. Return publicSuffix.

To obtain the registrable domain of a host host, run these steps. They return null or a domain formed by host’s public suffix and the domain label preceding it, if any.

1. If host’s public suffix is null or host’s public suffix equals host, then return null.

2. Let trailingDot be "." if host ends with "."; otherwise the empty string.

3. Let registrableDomain be the registrable domain determined by running the Public Suffix List algorithm with host as domain. [PSL]

4. Assert: registrableDomain is an ASCII string that ends with trailingDot.

5. Return registrableDomain.

The domain parser algorithm, given a scalar value string domain and a boolean beStrict, runs these steps. They return failure or a domain.

1. Let strictResult be the result of running domain parser ToASCII with domain and true.

2. If strictResult is a failure value, domain-to-ASCII validation error. This step does not return.

3. If beStrict is true:

   1. If strictResult is a failure value, then return failure.

   2. Return strictResult.

4. Let result be null.

5. If domain is an ASCII string, then set result to domain, lowercased.

   When beStrict is false and domain is an ASCII string, the algorithm returns domain lowercased regardless of Unicode ToASCII’s outcome, due to web compatibility. IgnoreInvalidPunycode is not sufficient on its own, as Punycode can decode successfully yet still fail validity criteria. E.g., xn--8i7caa decodes to ｗｗｗ, whose code points have status "mapped". [UTS46]

6. Otherwise:

   1. Set result to the result of running domain parser ToASCII with domain and false.

   2. If result is a failure value, then return failure.

7. If result is the empty string, then return failure.

8. If result contains a forbidden domain code point, then return failure.

   Due to web compatibility and compatibility with non-DNS-based systems the forbidden domain code points are a subset of those disallowed when UseSTD3ASCIIRules is true. See also issue #397.

9. Return result.

This document and the web platform at large use Unicode IDNA Compatibility Processing and not IDNA2008. For instance, ☕.example becomes xn--53h.example and not failure. [UTS46] [RFC5890]

The domain parser ToASCII algorithm, given a scalar value string domain and a boolean beStrict, returns the result of running Unicode ToASCII with domain_name set to domain, CheckHyphens set to beStrict, CheckBidi set to true, CheckJoiners set to true, UseSTD3ASCIIRules set to beStrict, Transitional_Processing set to false, VerifyDnsLength set to beStrict, and IgnoreInvalidPunycode set to false. [UTS46]

The domain to Unicode algorithm, given a domain domain, runs these steps:

1. Let result be the result of running Unicode ToUnicode with domain_name set to domain, CheckHyphens set to false, CheckBidi set to true, CheckJoiners set to true, UseSTD3ASCIIRules set to false, Transitional_Processing set to false, and IgnoreInvalidPunycode set to false. [UTS46]

2. If an error was recorded, then return domain.

   Because domain can only result from the host parser, any recorded errors will already have been signified as validation errors. Returning domain ensures the domain parser and domain to Unicode roundtrip on input such as xn--8i7caa.

3. Return result.

The host parser takes a scalar value string input with an optional boolean isOpaque (default false), and then runs these steps. They return failure or a host.

1. If input starts with U+005B ([), then:

   1. If input does not end with U+005D (]), IPv6-unclosed validation error, return failure.

   2. Return the result of IPv6 parsing input with its leading U+005B ([) and trailing U+005D (]) removed.

2. If isOpaque is true, then return the result of opaque-host parsing input.

3. Assert: input is not the empty string.

4. If input contains a percent-encoded byte, domain-percent-encoded validation error.

5. Let domain be the result of running UTF-8 decode without BOM on the percent-decoding of input.

   Alternatively UTF-8 decode without BOM or fail can be used, coupled with an early return for failure, as the domain parser fails on U+FFFD (�).

6. Let asciiDomain be the result of running domain parser with domain and false.

7. If asciiDomain is failure, then return failure.

8. If asciiDomain ends in a number:

   1. If domain is not an ASCII string, IPv4-non-ASCII-input validation error.

   2. Return the result of IPv4 parsing asciiDomain.

9. Return asciiDomain.

The ends in a number checker takes an ASCII string input and then runs these steps. They return a boolean.

1. Assert: input is not the empty string.

2. Let parts be the result of strictly splitting input on U+002E (.).

3. If the last item in parts is the empty string, then remove the last item from parts.

4. Let last be the last item in parts.

5. If last is not the empty string and contains only ASCII digits, then return true.

   The erroneous input "09" will be caught by the IPv4 parser at a later stage.

6. If parsing last as an IPv4 number does not return failure, then return true.

   This is equivalent to checking that last is "0X" or "0x", followed by zero or more ASCII hex digits.

7. Return false.

The IPv4 parser takes an ASCII string input and then runs these steps. They return failure or an IPv4 address.

The IPv4 parser is not to be invoked directly. Instead check that the return value of the host parser is an IPv4 address.

1. Let parts be the result of strictly splitting input on U+002E (.).

2. If the last item in parts is the empty string, then:

   1. IPv4-empty-part validation error.

   2. If parts’s size is greater than 1, then remove the last item from parts.

3. If parts’s size is less than 4, IPv4-too-few-parts validation error.

4. If parts’s size is greater than 4, IPv4-too-many-parts validation error, return failure.

5. Let numbers be an empty list.

6. For each part of parts:

   1. Let result be the result of parsing part.

   2. If result is failure, IPv4-non-numeric-part validation error, return failure.

   3. If result[1] is true, IPv4-non-decimal-part validation error.

   4. Append result[0] to numbers.

7. If any item in numbers is greater than 255, IPv4-out-of-range-part validation error.

8. If any but the last item in numbers is greater than 255, then return failure.

9. If the last item in numbers is greater than or equal to 256^{(5 − numbers’s size)}, then return failure.

10. Let ipv4 be the last item in numbers.

11. Remove the last item from numbers.

12. Let counter be 0.

13. For each n of numbers:

    1. Increment ipv4 by n × 256^{(3 − counter)}.

    2. Increment counter by 1.

14. Return ipv4.

The IPv4 number parser takes an ASCII string input and then runs these steps. They return failure or a tuple of a number and a boolean.

1. If input is the empty string, then return failure.

2. Let validationError be false.

3. Let R be 10.

4. If input contains at least two code points and the first two code points are either "0X" or "0x", then:

   1. Set validationError to true.

   2. Remove the first two code points from input.

   3. Set R to 16.

5. Otherwise, if input contains at least two code points and the first code point is U+0030 (0), then:

   1. Set validationError to true.

   2. Remove the first code point from input.

   3. Set R to 8.

6. If input is the empty string, then return (0, true).

7. If input contains a code point that is not a radix-R digit, then return failure.

8. Let output be the mathematical integer value that is represented by input in radix-R notation, using ASCII hex digits for digits with values 0 through 15.

9. Return (output, validationError).

The IPv6 parser takes a scalar value string input and then runs these steps. They return failure or an IPv6 address.

The IPv6 parser could in theory be invoked directly, but please discuss actually doing that with the editors of this document first.

1. Let address be a new IPv6 address whose pieces are all 0.

2. Let pieceIndex be 0.

3. Let compress be null.

4. Let pointer be a pointer for input.

5. If c is U+003A (:), then:

   1. If remaining does not start with U+003A (:), IPv6-invalid-compression validation error, return failure.

   2. Increase pointer by 2.

   3. Increase pieceIndex by 1 and then set compress to pieceIndex.

6. While c is not the EOF code point:

   1. If pieceIndex is 8, IPv6-too-many-pieces validation error, return failure.

   2. If c is U+003A (:), then:

      1. If compress is non-null, IPv6-multiple-compression validation error, return failure.

      2. Increase pointer and pieceIndex by 1, set compress to pieceIndex, and then continue.

   3. Let value and length be 0.

   4. While length is less than 4 and c is an ASCII hex digit, set value to value × 0x10 + c interpreted as a hexadecimal number, and increase pointer and length by 1.

   5. If c is U+002E (.), then:

      1. If length is 0, IPv4-in-IPv6-invalid-code-point validation error, return failure.

      2. Decrease pointer by length.

      3. If pieceIndex is greater than 6, IPv4-in-IPv6-too-many-pieces validation error, return failure.

      4. Let numbersSeen be 0.

      5. While c is not the EOF code point:

         1. Let ipv4Piece be null.

         2. If numbersSeen is greater than 0, then:

            1. If c is a U+002E (.) and numbersSeen is less than 4, then increase pointer by 1.

            2. Otherwise, IPv4-in-IPv6-invalid-code-point validation error, return failure.

         3. If c is not an ASCII digit, IPv4-in-IPv6-invalid-code-point validation error, return failure.

         4. While c is an ASCII digit:

            1. Let number be c interpreted as decimal number.

            2. If ipv4Piece is null, then set ipv4Piece to number.

            3. Otherwise, if ipv4Piece is 0, IPv4-in-IPv6-invalid-code-point validation error, return failure.

            4. Otherwise, set ipv4Piece to ipv4Piece × 10 + number.

            5. If ipv4Piece is greater than 255, IPv4-in-IPv6-out-of-range-part validation error, return failure.

            6. Increase pointer by 1.

         5. Set address[pieceIndex] to address[pieceIndex] × 0x100 + ipv4Piece.

         6. Increase numbersSeen by 1.

         7. If numbersSeen is 2 or 4, then increase pieceIndex by 1.

      6. If numbersSeen is not 4, IPv4-in-IPv6-too-few-parts validation error, return failure.

      7. Break.

   6. Otherwise, if c is U+003A (:):

      1. Increase pointer by 1.

      2. If c is the EOF code point, IPv6-invalid-code-point validation error, return failure.

   7. Otherwise, if c is not the EOF code point, IPv6-invalid-code-point validation error, return failure.

   8. If length is greater than 1 and value is less than 0x10^{length − 1}, IPv6-piece-leading-zero validation error.

   9. Set address[pieceIndex] to value.

   10. Increase pieceIndex by 1.

7. If compress is non-null, then:

   1. Let swaps be pieceIndex − compress.

   2. Set pieceIndex to 7.

   3. While pieceIndex is not 0 and swaps is greater than 0, swap address[pieceIndex] with address[compress + swaps − 1], and then decrease both pieceIndex and swaps by 1.

8. Otherwise, if compress is null and pieceIndex is not 8, IPv6-too-few-pieces validation error, return failure.

9. Return address.

The opaque-host parser takes a scalar value string input, and then runs these steps. They return failure or an opaque host.

1. If input contains a forbidden host code point, host-invalid-code-point validation error, return failure.

2. If input contains a code point that is not a URL code point and not U+0025 (%), invalid-URL-unit validation error.

3. If input contains a U+0025 (%) and the two code points following it are not ASCII hex digits, invalid-URL-unit validation error.

4. Return the result of running UTF-8 percent-encode on input using the C0 control percent-encode set.

The host serializer takes a host host and then runs these steps. They return an ASCII string.

1. If host is an IPv4 address, return the result of running the IPv4 serializer on host.

2. Otherwise, if host is an IPv6 address, return U+005B ([), followed by the result of running the IPv6 serializer on host, followed by U+005D (]).

3. Otherwise, host is a domain, opaque host, or empty host, return host.

The IPv4 serializer takes an IPv4 address address and then runs these steps. They return an ASCII string.

1. Let output be the empty string.

2. Let n be the value of address.

3. For each i in the range 1 to 4, inclusive:

   1. Prepend n % 256, serialized, to output.

   2. If i is not 4, then prepend U+002E (.) to output.

   3. Set n to floor(n / 256).

4. Return output.

The IPv6 serializer takes an IPv6 address address and then runs these steps. They return an ASCII string.

1. Let output be the empty string.

2. Let compress be the result of finding the IPv6 address compressed piece index given address.

3. Let ignore0 be false.

4. For each pieceIndex of address’s pieces’s indices:

   1. If ignore0 is true and address[pieceIndex] is 0, then continue.

   2. Otherwise, if ignore0 is true, set ignore0 to false.

   3. If compress is pieceIndex, then:

      1. Let separator be "::" if pieceIndex is 0; otherwise U+003A (:).

      2. Append separator to output.

      3. Set ignore0 to true and continue.

   4. Append address[pieceIndex], represented as the shortest possible lowercase hexadecimal number, to output.

   5. If pieceIndex is not 7, then append U+003A (:) to output.

5. Return output.

This algorithm requires the recommendation from A Recommendation for IPv6 Address Text Representation. [RFC5952]

To find the IPv6 address compressed piece index given an IPv6 address address:

1. Let longestIndex be null.

2. Let longestSize be 1.

3. Let foundIndex be null.

4. Let foundSize be 0.

5. For each pieceIndex of address’s pieces’s indices:

   1. If address’s pieces[pieceIndex] is not 0:

      1. If foundSize is greater than longestSize, then set longestIndex to foundIndex and longestSize to foundSize.

      2. Set foundIndex to null.
      3. Set foundSize to 0.

   2. Otherwise:

      1. If foundIndex is null, then set foundIndex to pieceIndex.

      2. Increment foundSize by 1.

6. If foundSize is greater than longestSize, then return foundIndex.

7. Return longestIndex.

In 0:f:0:0:f:f:0:0 it would point to the second 0.

To determine whether a host A equals host B, return true if A is B, and false otherwise.