com php-langspec: Define encoding of PHP scripts: spec/05-types.md spec/09-lexical-structure.md spec/10-expressions.md spec/19-grammar.md
tests/lexical_structure/unicode_string_escape_sequence/ unicode_escape.phpt

Commit: f35bd70531c012544e9061731415be8dcfe77f32 Author: agares <ja@agares.info> Thu, 21 Feb 2019 19:29:59 +0100 Committer: Nikita Popov <nikita.ppv@gmail.com> Fri, 15 Mar 2019 10:08:58 +0100 Parents: 15aa45273f4e5533ed252a612a5e6e7349a4d3ec Branches: master Link: http://git.php.net/?p=php-langspec.git;a=commitdiff;h=f35bd70531c012544e9061731415be8dcfe77f32 Log: Define encoding of PHP scripts 1. Remove references to unicode codepoints (as those were ambigouous without a defined encoding) and replace them with byte values 2. Define the encoding of PHP scripts as ASCII, where bytes greater than 0x7F are allowed Changed paths: M spec/05-types.md M spec/09-lexical-structure.md M spec/10-expressions.md M spec/19-grammar.md M tests/lexical_structure/unicode_string_escape_sequence/unicode_escape.phpt

diff --git a/spec/05-types.md b/spec/05-types.md index 11cfb3b..703da4f 100644 --- a/spec/05-types.md +++ b/spec/05-types.md @@ -128,10 +128,10 @@ str-whitespace:: str-whitespace-char:: new-line - "Space character (U+0020)" - "Horizontal-tab character (U+0009)" - "Vertical-tab character (U+000B)" - "Form-feed character (U+000C)" + "Space character (0x20)" + "Horizontal-tab character (0x09)" + "Vertical-tab character (0x0B)" + "Form-feed character (0x0C)" str-number:: digit-sequence @@ -147,10 +147,10 @@ str-number:: <i id="grammar-str-whitespace-char">str-whitespace-char::</i> <i><a href="09-lexical-structure.md#grammar-new-line">new-line</a></i> - Space character (U+0020) - Horizontal-tab character (U+0009) - Vertical-tab character (U+000B) - Form-feed character (U+000C) + Space character (0x20) + Horizontal-tab character (0x09) + Vertical-tab character (0x0B) + Form-feed character (0x0C) <i id="grammar-str-number">str-number::</i> <i><a href="09-lexical-structure.md#grammar-digit-sequence">digit-sequence</a></i> diff --git a/spec/09-lexical-structure.md b/spec/09-lexical-structure.md index 262b304..f4ec536 100644 --- a/spec/09-lexical-structure.md +++ b/spec/09-lexical-structure.md @@ -145,9 +145,9 @@ input-character:: "Any source character except" new-line new-line:: - "Carriage-return character (U+000D)" - "Line-feed character (U+000A)" - "Carriage-return character (U+000D) followed by line-feed character (U+000A)" + "Carriage-return character (0x0D)" + "Line-feed character (0x0A)" + "Carriage-return character (0x0D) followed by line-feed character (0x0A)" delimited-comment:: '/*' "No characters or any source character sequence except */" '*/' @@ -170,9 +170,9 @@ delimited-comment:: Any source character except <i><a href="#grammar-new-line">new-line</a></i> <i id="grammar-new-line">new-line::</i> - Carriage-return character (U+000D) - Line-feed character (U+000A) - Carriage-return character (U+000D) followed by line-feed character (U+000A) + Carriage-return character (0x0D) + Line-feed character (0x0A) + Carriage-return character (0x0D) followed by line-feed character (0x0A) <i id="grammar-delimited-comment">delimited-comment::</i> /* No characters or any source character sequence except */ */ @@ -212,8 +212,8 @@ white-space:: white-space-character:: new-line - "Space character (U+0020)" - "Horizontal-tab character (U+0009)" + "Space character (0x20)" + "Horizontal-tab character (0x09)" --> <pre> @@ -223,8 +223,8 @@ white-space-character:: <i id="grammar-white-space-character">white-space-character::</i> <i><a href="#grammar-new-line">new-line</a></i> - Space character (U+0020) - Horizontal-tab character (U+0009) + Space character (0x20) + Horizontal-tab character (0x09) </pre> **Semantics** @@ -290,7 +290,7 @@ name:: name-nondigit:: nondigit - "one of the characters U+0080–U+00ff" + "one of the characters 0x80–0xff" nondigit:: one of '_' @@ -324,7 +324,7 @@ nondigit:: one of <i id="grammar-name-nondigit">name-nondigit::</i> <i><a href="#grammar-nondigit">nondigit</a></i> - one of the characters U+0080–U+00ff + one of the characters 0x80–0xff <i id="grammar-nondigit">nondigit:: one of</i> _ @@ -344,7 +344,7 @@ Names are used to identify the following: [constants](06-constants.md#general), and names in [heredoc](#heredoc-string-literals) and [nowdoc comments](#nowdoc-string-literals). A *name* begins with an underscore (_), *name-nondigit*, or extended -name character in the range U+0080–-U+00ff. Subsequent characters can +name character in the range 0x80–-0xff. Subsequent characters can also include *digits*. A *variable name* is a name with a leading dollar ($). @@ -704,7 +704,7 @@ b-prefix:: one of **Semantics** A single-quoted string literal is a string literal delimited by -single-quotes (`'`, U+0027). The literal can contain any source character except +single-quotes (`'`, 0x27). The literal can contain any source character except single-quote (`'`) and backslash (`\\`), which can only be represented by their corresponding escape sequence. @@ -807,7 +807,7 @@ codepoint-digits:: **Semantics** A double-quoted string literal is a string literal delimited by -double-quotes (`"`, U+0022). The literal can contain any source character except +double-quotes (`"`, 0x22). The literal can contain any source character except double-quote (`"`) and backslash (`\\`), which can only be represented by their corresponding escape sequence. Certain other (and sometimes non-printable) characters can also be expressed as escape sequences. @@ -821,15 +821,15 @@ in the table below: Escape sequence | Character name | Unicode character --------------- | --------------| ------ -\$ | Dollar sign | U+0024 -\" | Double quote | U+0022 -\\ | Backslash | U+005C -\e | Escape | U+001B -\f | Form feed | U+000C -\n | New line | U+000A -\r | Carriage Return | U+000D -\t | Horizontal Tab | U+0009 -\v | Vertical Tab | U+000B +\$ | Dollar sign | 0x24 +\" | Double quote | 0x22 +\\ | Backslash | 0x5C +\e | Escape | 0x1B +\f | Form feed | 0x0C +\n | New line | 0x0A +\r | Carriage Return | 0x0D +\t | Horizontal Tab | 0x09 +\v | Vertical Tab | 0x0B \ooo | 1–3-digit octal digit value ooo \xhh or \Xhh | 1–2-digit hexadecimal digit value hh \u{xxxxxx} | UTF-8 encoding of Unicode codepoint U+xxxxxx | U+xxxxxx diff --git a/spec/10-expressions.md b/spec/10-expressions.md index 8c8af69..7184f31 100644 --- a/spec/10-expressions.md +++ b/spec/10-expressions.md @@ -1428,7 +1428,7 @@ shell-command-expression: ` <i><a href="09-lexical-structure.md#grammar-dq-char-sequence">dq-char-sequence</a></i><sub>opt</sub> ` </pre> -where \` is the GRAVE ACCENT character U+0060, commonly referred to as a +where \` is the GRAVE ACCENT character 0x60, commonly referred to as a *backtick*. **Semantics** @@ -2804,9 +2804,9 @@ character from the right-hand operand is stored at the designated location; all other characters in the right-hand operand string are ignored. If the designated location is beyond the end of the destination string, that string is extended to the new length with -spaces (U+0020) added as padding beyond the old end and before the newly +spaces (0x20) added as padding beyond the old end and before the newly added character. If the right-hand operand is an empty string, the null -character \\0 (U+0000) is stored. +character \\0 (0x00) is stored. **Examples** diff --git a/spec/19-grammar.md b/spec/19-grammar.md index 0cc4f68..6bdb90c 100644 --- a/spec/19-grammar.md +++ b/spec/19-grammar.md @@ -3,6 +3,9 @@ ## General The grammar notation is described in [Grammars section](09-lexical-structure.md#grammars). +PHP scripts are encoded as ASCII, but bytes 0x80 to 0xFF are allowed in some places, as defined in the grammar. +PHP scripts are parsed as a series of 8-bit bytes, rather than code points from Unicode or any other character repertoire. +Within this specification, bytes are represented by their ASCII interpretations where these are printable characters. ## Lexical Grammar @@ -32,9 +35,9 @@ The grammar notation is described in [Grammars section](09-lexical-structure.md# Any source character except <i><a href="#grammar-new-line">new-line</a></i> <i id="grammar-new-line">new-line::</i> - Carriage-return character (U+000D) - Line-feed character (U+000A) - Carriage-return character (U+000D) followed by line-feed character (U+000A) + Carriage-return character (0x0D) + Line-feed character (0x0A) + Carriage-return character (0x0D) followed by line-feed character (0x0A) <i id="grammar-delimited-comment">delimited-comment::</i> /* No characters or any source character sequence except */ */ @@ -45,8 +48,8 @@ The grammar notation is described in [Grammars section](09-lexical-structure.md# <i id="grammar-white-space-character">white-space-character::</i> <i><a href="#grammar-new-line">new-line</a></i> - Space character (U+0020) - Horizontal-tab character (U+0009) + Space character (0x20) + Horizontal-tab character (0x09) <i id="grammar-token">token::</i> <i><a href="#grammar-variable-name">variable-name</a></i> @@ -80,7 +83,7 @@ The grammar notation is described in [Grammars section](09-lexical-structure.md# <i id="grammar-name-nondigit">name-nondigit::</i> <i><a href="#grammar-nondigit">nondigit</a></i> - one of the characters U+0080–U+00ff + one of the characters 0x80–0xff <i id="grammar-nondigit">nondigit:: one of</i> _ diff --git a/tests/lexical_structure/unicode_string_escape_sequence/unicode_escape.phpt b/tests/lexical_structure/unicode_string_escape_sequence/unicode_escape.phpt index d729c63..cc19773 100644 --- a/tests/lexical_structure/unicode_string_escape_sequence/unicode_escape.phpt +++ b/tests/lexical_structure/unicode_string_escape_sequence/unicode_escape.phpt @@ -3,7 +3,7 @@ PHP Spec test generated from ./lexical_structure/unicode_string_escape_sequence/ --FILE-- <?php -var_dump("\u{61}"); // ASCII "a" - characters below U+007F just encode as ASCII, as it's UTF-8 +var_dump("\u{61}"); // ASCII "a" - characters below 0x7F just encode as ASCII, as it's UTF-8 var_dump("\u{FF}"); // y with diaeresis var_dump("\u{ff}"); // case-insensitive var_dump("\u{2603}"); // Unicode snowman