I'm not sure why you consider a formal definition like
LABEL [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*
not a definition. Personally I find this a simpler definition (and easier
to implement in a parser) than something like
UTF-8, UAX31-R1-2 with the standard Default-Ignorable Exclusion
Profile
You are right and I overreached. It is a definition, it is precise, and it
is far simpler to implement -- one character class against roughly 13.7 KiB
of generated tables. That is a real cost and I should not have written as
if PHP had nothing.
What I should have said is that it is a complete definition of *bytes* and
a silent one about *characters*. It fixes no encoding, so the same file is
two different programs depending on how you read it, and it cannot tell a
character from half of one.
The part that actually bothers me is that PHP does not stay at the byte
level. Case-insensitive symbol lookup is defined over characters -- ASCII
characters. So there is already one character-level rule bolted onto a
byte-level definition, and the seam between them is where
Stra<U+00DF>e/STRA<U+00DF>E/Stra<U+1E9E>e/Strasse comes from.
But then again I'm not really using non-ASCII identifiers today.
Then nothing here would ever fire for you. The check I described never
looks at an identifier whose bytes are all below 0x80, which is every
identifier in most codebases. That is not a footnote -- it is why the thing
can be considered at all.
And this is also something an LSP or Linter can help you with if it a real
concern for you.
I answered that at length to Juris and to Rowan and will not repeat it.
Two things I did not know then and do now: the measurement is 68 identifiers
with an invisible character across 520,802 files, and not all of them are
accidents -- math-php puts U+00A0 inside variable names deliberately. That
makes it warning-shaped rather than error-shaped, which is closer to your
position than to my original one.
Case-insensitive folding adds another problem: Would you be using
IntlChar::FOLD_CASE_DEFAULT or IntlChar::FOLD_CASE_EXCLUDE_SPECIAL_I to
fold "I"? Or would you base it on a language setting?
PHP has already answered this, which I think settles it in your favour.
The 8.2 RFC "Locale-independent case conversion" used precisely the Turkish
dotted I as its motivating example: before PHP 8.0 the engine inherited the
system locale, so case folding varied by installation. The resolution was
to make folding ASCII-only everywhere, deliberately and by vote.
So the answer to "which folding" is that PHP tried locale-sensitive folding,
found it unworkable, and retreated. I would not want to walk back into it.
In general I think most people consider the case-folding for identifiers
nowadays to be a bug, not a feature, so I would probably rather try to
reduce than extend it.
Agreed, and I want to be clear I never proposed extending it. I listed it
as evidence that the current rule is incoherent, not as something to grow.
Incoherent is equally an argument for reducing.
Worth saying though that reducing is not cheap either: making class and
function names case-sensitive is a far larger break than anything discussed
in this thread. I have no proposal there, only the observation that the
rule as it stands makes sense only if identifiers are ASCII, and 1,447 of
them in the top 5,000 packages are not.
Regards,
Luca