Re: What "Unicode support" really means

Andrea Faulds wrote:

I’m unconvinced that dealing with printed characters instead of codepoints is a good idea generally. It’s necessary when doing certain types of text manipulation, but I don’t think we should do this for substr() and general string manipulation. You can’t always necessarily split a group code points into distinct characters, where a code point joins two characters together visually, for example.

I've flagged up before that there are at least three string lengths. 1 - Number of bytes 2 - Number of codepoints 3 - Number of glyphs But the answer to those figures also depend on other factors :( Initially I thought that for string manipulation switching to UTF32 was a no-brainer, but even that has problems when a code point can actually have a number of different transforms and I am not sure that working with an array of codepoints is actually any easier than simply using the underlying UTF-8 string of bytes? My comment about a 'simple' UTF-8 locale for the core strings was intended to provide a subset of 'characters' that do not involve these multiple codepoint/glyph problems and I include the case wrapping problems with that so that the processing of 'metadata' can be kept as clean as possible. So the more complex 'locale' support can be kept to only being used in 'content' in much the same way that mbstring does work now? -- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk