Re: [RFC] UString

On 28/02/2015 06:48, Joe Watkins wrote:

Morning internals,

     This is just a quick note to announce my intention to ready this RFC

for voting next week.

     I know I'm a little late maybe, I was real sick most of last week, so

couldn't do anything useful.

     A couple of us intend to fix outstanding issues on github and those

raised here, tidy the RFC and open the vote for 7.

    I would ask anyone interested to scan through this thread and announce

concerns that are not mentioned asap.

I still think this class is trying to do several jobs, and not doing any of them very well, and I fear that people will see this class and expect it to solve problems which it actually ignores. Here are some concrete use cases I would like a simple interface to solve for me: - Take text from an ISO 88592-2 data source, pass it through generic text filters, and pass it to a UTF-16 data target. - Given a long string of Unicode text, give me a valid UTF-8 string which fits into a buffer with fixed byte size; i.e. give me the largest number of whole code points which fit into that number of bytes once encoded. - As above, but without stripping diacritics off the last character of the resulting string, i.e. give me the largest number of whole graphemes which fit. - Split a string into equal sized chunks of readable characters (graphemes), regardless of how many bytes or code points each chunk contains. UString currently falls short of all of these: - I can specify my input encoding (in the constructor or helper method, over-riding a static default, which is equivalent to ext/mbstring's global setting), but not my output encoding (there is no method to ask for a byte representation other than a string cast, which by definition has no parameters). - I can ask for a fixed number of code points, but don't know how many bytes these will take until I cast to a UTF-8 string. - I can't manipulate anything at the grapheme level at all, even though this is the most meaningful level of operation in most cases. Things it does do: - a handful of methods give meaningful international text support: toUpper(), toLower(), trim() - some methods could be done on byte strings if I ensure they're all in UTF-8: replace(), contains(), startsWith(), endsWith(), repeat() - there may be limited situations where I want to dive into the code points which make up a string, although I can't think of many: $length, pad(), indexOf(), lastIndexOf(), charAt(), replaceSlice() - remaining methods avoid me creating invalid UTF-8, but don't help me much with real-life text: chunk(), split(), substring() - I can ask what codepage my Unicode string is in; I don't even understand what this means I think an efficient OO wrapper around ICU is a great idea, but more thought needs to go into what methods are exposed, and how people are going to use them in real code. Regards, -- Rowan Collins [IMSoP]