note 103051 added to function.strtok

If you have memory-usage critical solution, you should keep in mind, that strtok function holds input string parameter (or reference to it?) in memory after usage. <?php function tokenize($str, $token_symbols) { $word = strtok($str, $token_symbols); while (false !== $word) { // do something here... $word = strtok($token_symbols); } } ?> Test-cases with handling ~10MB plain-text file: Case #1 - unset $str variable <?php $token_symbols = " \t\n"; $str = file_get_contents('10MB.txt'); // mem usage 9.75383758545 MB (memory_get_usage() / 1024 / 1024)); tokenize($str, $token_symbols); // mem usage 9.75400161743 MB unset($str); // 9.75395584106 MB ?> Case #1 result: memory is still used Case #2 - call strtok again <?php $token_symbols = " \t\n"; $str = file_get_contents('10MB.txt'); // 9.75401306152 MB tokenize($str, $token_symbols); // 9.75417709351 strtok('', ''); // 9.75421524048 ?> Case #2 result: memory is still used Case #3 - call strtok again AND unset $str variable <?php $token_symbols = " \t\n"; $str = file_get_contents('10MB.txt'); // 9.75410079956 MB tokenize($str, $token_symbols); // 9.75426483154 MB unset($str); strtok('', ''); // 0.0543975830078 MB ?> Case #3 result: memory is free So, better solution for tokenize function: <?php function tokenize($str, $token_symbols, $token_reset = true) { $word = strtok($str, $token_symbols); while (false !== $word) { // do something here... $word = strtok($token_symbols); } if($token_reset) strtok('', ''); } ?> ---- Server IP: 217.146.68.100 Probable Submitter: 82.131.4.229 ---- Manual Page -- http://www.php.net/manual/en/function.strtok.php Edit -- https://master.php.net/note/edit/103051 Del: integrated -- https://master.php.net/note/delete/103051/integrated Del: useless -- https://master.php.net/note/delete/103051/useless Del: bad code -- https://master.php.net/note/delete/103051/bad+code Del: spam -- https://master.php.net/note/delete/103051/spam Del: non-english -- https://master.php.net/note/delete/103051/non-english Del: in docs -- https://master.php.net/note/delete/103051/in+docs Del: other reasons-- https://master.php.net/note/delete/103051 Reject -- https://master.php.net/note/reject/103051 Search -- https://master.php.net/manage/user-notes.php