Re: cvs: pear /SQL_Parser Lexer.php

Brent, If you want to have a look at this, it generates a php tokenizer (like flex from a standard lex like file) http://devel.akbkhome.com/cs2phplex.tgz example usage: (after untarring) .. (needs mono or the Windows .NET Runtime) mono csLex.exe cs-tokenizer.lex will generate cs-tokenizer_lex.php if you just replace the syntax in cs-tokenizer.lex with that for SQL (which might be publicly available?), it might be a bit easier to maintain... Regards Alan Cook wrote:

busterb Fri Dec 27 16:04:43 2002 EDT

 Modified files:                 /pear/SQL_Parser	Lexer.php  Log:

add support for Windows/Mac newlines, update isOper() Index: pear/SQL_Parser/Lexer.php diff -u pear/SQL_Parser/Lexer.php:1.4 pear/SQL_Parser/Lexer.php:1.5 --- pear/SQL_Parser/Lexer.php:1.4 Tue Dec 24 09:37:01 2002 +++ pear/SQL_Parser/Lexer.php Fri Dec 27 16:04:42 2002 @@ -18,7 +18,7 @@ // | Foundation, Inc., 59 Temple Place, Suite 330,Boston,MA 02111-1307 USA| // +----------------------------------------------------------------------+ // -// $Id: Lexer.php,v 1.4 2002/12/24 14:37:01 busterb Exp $ +// $Id: Lexer.php,v 1.5 2002/12/27 21:04:42 busterb Exp $ // include 'SQL/ctype.php'; @@ -81,8 +81,34 @@

    }

    function isCompop($c) {
-        return (($c == '<') || ($c == '>') || ($c == '='));
+        return (($c == '<') || ($c == '>') || ($c == '=') || ($c == '!'));
    }

+

+    function isWhiteSpace($c) {
+        return (($c == ' ') || ($c == "\t") || ($c == "\n") || ($c == "\r"));
+    }

+

+    function skipWhiteSpace() {
+        $c = $this->get();
+        while ($this->isWhiteSpace($c)) {
+            // Handle Unix line endings
+            if ($c == "\n") {
+                ++$this->lineNo;
+                $this->lineBegin = $this->tokPtr;
+            // Handle Mac/Windows line endings
+            } elseif ($c == "\r") {
+                $c = $this->skip;
+                if ($c != "\n")
+                    $this->unget();
+                ++$this->lineNo;
+                $this->lineBegin = $this->tokPtr;
+            }
+            $c = $this->skip();
+            $this->tokLen = 1;
+        }
+        return $c;
+    }

+ // }}} // {{{ lex() @@ -98,15 +124,7 @@

                $this->tokPtr = $this->tokStart;
                $this->tokText = '';
                $this->tokLen = 0;
-                $c = $this->get();
-                while (($c == ' ') || ($c == "\t") || ($c == "\n")) {
-                    if ($c == "\n") {
-                        ++$this->lineNo;
-                        $this->lineBegin = $this->tokPtr;
-                    }
-                    $c = $this->skip();
-                    $this->tokLen = 1;
-                }
+                $c = $this->skipWhiteSpace();
                if (($c == '\'') || ($c == '"')) {
                    $quote = $c;
                    $state = 12;

@@ -315,8 +333,15 @@

            // {{{ State 14: Comment
            case 14:
                $c = $this->skip();
-                if ($c == '\n') {
+                // Handle Unix line endings
+                if ($c == "\n") {
                    $state = 0;
+                // Handle Mac/Windows line endings
+                } elseif ($c == "\r") {
+                    $c = $this->skip();
+                    if ($c != "\n") {
+                        $this->unget();
+                    }
                } else {
                    $state = 14;
                }

-- Can you help out? Need Consulting Services or Know of a Job? http://www.akbkhome.com