Re: write-test.php alternative

Hello Ligaya, Monday, August 7, 2006, 1:48:21 AM, you wrote: > Marcus Boerger wrote: >> Hello Ligaya, >> >> sorry missed it, erm can you sent it as a text file again, please? >> >> marcus >> >> Monday, August 7, 2006, 12:38:27 AM, you wrote: >> >> >>>Ligaya Turmelle wrote: >>> >>>>OK - really hope I did this right. Attached is an alternate for the >>>>current write-test.php qa doc page. The new page has some additional >>>>info and explanations as well as some reorganization of what is already >>>>there. Hopefully it will be a bit easier for beginners. I did the >>>>update and it merged. Let me know if there are any problems. >>>> >>>> >>>>------------------------------------------------------------------------ >>>> >>>>Index: write-test.php >>>>=================================================================== >>>>RCS file: /repository/qaweb/write-test.php,v >>>>retrieving revision 1.7 >>>>diff -u -r1.7 write-test.php >>>>--- write-test.php 11 Jul 2006 22:50:53 -0000 1.7 >>>>+++ write-test.php 18 Jul 2006 05:53:43 -0000 >>>>@@ -18,10 +18,76 @@ >>>> <tr> >>>> <td width="10">&nbsp;</td> >>>> <td width="100%"> >>>>-<p>Writing a test file is very easy if you are used to PHP. >>>>-Here is an actual test file from standard module:</p> >>>>+<h2>phpt Test Basics</h2> >>>>+<ul> >>>>+ <li> >>>>+ <b>So what are phpt tests?</b> >>>>+ <p>A phpt test is a little script used by the php internal and >>>>quality >>>>+assurance teams to test PHP's functionality. It can be used with new releases >>>>to make >>>>+sure they can do all the things that previous releases can, or to help find bugs in >>>>current >>>>+releases. By writing phpt tests you are helping to make PHP stabler.</p> >>>>+ </li> >>>>+ >>>>+ <li> >>>>+ <b>What skills are needed to write a phpt test?</b> >>>>+ <p>All that is really needed to write a phpt test >>>>+is a basic understanding of the PHP language, a text editor, and a way to get the >>>>results >>>>+of your code. That is it. So if you have been writing and running PHP scripts >>>>already - >>>>+you have everything you need.</p> >>>>+ </li> >>>>+ >>>>+ <li> >>>>+ <b>What do you write phpt tests on?</b> >>>>+ <p>Basically you can write a phpt test on one of the various >>>>+php functions available. You can write a test on a basic language function (a >>>>string >>>>+function or an array function) , or a function provided by one of PHP's >>>>numerous extensions >>>>+(a mysql function or a image function or a mcrypt function). You can find out what >>>>functions >>>>+already have phpt tests on them by looking in the <a >>>>href="http://cvs.php.net/viewvc.cgi/">html >>>>+version</a> of the cvs (php-src/ext/standard/tests/ is a good place to start >>>>looking - though >>>>+<i>all</i> the tests currently written are NOT in there). After that - >>>>you >>>>+can pick any function you want. If you want more guidance than that - you can >>>>always ask >>>>+the PHP Quality Assurance Team on their mailing list >>>>+(<a >>>>href="mailto:php-qa@lists.php.net">php-qa@lists.php.net</a>) >>>>where they >>>>+would like you to direct your attentions.</p> >>>>+ </li> >>>>+ >>>>+ <li> >>>>+ <b>How is a phpt test is used?</b> >>>>+ <p>When a test is called by the run-test.php script it takes >>>>various >>>>+parts of the phpt file to name and create a .php file. That .php file is then >>>>executed. The >>>>+output of the .php file is then compared to a different section of the phpt file. >>>>If the output of >>>>+the script "matches" the output provided in the phpt script - it >>>>passes.</p> >>>>+ </li> >>>>+ >>>>+ <li> >>>>+ <b>What should a phpt test do?</b> >>>>+ <p>Basically - it should try and break the PHP function. It >>>>should check not >>>>+only the functions normal parameters, but it should also check edge cases. >>>>Intentionally generating >>>>+an error is allowed.</p> >>>>+ </li> >>>>+</ul> >>>>+ >>>>+<h2>Writing phpt Tests</h2> >>>>+<h3>Naming Conventions</h3> >>>>+<p>Phpt tests follow a very strict naming convention. This is done to easily >>>>identify what each phpt >>>>+test is for. Tests should be named according to the following list: >>>>+ <ul type="none"> >>>>+ <li><i>Tests for bugs</i><br /> >>>>+ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <!-- >>>>Give me some space! --> >>>>+ bug&lt;bugid&gt;.phpt (bug17123.phpt)</li> >>>>+ <li><i>Tests for functions</i><br /> >>>>+ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; >>>><!-- Give me some space! --> >>>>+ &lt;functionname&gt;.phpt (dba_open.phpt)</li> >>>>+ <li><i>General tests for extensions</i><br /> >>>>+ &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; >>>><!-- Give me some space! --> >>>>+ &lt;extname&gt;&lt;no&gt;.phpt >>>>(dba_003.phpt)</li> >>>>+ </ul> >>>>+</p> >>>> >>>>-<h3>ext/standard/tests/strings/strtr.phpt</h3> >>>>+<h3>Basic Format</h3> >>>>+<p>A test must contain the sections TEST, FILE and either EXPECT or EXPECTF >>>>at a minimum. The example >>>>+below illustrates a minimal test.</p> >>>>+<i>ext/standard/tests/strings/strtr.phpt</i> >>>> <pre> >>>> --TEST-- >>>> strtr() function >>>>@@ -35,33 +101,37 @@ >>>> string(32) "# hello All, I sAid hi planet! #" >>>> </pre> >>>> >>>>-<p>As you can see the file is devided into several sections. Below is a >>>>-list of all possible sections:</p> >>>>+<p>As you can see the file is divided into several sections. The TEST >>>>section holds a one line comment >>>>+about the phpt test. The phpt files name is used when generating a .php file. The >>>>FILE section is used >>>>+as the body of the .php file, so don't forget to open and close your php tags. >>>> The EXPECT section is >>>>+the part used as a comparison to see if the test passes. It is a >>>>+good idea to generate output with var_dump() calls.</p> >>>> >>>>+<h3>All Sections</h3> >>>>+A phpt test can have many more parts then just the minimum. In fact some of the >>>>manditory parts have >>>>+alternatives that may be used if the situation warrents it. >>>> <dl> >>>> <dt>--TEST--</dt> >>>> <dd>title of the test. (required)</dd> >>>> >>>> <dt>--SKIPIF--</dt> >>>>-<dd>is condition when to skip this test. (optional)</dd> >>>>+<dd>a condition when to skip this test. (optional)</dd> >>>> >>>> <dt>--POST--</dt> >>>>-<dd>is POST variable passed to test script. (optional)</dd> >>>>+<dd>POST variables to be passed to the test script. (optional)</dd> >>>> >>>> <dt>--GET--</dt> >>>>-<dd>is GET variable passed to test script. (optional)</dd> >>>>+<dd>GET variables to be passed to the test script. (optional)</dd> >>>> >>>> <dt>--INI--</dt> >>>>-<dd>each line contains an ini setting e.g. foo=bar. (optional)</dd> >>>>+<dd>to be used if you need a specific php.ini setting for the test. >>>>+ Each line contains an ini setting e.g. foo=bar. (optional)</dd> >>>> >>>> <dt>--ARGS--</dt> >>>> <dd>a single line defining the arguments passed to php. (optional)</dd> >>>> >>>> <dt>--ENV--</dt> >>>>-<dd>configures the environment to be used ofr php. (optional)</dt> >>>>- >>>>-<dt>--FILE--</dt> >>>>-<dd>the test script. (required)</dd> >>>>+<dd>configures the environment to be used for php. (optional)</dt> >>>> >>>> <dt>--FILEEOF--</dt> >>>> <dd>an alternative to --FILE-- where any trainling line break is omitted. >>>>@@ -101,13 +171,9 @@ >>>> <dd>This is only available in the --FILE-- section. Any part after this line >>>> is not going into the actual test script (see below for more).</dd> >>>> >>>>-<p>A test must at least contain the sections TEST, FILE and either EXPECT >>>>-or EXPECTF. When a test is called run-test.php takes the name from the >>>>-TEST section and writes the FILE section into a ".php" file with the >>>>-same name as the ".phpt" file. This ".php" file will then be >>>>executed >>>>-and its output compared to the contents of the EXPECT section. It is a >>>>-good idea to generate output with var_dump() calls.</p> >>>>+<h2>Examples</h2> >>>> >>>>+<h3>EXPECTF</h3> >>>> <p>/ext/standard/tests/strings/str_shuffle.phpt is a good example for using >>>> EXPECTF instead of EXPECT. From time to time the algorithm used for shuffle >>>> changed and sometimes the machine used to execute the code has influence >>>>@@ -115,8 +181,7 @@ >>>> detectable by %s. Other scan-able forms are %i for integers, %d for numbers >>>> only, %f for floating point values, %c for single characters and %x for >>>> hexadecimal values.</p> >>>>- >>>>-<h3>/ext/standard/tests/strings/str_shuffle.phpt</h3> >>>>+<i>/ext/standard/tests/strings/str_shuffle.phpt</i> >>>> <pre> >>>> --TEST-- >>>> Testing str_shuffle. >>>>@@ -132,12 +197,13 @@ >>>> string(3) "123" >>>> </pre> >>>> >>>>+<h3>EXPECTREGEX</h3> >>>> <p>/ext/standard/tests/strings/strings001.phpt is a good example for using >>>> EXPECTREGEX instead of EXPECT. This test also shows that in EXPECTREGEX >>>> some characters need to be escaped since otherwise they would be >>>> interpreted as a regular expression.</p> >>>> >>>>-<h3>/ext/standard/tests/strings/strings001.phpt</h3> >>>>+<i>/ext/standard/tests/strings/strings001.phpt</i> >>>> <pre> >>>> --TEST-- >>>> Test whether strstr() and strrchr() are binary safe. >>>>@@ -153,6 +219,7 @@ >>>> string\(19\) \" nica\x00turska panica\" >>>> </pre> >>>> >>>>+<h3>SKIPIF</h3> >>>> <p>Some tests depend on modules or functions available only in certain >>>> versions >>>> or they even require minimum version of php or zend. These tests should be >>>> skipped when the requirement cannot be fullfilled. To achieve this you can >>>>@@ -160,7 +227,7 @@ >>>> the SKIPIF section must print out the word "skip" followed by a reason >>>> why >>>> the test should skip.</p> >>>> >>>>-<h3>/ext/exif/tests/exif005.phpt</h3> >>>>+<i>/ext/exif/tests/exif005.phpt</i> >>>> <pre> >>>> --TEST-- >>>> Check for exif_read_data, unusual IFD start >>>>@@ -188,26 +255,16 @@ >>>> <p>Test script and SKIPIF code should be directly written into *.phpt. >>>> However, >>>> it is recommended to use include files when more test scripts depend on the >>>> same SKIPIF code or when certain test files need the same values for some >>>>-input. But no file used by any test should have one of the following >>>>-extensions: ".php", ".log", ".exp", ".out" >>>>or ".diff".</p> >>>>- >>>>-<p>Tests should be named according to the following list: >>>>-<dl> >>>>-<dt>Tests for bugs</dt> >>>>-<dd>bug&lt;bugid&gt;.phpt (bug17123.phpt)</dd> >>>>- >>>>-<dt>Tests for functions</dt> >>>>-<dd>&lt;functionname&gt;.phpt (dba_open.phpt)</dd> >>>>+input. </p> >>>> >>>>-<dt>General tests for extensions</dt> >>>>-<dd>&lt;extname&gt;&lt;no&gt;.phpt (dba_003.phpt)</dd> >>>>-</dl> >>>>-</p> >>>> >>>>-<p>When you use an include file for the SKIPIF section it should be named >>>>-"skipif.inc" and an include file used in the FILE section of many tests >>>>-should be named "test.inc".</p> >>>>+<p> >>>>+<b>Note:</b> no file used by any test should have one of the following >>>>extensions: >>>>+".php", ".log", ".exp", ".out" or >>>>".diff". When you use an include file for the >>>>+SKIPIF section it should be named "skipif.inc" and an include file used >>>>in the >>>>+FILE section of many tests should be named "test.inc".</p> >>>> >>>>+<h2> Final Notes</h2> >>>> <h3>Redirecting tests</h3> >>>> <p>Using --REDIRECTTEST-- it is possible to redirect from one test to a bunch >>>> >>>> of other tests. That way multiple extensions can refer to the same set of >>>>@@ -222,11 +279,9 @@ >>>> </p> >>>> >>>> <p>Redirect tests may especially contain --SKIPIF--, --ENV-- and --ARGS-- >>>>-sections but they no not use any --EXPECT-- section. >>>>-</p> >>>>+sections but they no not use any --EXPECT-- section.</p> >>>> >>>Was there something wrong with this or did people just not like the >>>rewrite? If so what would you like changed? >> >> >> >> >> >> Best regards, >> Marcus >> >> >> >> >> > Sure. here it is - hopefully it works better then last time. Yep, comitted. Best regards, Marcus