Re: write-test.php alternative

Hello Ligaya, sorry missed it, erm can you sent it as a text file again, please? marcus Monday, August 7, 2006, 12:38:27 AM, you wrote: > Ligaya Turmelle wrote: >> OK - really hope I did this right. Attached is an alternate for the >> current write-test.php qa doc page. The new page has some additional >> info and explanations as well as some reorganization of what is already >> there. Hopefully it will be a bit easier for beginners. I did the >> update and it merged. Let me know if there are any problems. >> >> >> ------------------------------------------------------------------------ >> >> Index: write-test.php >> =================================================================== >> RCS file: /repository/qaweb/write-test.php,v >> retrieving revision 1.7 >> diff -u -r1.7 write-test.php >> --- write-test.php 11 Jul 2006 22:50:53 -0000 1.7 >> +++ write-test.php 18 Jul 2006 05:53:43 -0000 >> @@ -18,10 +18,76 @@ >> <tr> >> <td width="10">&nbsp;</td> >> <td width="100%"> >> -<p>Writing a test file is very easy if you are used to PHP. >> -Here is an actual test file from standard module:</p> >> +<h2>phpt Test Basics</h2> >> +<ul> >> + <li> >> + <b>So what are phpt tests?</b> >> + <p>A phpt test is a little script used by the php internal and quality >> +assurance teams to test PHP's functionality. It can be used with new releases to >> make >> +sure they can do all the things that previous releases can, or to help find bugs in >> current >> +releases. By writing phpt tests you are helping to make PHP stabler.</p> >> + </li> >> + >> + <li> >> + <b>What skills are needed to write a phpt test?</b> >> + <p>All that is really needed to write a phpt test >> +is a basic understanding of the PHP language, a text editor, and a way to get the results >> +of your code. That is it. So if you have been writing and running PHP scripts already - >> +you have everything you need.</p> >> + </li> >> + >> + <li> >> + <b>What do you write phpt tests on?</b> >> + <p>Basically you can write a phpt test on one of the various >> +php functions available. You can write a test on a basic language function (a string >> +function or an array function) , or a function provided by one of PHP's numerous >> extensions >> +(a mysql function or a image function or a mcrypt function). You can find out what >> functions >> +already have phpt tests on them by looking in the <a >> href="http://cvs.php.net/viewvc.cgi/">html >> +version</a> of the cvs (php-src/ext/standard/tests/ is a good place to start looking >> - though >> +<i>all</i> the tests currently written are NOT in there). After that - you >> +can pick any function you want. If you want more guidance than that - you can always ask >> +the PHP Quality Assurance Team on their mailing list >> +(<a href="‹†Í¨ šÔ/¨"má >> qŒmailto:php-qa@lists.php.net">php-qa@lists.php.net</a>) where they >> +would like you to direct your attentions.</p> >> + </li> >> + >> + <li> >> + <b>How is a phpt test is used?</b> >> + <p>When a test is called by the run-test.php script it takes various >> +parts of the phpt file to name and create a .php file. That .php file is then executed. >> The >> +output of the .php file is then compared to a different section of the phpt file. If the >> output of >> +the script "matches" the output provided in the phpt script - it >> passes.</p> >> + </li> >> + >> + <li> >> + <b>What should a phpt test do?</b> >> + <p>Basically - it should try and break the PHP function. It should >> check not >> +only the functions normal parameters, but it should also check edge cases. Intentionally >> generating >> +an error is allowed.</p> >> + </li> >> +</ul> >> + >> +<h2>Writing phpt Tests</h2> >> +<h3>Naming Conventions</h3> >> +<p>Phpt tests follow a very strict naming convention. This is done to easily >> identify what each phpt >> +test is for. Tests should be named according to the following list: >> + <ul type="none"> >> + <li><i>Tests for bugs</i><br /> >> + &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <!-- Give me >> some space! --> >> + bug&lt;bugid&gt;.phpt (bug17123.phpt)</li> >> + <li><i>Tests for functions</i><br /> >> + &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <!-- >> Give me some space! --> >> + &lt;functionname&gt;.phpt (dba_open.phpt)</li> >> + <li><i>General tests for extensions</i><br /> >> + &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <!-- >> Give me some space! --> >> + &lt;extname&gt;&lt;no&gt;.phpt >> (dba_003.phpt)</li> >> + </ul> >> +</p> >> >> -<h3>ext/standard/tests/strings/strtr.phpt</h3> >> +<h3>Basic Format</h3> >> +<p>A test must contain the sections TEST, FILE and either EXPECT or EXPECTF at a >> minimum. The example >> +below illustrates a minimal test.</p> >> +<i>ext/standard/tests/strings/strtr.phpt</i> >> <pre> >> --TEST-- >> strtr() function >> @@ -35,33 +101,37 @@ >> string(32) "# hello All, I sAid hi planet! #" >> </pre> >> >> -<p>As you can see the file is devided into several sections. Below is a >> -list of all possible sections:</p> >> +<p>As you can see the file is divided into several sections. The TEST section holds >> a one line comment >> +about the phpt test. The phpt files name is used when generating a .php file. The FILE >> section is used >> +as the body of the .php file, so don't forget to open and close your php tags. The >> EXPECT section is >> +the part used as a comparison to see if the test passes. It is a >> +good idea to generate output with var_dump() calls.</p> >> >> +<h3>All Sections</h3> >> +A phpt test can have many more parts then just the minimum. In fact some of the manditory >> parts have >> +alternatives that may be used if the situation warrents it. >> <dl> >> <dt>--TEST--</dt> >> <dd>title of the test. (required)</dd> >> >> <dt>--SKIPIF--</dt> >> -<dd>is condition when to skip this test. (optional)</dd> >> +<dd>a condition when to skip this test. (optional)</dd> >> >> <dt>--POST--</dt> >> -<dd>is POST variable passed to test script. (optional)</dd> >> +<dd>POST variables to be passed to the test script. (optional)</dd> >> >> <dt>--GET--</dt> >> -<dd>is GET variable passed to test script. (optional)</dd> >> +<dd>GET variables to be passed to the test script. (optional)</dd> >> >> <dt>--INI--</dt> >> -<dd>each line contains an ini setting e.g. foo=bar. (optional)</dd> >> +<dd>to be used if you need a specific php.ini setting for the test. >> + Each line contains an ini setting e.g. foo=bar. (optional)</dd> >> >> <dt>--ARGS--</dt> >> <dd>a single line defining the arguments passed to php. (optional)</dd> >> >> <dt>--ENV--</dt> >> -<dd>configures the environment to be used ofr php. (optional)</dt> >> - >> -<dt>--FILE--</dt> >> -<dd>the test script. (required)</dd> >> +<dd>configures the environment to be used for php. (optional)</dt> >> >> <dt>--FILEEOF--</dt> >> <dd>an alternative to --FILE-- where any trainling line break is omitted. >> @@ -101,13 +171,9 @@ >> <dd>This is only available in the --FILE-- section. Any part after this line >> is not going into the actual test script (see below for more).</dd> >> >> -<p>A test must at least contain the sections TEST, FILE and either EXPECT >> -or EXPECTF. When a test is called run-test.php takes the name from the >> -TEST section and writes the FILE section into a ".php" file with the >> -same name as the ".phpt" file. This ".php" file will then be executed >> -and its output compared to the contents of the EXPECT section. It is a >> -good idea to generate output with var_dump() calls.</p> >> +<h2>Examples</h2> >> >> +<h3>EXPECTF</h3> >> <p>/ext/standard/tests/strings/str_shuffle.phpt is a good example for using >> EXPECTF instead of EXPECT. From time to time the algorithm used for shuffle >> changed and sometimes the machine used to execute the code has influence >> @@ -115,8 +181,7 @@ >> detectable by %s. Other scan-able forms are %i for integers, %d for numbers >> only, %f for floating point values, %c for single characters and %x for >> hexadecimal values.</p> >> - >> -<h3>/ext/standard/tests/strings/str_shuffle.phpt</h3> >> +<i>/ext/standard/tests/strings/str_shuffle.phpt</i> >> <pre> >> --TEST-- >> Testing str_shuffle. >> @@ -132,12 +197,13 @@ >> string(3) "123" >> </pre> >> >> +<h3>EXPECTREGEX</h3> >> <p>/ext/standard/tests/strings/strings001.phpt is a good example for using >> EXPECTREGEX instead of EXPECT. This test also shows that in EXPECTREGEX >> some characters need to be escaped since otherwise they would be >> interpreted as a regular expression.</p> >> >> -<h3>/ext/standard/tests/strings/strings001.phpt</h3> >> +<i>/ext/standard/tests/strings/strings001.phpt</i> >> <pre> >> --TEST-- >> Test whether strstr() and strrchr() are binary safe. >> @@ -153,6 +219,7 @@ >> string\(19\) \" nica\x00turska panica\" >> </pre> >> >> +<h3>SKIPIF</h3> >> <p>Some tests depend on modules or functions available only in certain versions >> or they even require minimum version of php or zend. These tests should be >> skipped when the requirement cannot be fullfilled. To achieve this you can >> @@ -160,7 +227,7 @@ >> the SKIPIF section must print out the word "skip" followed by a reason why >> the test should skip.</p> >> >> -<h3>/ext/exif/tests/exif005.phpt</h3> >> +<i>/ext/exif/tests/exif005.phpt</i> >> <pre> >> --TEST-- >> Check for exif_read_data, unusual IFD start >> @@ -188,26 +255,16 @@ >> <p>Test script and SKIPIF code should be directly written into *.phpt. However, >> it is recommended to use include files when more test scripts depend on the >> same SKIPIF code or when certain test files need the same values for some >> -input. But no file used by any test should have one of the following >> -extensions: ".php", ".log", ".exp", ".out" or >> ".diff".</p> >> - >> -<p>Tests should be named according to the following list: >> -<dl> >> -<dt>Tests for bugs</dt> >> -<dd>bug&lt;bugid&gt;.phpt (bug17123.phpt)</dd> >> - >> -<dt>Tests for functions</dt> >> -<dd>&lt;functionname&gt;.phpt (dba_open.phpt)</dd> >> +input. </p> >> >> -<dt>General tests for extensions</dt> >> -<dd>&lt;extname&gt;&lt;no&gt;.phpt (dba_003.phpt)</dd> >> -</dl> >> -</p> >> >> -<p>When you use an include file for the SKIPIF section it should be named >> -"skipif.inc" and an include file used in the FILE section of many tests >> -should be named "test.inc".</p> >> +<p> >> +<b>Note:</b> no file used by any test should have one of the following >> extensions: >> +".php", ".log", ".exp", ".out" or >> ".diff". When you use an include file for the >> +SKIPIF section it should be named "skipif.inc" and an include file used in the >> +FILE section of many tests should be named "test.inc".</p> >> >> +<h2> Final Notes</h2> >> <h3>Redirecting tests</h3> >> <p>Using --REDIRECTTEST-- it is possible to redirect from one test to a bunch >> of other tests. That way multiple extensions can refer to the same set of >> @@ -222,11 +279,9 @@ >> </p> >> >> <p>Redirect tests may especially contain --SKIPIF--, --ENV-- and --ARGS-- >> -sections but they no not use any --EXPECT-- section. >> -</p> >> +sections but they no not use any --EXPECT-- section.</p> >> > Was there something wrong with this or did people just not like the > rewrite? If so what would you like changed? Best regards, Marcus