Re: write-test.php alternative
| From: | Ligaya Turmelle | Date: | Sun, 06 Aug 2006 22:38:27 +0000 |
| Subject: | Re: write-test.php alternative | ||
| References: | 1 | Groups: | php.qa |
| Request: | Send a blank email to php-qa+get-26856@lists.php.net to get a copy of this message | ||
Ligaya Turmelle wrote:
OK - really hope I did this right. Attached is an alternate for the current write-test.php qa doc page. The new page has some additional info and explanations as well as some reorganization of what is already there. Hopefully it will be a bit easier for beginners. I did the update and it merged. Let me know if there are any problems. ------------------------------------------------------------------------ Index: write-test.php =================================================================== RCS file: /repository/qaweb/write-test.php,v retrieving revision 1.7 diff -u -r1.7 write-test.php --- write-test.php 11 Jul 2006 22:50:53 -0000 1.7 +++ write-test.php 18 Jul 2006 05:53:43 -0000 @@ -18,10 +18,76 @@-- life is a game... so have fun.<tr> <td width="10"> </td> <td width="100%"> -<p>Writing a test file is very easy if you are used to PHP. -Here is an actual test file from standard module:</p>+<h2>phpt Test Basics</h2> +<ul> + <li> + <b>So what are phpt tests?</b> + <p>A phpt test is a little script used by the php internal and quality +assurance teams to test PHP's functionality. It can be used with new releases to make +sure they can do all the things that previous releases can, or to help find bugs in current +releases. By writing phpt tests you are helping to make PHP stabler.</p> + </li> + + <li> + <b>What skills are needed to write a phpt test?</b> + <p>All that is really needed to write a phpt test +is a basic understanding of the PHP language, a text editor, and a way to get the results +of your code. That is it. So if you have been writing and running PHP scripts already - +you have everything you need.</p> + </li> + + <li> + <b>What do you write phpt tests on?</b> + <p>Basically you can write a phpt test on one of the various +php functions available. You can write a test on a basic language function (a string +function or an array function) , or a function provided by one of PHP's numerous extensions +(a mysql function or a image function or a mcrypt function). You can find out what functions +already have phpt tests on them by looking in the <a href="http://cvs.php.net/viewvc.cgi/">html +version</a> of the cvs (php-src/ext/standard/tests/ is a good place to start looking - though +<i>all</i> the tests currently written are NOT in there). After that - you +can pick any function you want. If you want more guidance than that - you can always ask +the PHP Quality Assurance Team on their mailing list +(<a href="mailto:php-qa@lists.php.net">php-qa@lists.php.net</a>) where they +would like you to direct your attentions.</p> + </li> + + <li> + <b>How is a phpt test is used?</b> + <p>When a test is called by the run-test.php script it takes various +parts of the phpt file to name and create a .php file. That .php file is then executed. The +output of the .php file is then compared to a different section of the phpt file. If the output of +the script "matches" the output provided in the phpt script - it passes.</p> + </li> + + <li> + <b>What should a phpt test do?</b> + <p>Basically - it should try and break the PHP function. It should check not +only the functions normal parameters, but it should also check edge cases. Intentionally generating +an error is allowed.</p> + </li> +</ul> + +<h2>Writing phpt Tests</h2> +<h3>Naming Conventions</h3> +<p>Phpt tests follow a very strict naming convention. This is done to easily identify what each phpt +test is for. Tests should be named according to the following list: + <ul type="none"> + <li><i>Tests for bugs</i><br /> + <!-- Give me some space! -->+ bug<bugid>.phpt (bug17123.phpt)</li>+ <li><i>Tests for functions</i><br /> + <!-- Give me some space! -->+ <functionname>.phpt (dba_open.phpt)</li>+ <li><i>General tests for extensions</i><br /> + <!-- Give me some space! --> + <extname><no>.phpt (dba_003.phpt)</li> + </ul> +</p> -<h3>ext/standard/tests/strings/strtr.phpt</h3> +<h3>Basic Format</h3> +<p>A test must contain the sections TEST, FILE and either EXPECT or EXPECTF at a minimum. The example +below illustrates a minimal test.</p> +<i>ext/standard/tests/strings/strtr.phpt</i> <pre> --TEST-- strtr() function @@ -35,33 +101,37 @@ string(32) "# hello All, I sAid hi planet! #" </pre> -<p>As you can see the file is devided into several sections. Below is a -list of all possible sections:</p> +<p>As you can see the file is divided into several sections. The TEST section holds a one line comment +about the phpt test. The phpt files name is used when generating a .php file. The FILE section is used +as the body of the .php file, so don't forget to open and close your php tags. The EXPECT section is +the part used as a comparison to see if the test passes. It is a +good idea to generate output with var_dump() calls.</p> +<h3>All Sections</h3> +A phpt test can have many more parts then just the minimum. In fact some of the manditory parts have +alternatives that may be used if the situation warrents it. <dl> <dt>--TEST--</dt> <dd>title of the test. (required)</dd> <dt>--SKIPIF--</dt> -<dd>is condition when to skip this test. (optional)</dd> +<dd>a condition when to skip this test. (optional)</dd> <dt>--POST--</dt> -<dd>is POST variable passed to test script. (optional)</dd> +<dd>POST variables to be passed to the test script. (optional)</dd> <dt>--GET--</dt> -<dd>is GET variable passed to test script. (optional)</dd> +<dd>GET variables to be passed to the test script. (optional)</dd> <dt>--INI--</dt> -<dd>each line contains an ini setting e.g. foo=bar. (optional)</dd> +<dd>to be used if you need a specific php.ini setting for the test. + Each line contains an ini setting e.g. foo=bar. (optional)</dd> <dt>--ARGS--</dt> <dd>a single line defining the arguments passed to php. (optional)</dd> <dt>--ENV--</dt> -<dd>configures the environment to be used ofr php. (optional)</dt> - -<dt>--FILE--</dt> -<dd>the test script. (required)</dd> +<dd>configures the environment to be used for php. (optional)</dt> <dt>--FILEEOF--</dt> <dd>an alternative to --FILE-- where any trainling line break is omitted. @@ -101,13 +171,9 @@ <dd>This is only available in the --FILE-- section. Any part after this line is not going into the actual test script (see below for more).</dd> -<p>A test must at least contain the sections TEST, FILE and either EXPECT -or EXPECTF. When a test is called run-test.php takes the name from the -TEST section and writes the FILE section into a ".php" file with the -same name as the ".phpt" file. This ".php" file will then be executed -and its output compared to the contents of the EXPECT section. It is a -good idea to generate output with var_dump() calls.</p> +<h2>Examples</h2> +<h3>EXPECTF</h3> <p>/ext/standard/tests/strings/str_shuffle.phpt is a good example for using EXPECTF instead of EXPECT. From time to time the algorithm used for shuffle changed and sometimes the machine used to execute the code has influence @@ -115,8 +181,7 @@ detectable by %s. Other scan-able forms are %i for integers, %d for numbers only, %f for floating point values, %c for single characters and %x for hexadecimal values.</p> - -<h3>/ext/standard/tests/strings/str_shuffle.phpt</h3> +<i>/ext/standard/tests/strings/str_shuffle.phpt</i> <pre> --TEST-- Testing str_shuffle. @@ -132,12 +197,13 @@ string(3) "123" </pre> +<h3>EXPECTREGEX</h3> <p>/ext/standard/tests/strings/strings001.phpt is a good example for using EXPECTREGEX instead of EXPECT. This test also shows that in EXPECTREGEX some characters need to be escaped since otherwise they would be interpreted as a regular expression.</p> -<h3>/ext/standard/tests/strings/strings001.phpt</h3> +<i>/ext/standard/tests/strings/strings001.phpt</i> <pre> --TEST-- Test whether strstr() and strrchr() are binary safe. @@ -153,6 +219,7 @@ string\(19\) \" nica\x00turska panica\" </pre> +<h3>SKIPIF</h3> <p>Some tests depend on modules or functions available only in certain versions or they even require minimum version of php or zend. These tests should be skipped when the requirement cannot be fullfilled. To achieve this you can @@ -160,7 +227,7 @@ the SKIPIF section must print out the word "skip" followed by a reason why the test should skip.</p> -<h3>/ext/exif/tests/exif005.phpt</h3> +<i>/ext/exif/tests/exif005.phpt</i> <pre> --TEST-- Check for exif_read_data, unusual IFD start @@ -188,26 +255,16 @@ <p>Test script and SKIPIF code should be directly written into *.phpt. However, it is recommended to use include files when more test scripts depend on the same SKIPIF code or when certain test files need the same values for some -input. But no file used by any test should have one of the following -extensions: ".php", ".log", ".exp", ".out" or ".diff".</p> - -<p>Tests should be named according to the following list: -<dl> -<dt>Tests for bugs</dt> -<dd>bug<bugid>.phpt (bug17123.phpt)</dd> - -<dt>Tests for functions</dt> -<dd><functionname>.phpt (dba_open.phpt)</dd> +input. </p> -<dt>General tests for extensions</dt> -<dd><extname><no>.phpt (dba_003.phpt)</dd> -</dl> -</p> -<p>When you use an include file for the SKIPIF section it should be named -"skipif.inc" and an include file used in the FILE section of many tests -should be named "test.inc".</p> +<p> +<b>Note:</b> no file used by any test should have one of the following extensions: +".php", ".log", ".exp", ".out" or ".diff". When you use an include file for the +SKIPIF section it should be named "skipif.inc" and an include file used in the +FILE section of many tests should be named "test.inc".</p> +<h2> Final Notes</h2> <h3>Redirecting tests</h3> <p>Using --REDIRECTTEST-- it is possible to redirect from one test to a bunch of other tests. That way multiple extensions can refer to the same set of @@ -222,11 +279,9 @@ </p> <p>Redirect tests may especially contain --SKIPIF--, --ENV-- and --ARGS-- -sections but they no not use any --EXPECT-- section. -</p> +sections but they no not use any --EXPECT-- section.</p> Was there something wrong with this or did people just not like the rewrite? If so what would you like changed?