Re: Req #16393 [Com]: Better support for file-level code

Hi Dave, I'm answering by email because the pear bug-tracker is a PITA when it comes to quote text... dw-pear@digger.net a écrit :

There are two things the second patch allows... First is the ultimate simple case of testing a script that does one thing, and so doesn't need any setup or test code. For example, a script that responds to an AJAX request with a list of countries from a database: /** * <code> * // flags: NORMALIZE_WHITESPACE, ELLIPSIS * // expects: * // <countries>

 * //    [...]
 * //    <country>
 * //       <code>US</code>
 * //       <name>United States</name>
 * //    </country>
 * //    [...]

* // </countries> * </code> */ $data = getCountriesFromDatabase(); echo formatAsXML($data);

This is not good testing practices, the test should live in the *getCountriesFromDatabase* and *formatAsXML* functions docblocks, not here.

In that case, you are correct, there can only be one test in the file. But not because other tests would be broken. The file only does one thing, and either succeeds or fails, so there isn't anything for more than one test to do.

The question is more: "Can this file contain another testable code bloc (a class, a function, a variable assignment in a if test, etc..) ?" The answer is "yes", and in this case, Testing_DocTest will fail, because the script will always flush "formatAsXML($data)" to stdout. In short: each test run in its own process *but* for each test, the whole file is included, and, as a consequence, its procedural code gets executed. Consider this: <?php /** * <code> * echo foo(); * // expects: * // foo ! * </code> */ function foo() {

    return 'foo !';

} /** * <code> * sleep(0.1); * // expects: * // foo ! * </code> */ echo "foo !"; ?> $ phpdt test.php Processing /home/izi/pear/pear_svn/Testing_DocTest/test.php [FAIL] function foo =============================== Expected =============================== foo ! ================================ Actual ================================ foo !foo ! [PASS] file level test See ? Naively I would think: "wtf? why my first test fails?". But here, I *know* what i'm doing, since I explicitely wrote the sleep() line as a workaround, so i would just think: "Ok. Testing_DocTest won't work when there's multiple tests in a file that outputs to stdout... not a big deal...". [snip...]

As you can see, in this case you CAN have multiple tests in the file, that test different parts of the code based on setup values. But there is no need for any test code, because the work has been done by the time the test code would have run. Examples like this are the main reason I needed the setup: keyword and wanted the test code optional. Currently in 0.4.1 you can write all kinds of tests for the functions called by this example, but this code itself is untestable.

It's true that here you can have other independent tests because the script output data to stdout only given a condition, but again, it's a very specific case... what if you add this to the switch: """ default:

    echo "<?xml?><error>No data</error>";

""" ?

The critical fix is adding setup:, which makes this now testable. It's just a bit ugly to have to put in test code that does nothing, only to meet the requirement that test code be present.

This requirement is necessary to keep Testing_DocTest consistant in every situation, to encourage users to test "code blocks" and not the whole output of a script. If you put fake code to meet the requirement, then you know what you are doing, and you won't be surprised if other tests crash.

It'd be cleaner with the test code optional. But if you still think it's not worthwhile to do that, at least it can be worked around. :)

Yep ;) this won't go in, sorry. The best ! -- David