Bug #60285 [Com]: setting REPORT_EXIT_STATUS for make test won't work
Edit report at https://bugs.php.net/bug.php?id=60285&edit=1
ID: 60285
Comment by: reeze dot xia at gmail dot com
Reported by: tyrael@php.net
Summary: setting REPORT_EXIT_STATUS for make test won't work
Status: Open
Type: Bug
Package: Testing related
PHP Version: trunk-SVN-2011-11-13 (SVN)
Block user comment: N
Private report: N
New Comment:
Yes, I got the same problem, I use Travis to test my extension, but make test
didn't report failed test correctly.
http://travis-ci.org/#!/reeze/php-sundown/jobs/1670528
That commit ignore the status code, the lastest version add another command
after run-test:
rm $(top_builddir)/tmp-php.ini; \
This makes it never return exit code.
--------------------------
Previous Comments:
------------------------------------------------------------------------
[2011-11-13 18:45:24] tyrael@php.net
I just checked, it was introduced way back in 2003:
http://svn.php.net/viewvc?view=revision&revision=135844
------------------------------------------------------------------------
[2011-11-13 18:35:57] tyrael@php.net
Description:
------------
when setting up the ci environment, I noticed, that passing the REPORT_EXIT_STATUS
environment variable to run-tests.php works, but passing the same thing to make
test won't.
after some debugging with Hannes, we figured out, that the test target in the
Makefile overuses the error suppression
operator(http://sunsite.ualberta.ca/Documentation/Gnu/make-
3.79/html_chapter/make_5.html#SEC48).
if you check the test target (http://svn.php.net/viewvc/php/php-
src/trunk/Makefile.global?view=markup#l88) you can see that the whole target is a
one-liner prefixed with -
which means that anything can fail in the target, it will still report success. :/
the same pattern is used for many of the targets.
Could somebody look into this?
Test script:
---------------
REPORT_EXIT_STATUS=1 make test;
echo $?;
Expected result:
----------------
should print non-zero if there are failed tests.
Actual result:
--------------
always prints 0
------------------------------------------------------------------------
--
Edit this bug report at https://bugs.php.net/bug.php?id=60285&edit=1
Thread (4 messages)