Bug #76280 [Com]: Performance degrades permantly when overwriting PHP files, until apache restart

From: Date: Sat, 28 Apr 2018 15:00:05 +0000
Subject: Bug #76280 [Com]: Performance degrades permantly when overwriting PHP files, until apache restart
References: 1  Groups: php.bugs 
Request: Send a blank email to php-bugs+get-214969@lists.php.net to get a copy of this message
Edit report at https://bugs.php.net/bug.php?id=76280&edit=1

 ID:                 76280
 Comment by:         spam2 at rhsoft dot net
 Reported by:        teo8976 at gmail dot com
 Summary:            Performance degrades permantly when overwriting PHP
                     files, until apache restart
 Status:             Feedback
 Type:               Bug
 Package:            opcache
 Operating System:   debian
 PHP Version:        5.6.36
 Block user comment: N
 Private report:     N

 New Comment:

>> Please check whether the deployment 
>> of new files triggered an opcache restart
>>
> How can I check that?

look at the opcache vars and in doubt feed "opcache status panel" into Google and ask
yourself why you don't have such tool installed - not knowing what a server does would i
personally call a security issue (and no such a panel out-of-the-box is no good idea because *that*
would be a security issue)

> Something that can cause a website to go down 
> permanently because of a performance degradation 
> with no good reason, I would call a security issue

nosense - following your logic running a server without opcache would be insecure at all


Previous Comments:
------------------------------------------------------------------------
[2018-04-28 14:54:14] teo8976 at gmail dot com

> Please check whether the deployment 
> of new files triggered an opcache restart

How can I check that?

> Generally any slowdown should only be temporary though

It isn't. When this happens, the slowdown is permanent until I restart apache.
In the screenshot I attached (where I unintentionally cut off the part that showed the times), it
went on for several hours. It was also surprisingly "stable", everything being
consistently X% slower (of course it is possible that the slowdown was affecting a specific piece of
code being called by all scripts)

> In any case, PHP 5 is no longer supported

http://php.net/supported-versions.php says
"security support until 31 december" this year. Something that can cause a website to go
down permanently because of a performance degradation with no good reason, I would call a security
issue.

------------------------------------------------------------------------
[2018-04-27 19:44:35] nikic@php.net

Please check whether the deployment of new files triggered an opcache restart (because the opcache
SHM memory limit was reached). Generally any slowdown should only be temporary though.

In any case, PHP 5 is no longer supported and PHP 7 has quite a lot of changes to improve
performance, including in opcache. Additionally, the second-level file cache can be used to reduce
the performance impact of opcache restarts.

------------------------------------------------------------------------
[2018-04-27 19:18:44] teo8976 at gmail dot com

Here's some example performance data (oh come on, I can't attach a screenshot??) https://ibb.co/h3J6bx

This is avarage execution time (measured on the server side) of all requests, but graphs for any
given single url would show exactly the same pattern.

------------------------------------------------------------------------
[2018-04-27 19:09:56] teo8976 at gmail dot com

Description:
------------
I have a website (well, it's not mine actually) that gets about 1k requests per minute.

Sometimes I upload updated PHP files. Obviously this means I'm overwriting files at the very
same time they are being executed. I expect that to cause errors (experienced by the users) during
the short period of time while files are being overwritten (usually a few seconds).

However, what I wouldn't expect and happens VERY often is that: after that, general performance
degrades, meaning that the execution of EVERY SINGLE script in every single request is significantly
slower. Sometimes it's just a few dozens % increase in execution time, sometimes it explodes to
the point that the whole websites goes down.

Restarting apache returns performance to normal every time, whether there had been a total collapse
or just a degradation in performance.

Expected result:
----------------
When you overwrite PHP files in a website that is being visited by thousands of people, you'd
expect a small percentage of them to experience random errors, if a file is being overwritten just
when it's being executing.

However, as soon as all the requests that were being handled are done, everything should go back to
working normally, and there shoul be no need to restart Apache

Actual result:
--------------
See description.

Seriously, this is fucking pathetic.


------------------------------------------------------------------------



--
Edit this bug report at https://bugs.php.net/bug.php?id=76280&edit=1


Thread (16 messages)

« previous php.bugs (#214969) next »