[PATCH] suppressing realpath() with APC + tiny patches

Hello, These days I've been working on some tasks to reduce I/O system calls (lstat64, stat64, open and so on) of PHP for some reasons and as the result of it, I noticed that with the APC + very small two pathes we can suppress most of realpath() calls (although currently I'm not sure this is the correct method or not...): http://eth.jp/php-cvs-20040506-delayed-fopen.patch.txt (patch against current 4.3 tree, because APC is not yet fully compatible with PHP 5) http://eth.jp/apc-2.0.3-delayed-fopen.patch.txt (patch against APC-2.0.3, current CVS-HEAD is somehow unstable...) So please try and review if you're curious about suppressing realpath() calls:) Usage: 1) apply patches to current 4.3 tree, and APC-2.0.3 2) buidconf and configure PHP as always with --enable-delayed-fopen option 3) build and install PHP and APC as always 4) add "delayed_fopen=1" entry to your php.ini And details are as follows: As the recent thread and some other threads show (which listed below), PHP requires quite a few I/O system calls especially when large number of scripts are include_once()ed. And these are not ignorable for the websites with more than 1 million hits per hour although it has nothing to do with small sites. [Performance Consideration of 1 class per file] http://marc.theaimsgroup.com/?l=php-dev&m=108279256414391&w=2 [Performance degradation] http://www.phpbuilder.com/lists/php-developer-list/2003022/0226.php The main concern here is the lstat64() by realpath(), and open() by searching for include_paths (when many include_paths are set). I think suppressing open() (which will result in ENOENT) is not so easy (because we need some other new caching system for include_paths), but perhaps we can easily reduce realpath() for included scritpts with APC, because APC does not use an opned file handle when cache hits. So, my patch skips zend_open() until open_file_scanning() is called inside zend_compile_file() when include_once() or require_once() is called, so that zend_open() is not called when the target script is cached in APC. As a result of this, we can reduce most of realpath() calls by include_once() and required_once(). These patches are not yet verified well so comments are welcome! Best regards, Masaki Fujimoto fujimoto@php.net

diff -ru APC-2.0.3.bak/apc_main.c APC-2.0.3/apc_main.c --- APC-2.0.3.bak/apc_main.c Fri Mar 26 03:53:43 2004 +++ APC-2.0.3/apc_main.c Thu May 6 20:10:12 2004 @@ -166,6 +166,11 @@ /* search for the file in the cache */ cache_entry = apc_cache_find(APCG(cache), key); if (cache_entry != NULL) { +#if DELAYED_FOPEN + if (h->opened_path == NULL) { + h->opened_path = estrdup(cache_entry->filename); + } +#endif /* DELAYED_FOPEN */ zend_llist_add_element(&CG(open_files), h); /* XXX kludge */ apc_stack_push(APCG(cache_stack), cache_entry); return cached_compile(TSRMLS_C);Index: Zend/Zend.m4 =================================================================== RCS file: /repository/Zend/Attic/Zend.m4,v retrieving revision 1.35.2.6 diff -u -r1.35.2.6 Zend.m4 --- Zend/Zend.m4 7 Oct 2003 22:49:27 -0000 1.35.2.6 +++ Zend/Zend.m4 6 May 2004 13:02:06 -0000 @@ -143,6 +143,14 @@ ZEND_MEMORY_LIMIT=no ]) +AC_ARG_ENABLE(delayed-fopen, +[ --enable-delayed-fopen Delay fopen for include_once/require_once + until compilcation.], [ + ZEND_DELAYED_FOPEN=$enableval +],[ + ZEND_DELAYED_FOPEN=no +]) + AC_ARG_ENABLE(zend-multibyte, [ --enable-zend-multibyte Compile with zend multibyte support. ], [ ZEND_MULTIBYTE=$enableval @@ -159,6 +167,9 @@ AC_MSG_CHECKING(whether to enable a memory limit) AC_MSG_RESULT($ZEND_MEMORY_LIMIT) +AC_MSG_CHECKING(whether to enable delayed fopen) +AC_MSG_RESULT($ZEND_DELAYED_FOPEN) + AC_MSG_CHECKING(whether to enable Zend debugging) AC_MSG_RESULT($ZEND_DEBUG) @@ -191,6 +202,12 @@ AC_DEFINE(MEMORY_LIMIT, 0, [Memory limit]) fi +if test "$ZEND_DELAYED_FOPEN" = "yes"; then + AC_DEFINE(DELAYED_FOPEN, 1, [Delayed fopen]) +else + AC_DEFINE(DELAYED_FOPEN, 0, [Delayed fopen]) +fi + if test "$ZEND_MULTIBYTE" = "yes"; then AC_DEFINE(ZEND_MULTIBYTE, 1, [ ]) fi Index: Zend/zend_execute.c =================================================================== RCS file: /repository/Zend/Attic/zend_execute.c,v retrieving revision 1.316.2.34 diff -u -r1.316.2.34 zend_execute.c --- Zend/zend_execute.c 1 Apr 2004 22:05:38 -0000 1.316.2.34 +++ Zend/zend_execute.c 6 May 2004 13:02:07 -0000 @@ -2137,6 +2137,25 @@ int dummy = 1; zend_file_handle file_handle = {0}; +#if DELAYED_FOPEN + if (EG(delayed_fopen)) { + file_handle.type = ZEND_HANDLE_FILENAME; + file_handle.filename = inc_filename->value.str.val; + file_handle.free_filename = 0; + new_op_array = zend_compile_file(&file_handle, (EX(opline)->op2.u.constant.value.lval==ZEND_INCLUDE_ONCE?ZEND_INCLUDE:ZEND_REQUIRE) TSRMLS_CC); + if (!file_handle.opened_path) { + file_handle.opened_path = estrndup(inc_filename->value.str.val, inc_filename->value.str.len); + } + if (zend_hash_add(&EG(included_files), file_handle.opened_path, strlen(file_handle.opened_path)+1, (void *)&dummy, sizeof(int), NULL)==SUCCESS) { + /* nothing to do in this case */ + } else { + /* free ops */ + destroy_op_array(new_op_array); + efree(new_op_array); + new_op_array = NULL; + } + } else +#endif /* DELAYD_FOPEN */ if (zend_open(inc_filename->value.str.val, &file_handle) == SUCCESS && ZEND_IS_VALID_FILE_HANDLE(&file_handle)) { Index: Zend/zend_globals.h =================================================================== RCS file: /repository/Zend/Attic/zend_globals.h,v retrieving revision 1.93.2.4 diff -u -r1.93.2.4 zend_globals.h --- Zend/zend_globals.h 31 May 2003 01:37:43 -0000 1.93.2.4 +++ Zend/zend_globals.h 6 May 2004 13:02:07 -0000 @@ -211,6 +211,10 @@ char float_separator[1]; void *reserved[ZEND_MAX_RESERVED_RESOURCES]; + +#if DELAYED_FOPEN + zend_bool delayed_fopen; +#endif /* DELAYED_FOPEN */ }; Index: main/main.c =================================================================== RCS file: /repository/php-src/main/main.c,v retrieving revision 1.512.2.53 diff -u -r1.512.2.53 main.c --- main/main.c 9 Feb 2004 04:05:56 -0000 1.512.2.53 +++ main/main.c 6 May 2004 13:02:07 -0000 @@ -160,6 +160,20 @@ /* }}} */ #endif +#if DELAYED_FOPEN +/* {{{ PHP_INI_MH + */ +static PHP_INI_MH(OnChangeDelayedFopen) +{ + if (new_value) { + EG(delayed_fopen) = zend_atoi(new_value, new_value_length); + } else { + EG(delayed_fopen) = 0; + } + return SUCCESS; +} +/* }}} */ +#endif /* DELAYED_FOPEN */ /* {{{ php_disable_functions */ @@ -354,6 +368,9 @@ #if MEMORY_LIMIT PHP_INI_ENTRY("memory_limit", "8M", PHP_INI_ALL, OnChangeMemoryLimit) #endif +#if DELAYED_FOPEN + PHP_INI_ENTRY("delayed_fopen", "0", PHP_INI_ALL, OnChangeDelayedFopen) +#endif /* DELAYED_FOPEN */ PHP_INI_ENTRY("precision", "14", PHP_INI_ALL, OnSetPrecision) PHP_INI_ENTRY("sendmail_from", NULL, PHP_INI_ALL, NULL) PHP_INI_ENTRY("sendmail_path", DEFAULT_SENDMAIL_PATH, PHP_INI_SYSTEM, NULL)