Req #29571 [Opn->Csd]: [PATCH] Many uploads will slow down Linux machine

Edit report at https://bugs.php.net/bug.php?id=29571&edit=1 ID: 29571 Updated by: ilutov@php.net Reported by: k at ailis dot de Summary: [PATCH] Many uploads will slow down Linux machine -Status: Open +Status: Closed Type: Feature/Change Request Package: *General Issues Operating System: Linux PHP Version: * -Assigned To: +Assigned To: ilutov Block user comment: N Private report: N New Comment: Thanks for the patch! I'm not sure if this is still relevant with more modern file systems. If it is, please create a new issue or preferably a patch on GitHub. Previous Comments: ------------------------------------------------------------------------ [2015-01-08 23:08:59] ajf@php.net Seems reasonable. ------------------------------------------------------------------------ [2004-08-08 14:42:00] k at ailis dot de Description: ------------ This is a mix of "performance problem" and "feature request". So feel free to reclassify this report as "Feature request" if you think it fits better. In a large project we have the problem that many users are uploading many large files to our PHP server. PHP can handle this very well but the problem is, that all files are stored in the same directory. Our upload directory sometimes contains more than 20000 files. This produces high loads because accessing these files is very I/O intensive because of the flat directory hierarchy. You already have addressed this problem with session files by introducing a "directory depth". Its possible to use "2;/var/lib/php/uploads" as session.save_path which is a very nice feature. Exactly this feature would be a cool thing if it were also implemented for "upload_tmp_dir". Here is a small patch as an example how this feature could be implemented. We are using this patch on our servers and it works great. This patch not only implements this feature for upload_tmp_dir but for all other modules which are using the php_open_temporary_file() function. --- php_open_temporary_file.c.orig Sun Aug 8 14:29:06 2004 +++ php_open_temporary_file.c Sun Aug 8 14:29:12 2004 @@ -106,6 +106,10 @@ { char *trailing_slash; char *opened_path; + char *p; + char *sub_dirs; + size_t dirdepth, i; + int rand_value; int fd = -1; #ifndef HAVE_MKSTEMP int open_flags = O_CREAT | O_TRUNC | O_RDWR @@ -125,6 +129,30 @@ if (!(opened_path = emalloc(MAXPATHLEN))) { return -1; } + + /* Check subdirectory depth and point p to real path name */ + if ((p = strchr(path, ';'))) + { + dirdepth = (size_t) strtol(path, NULL, 10); + p++; + } + else + { + dirdepth = 0; + p = (char *) path; + } + + /* Build subdirectory tree if needed */ + if (dirdepth) { + if (!(sub_dirs = emalloc(dirdepth * 2 + 1))) return -1; + sub_dirs[0] = 0; + for (i = 0; i < dirdepth; i++) + { + strcat(sub_dirs, "x/"); + rand_value = php_rand() & 0xf; + sub_dirs[i * 2] = rand_value < 10 ? rand_value + 48 : rand_value + 87; + } + } else sub_dirs = ""; if (IS_SLASH(path[strlen(path)-1])) { trailing_slash = ""; @@ -132,7 +160,10 @@ trailing_slash = "/"; } - (void)snprintf(opened_path, MAXPATHLEN, "%s%s%sXXXXXX", path, trailing_slash, pfx); + (void)snprintf(opened_path, MAXPATHLEN, "%s%s%s%sXXXXXX", p, trailing_slash, sub_dirs, pfx); + + /* Free memory if it was used */ + if (dirdepth) efree(sub_dirs); #ifdef PHP_WIN32 if (GetTempFileName(path, pfx, 0, opened_path)) { @@ -140,7 +171,7 @@ * which means that opening it will fail... */ VCWD_CHMOD(opened_path, 0600); fd = VCWD_OPEN_MODE(opened_path, open_flags, 0600); - } +l1 } #elif defined(NETWARE) /* Using standard mktemp() implementation for NetWare */ file_path = mktemp(opened_path); ------------------------------------------------------------------------ -- Edit this bug report at https://bugs.php.net/bug.php?id=29571&edit=1