Re: Re: [BULK] Re: [PHP] OK to have many files in one folder?

Daniel Brown wrote:

On 6/18/07, Tijnema <tijnema@gmail.com> wrote:

On 6/18/07, Daniel Brown <parasane@gmail.com> wrote:

On 6/18/07, Tijnema <tijnema@gmail.com> wrote:

On 6/18/07, Daniel Brown <parasane@gmail.com> wrote:

On 6/18/07, Colin Guthrie <gmane@colin.guthr.ie> wrote:

clive wrote:

I have no clue how big the files are, but you might want to

store them

in a database. That can speed up things, but don't ask me

how much ;)

Tijnema

no dude, while database are convenient, files systems are

faster, I mean

thats what they were designed for, serving files. For lots of files I would store them in directories and sub

directories.

Yeah amen! I generally use a two character hex hash in my main folder to

give 255

sub folders each containing files. In my case the hash is calculated from some known info - e.g.

a database

ID and then stored in a meta data table. Col -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php

Tij, Referring to one of my earlier posts in this thread, as a refresher, database information is stored in files. So to store

files

in a database means the following has to take place:

       1.) Data is uploaded to the server.
       2.) Data is processed by the database server.
       3.) Data is compressed and encrypted.
       4.) Data is written to disk.
           (4b. - optional) Data file is checked for integrity and

correctness of write. Then, to serve the content, a similar reverse occurs:

       1.) Request is sent to the server via a script.
       2.) The script interfaces with the database to locate the

desired data row.

       3.) Once located, the data is pulled in raw form from the 

database file.

4.) The data is then decompressed and decrypted. 5.) The data is streamed back to the script. 6.) The script decides how to handle it, based upon

hard-coded options.

For a filesystem, it's simpler. Uploading:

       1.) Data is uploaded.
       2.) Data is written to disk.

Serving:

       1.) Request is sent via script, FTP, socket connection, or

local request.

       2.) File is served through via the method requested.

So there are several pros and cons for each, but just to list three of each with regard to database storage: Pros:

       Can be stored centralized and portable, without needing 

to archive.

Automatically encrypted and compressed upon storage. Easier and faster to search. Cons:

       Slower storage and retrieval
       Much greater risk of corruption
       Not always cross-compatible across platforms, versions,

installations, etc. -- Daniel P. Brown

How sure are you about all this? Benchmarks? What about search time for the harddrives etc? It needs to scan the inode table to find the file, etc.., then it needs to locate that file on the harddrive, while the second file might be on the other side of the disk.... Storing all files in one single filesystem means you have only 1 inode :) All data is stored at same place (atleast, it should be) Tijnema

Do I have benchmarks? No. Do I have the time to do them now? No. Do I feel confident enough that if you were to run those benchmark tests that it would prove me right? Damn straight. ;-P -- Daniel P. Brown

I think I have some time to write a simple script in a few hours, I'd like to see what the performance difference is :) Tijnema

Well, keep in mind that I don't expect that you'd see much difference at all with a single file, but try doing multiple files of ASCII and binary, various sizes, and also several simultaneous connections. Two other things I forgot to mention with database storage are a pro and con, respectively - caching and accessibility.

and the fact that a lot of hosting environments have a DB on a different machine. This would make for a much worse performance then the local filesystem. -- Jim Lucas "Some men are born to greatness, some achieve greatness, and some have greatness thrust upon them." Twelfth Night, Act II, Scene V by William Shakespeare