Re: OK to have many files in one folder?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Since it depends on many factors, some out of your control, some others that might vary over the lifetime of the application, one thing you might do, to be on the safe side is try to put files in different directories, for example, by breaking up their filenames every three characters and making each segment a folder but no more than three levels deep so that the file ./abcdefghijkl.jpg would be located at ./abc/def/ghijkl.jpg. This breakup would allow even to mount extra disk volumes for some of the folders, should they exceed certain capacity or to split the risk. Whether to break filenames every two, three or more characters might be a matter of optimization, but I would guess that around three (or four) for the first two segments and whatever is left for the filename would be a reasonable number. I wouldn't go more than three levels deep either, because jumping from one directory level to the next also takes some time, thus it is a compromise in between searching sequentially in a directory for a filename (for those filesystems that do so) and going deep into the directory tree.

Satyam



----- Original Message ----- From: "Brian Dunning" <brian@xxxxxxxxxxxxxxxx>
To: <php-general@xxxxxxxxxxxxx>
Sent: Friday, June 15, 2007 9:38 PM
Subject: Re:  OK to have many files in one folder?


Thanks for the replies but I'm not sure I know what to do with them. Is the problem with the number of files, or is the problem with the activity? Can you dumb down your answers at all for me? :)


On Jun 15, 2007, at 11:51 AM, Daniel Brown wrote:

6/15/07, Jay Blanchard <jblanchard@xxxxxxxxxx> wrote:
[snip]
Server is running Linux, and PHP is constantly creating and modifying
images in a directory. Apache is constantly serving these same
images. There are about 250,000 of them in the same directory. Seems
to be running OK, no problematic CPU load, but I'm wondering if
anyone knows whether I'm living dangerously having that many docs in
one directory with that much activity. It is an extremely busy  server.

Sorry is this seems like more of a linux sysad question than a PHP
question. Thanks...  :)
[/snip]

Two words. Beware the inode.

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



   If that directory is all on one single partition, God help you if
there's any corruption.  Check into RAID 0+1/RAID 5 disk striping if
you've got that many images.  And remember, the fact that they're all
in one directory doesn't matter at all to the system, as directories,
folders, et cetera, are just representations for human readability and
organization.  In fact, those files reside on several sectors
throughout the drive, and each file itself is probably fragmented many
times.

Just for fun, though, if you want to crash your server, you could always do:
       `while [ 1 = 1 ]; do ls -l;done`

--
Daniel P. Brown
[office] (570-) 587-7080 Ext. 272
[mobile] (570-) 766-8107

--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php



--
No virus found in this incoming message.
Checked by AVG Free Edition. Version: 7.5.472 / Virus Database: 269.8.16/849 - Release Date: 14/06/2007 12:44



--
PHP General Mailing List (http://www.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php


[Index of Archives]     [PHP Home]     [Apache Users]     [PHP on Windows]     [Kernel Newbies]     [PHP Install]     [PHP Classes]     [Pear]     [Postgresql]     [Postgresql PHP]     [PHP on Windows]     [PHP Database Programming]     [PHP SOAP]

  Powered by Linux