Re: problems with large directories?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 03/08/2010 08:23 PM, Mitch Trachtenberg wrote:
Hi,

I have an application that deals with 100,000 to 1,000,000 image files.

I initially structured it to use multiple directories, so that file
123456 would be stored in /12/34/123456.  I'm now wondering if that's
pointless, as it would simplify things to simply store the file in /123456.

Can anyone indicate whether I'm gaining anything by using smaller
directories in ext3/ext4?  Thanks.

Mitch


I think that breaking up your files into subdirectories makes it easier to navigate the tree and find files from a human point of view. Even better if the bytes reflect something like year/month/day/hour/min (assuming your pathname has a date based guid or similar encoding).

You can have a million files in one large directory, but be careful to iterate and copy them in a sorted order (sorted by inode) to avoid nasty performance issues that are side effects of the way we hash file names in ext3/4.

Good luck!

Ric

_______________________________________________
Ext3-users mailing list
Ext3-users@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/ext3-users

[Index of Archives]         [Linux RAID]     [Kernel Development]     [Red Hat Install]     [Video 4 Linux]     [Postgresql]     [Fedora]     [Gimp]     [Yosemite News]

  Powered by Linux