On 2/09/2013 8:40 a.m., Golden Shadow wrote:
Hello there! I've read that the number of first level and second level aufs subdirectories should be selected so that the number of objects inside each second level subdirectory should be no more than few hundreds.
Not sure where that came from. "few hundreds" sounds like advice for working with FAT-16 formatted disks - or worse. Most of the modern filesystems can handle several thousands easily.
The real reason for these parameters is that some filesystems start producing errors. For example; Squid tores 2^24 objects in its cache_dir but - FAT16 fails with more than 2^14 files in one directory - and IIRC ext2 and/or ext3 starts giving me trouble around 2^16 files in one directory. I'm not sure about other OS, I've not hit their limits myself.
In one of the subdirectories of the following cache_dir on my 3.3.8 squid, there is more than 15000 objects! In other subdirectories, there is ZERO objects, is this normal?!
Yes. It depends entirely on how many objects are in the cache. The earlier fileno entries fill up first, so the directories where those fileo map to will show lots of files while later ones do not.
NOTE: when changing these L1/L2 values the entire cache_dir fileno->file mapping gets screwed up. So you need to erase the contents and use an empty location/directory to build the new structure shape inside.
cache_dir aufs /mnt/cachedrive2/small 250000 64 256 min-size=32000 max-size=200000
If you want to increase that spread you can change the 64 to 128. It should halve the number of files in the fullest directory. With 250GB storing 32KB-200KB sized objects you are looking at a total of between 1,310,720 and 8,192,000 objects in that particular cache.
Amos
