How many subfolders in parent folders ?

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Alain,

alain meunier wrote on 20130507:
> I will have a lot (many hundreds of millions) of subdirectories inside a directory, named with md5 hashes.
> I thought of a subdividivision abcd/efgh/... or maybe a/b/c/...
> But it will create a bunch of subfolders.
> Will gluster handler those files without problems ?

Yes, but slow. Very slow IMO :-/ To give you an idea : a recursive walk
through my Distributed-Replicate 14 x 2 filesystem (2M directories, max
100 per subdirectory) takes several days.

> One last thing : hardlinks will be involved to point the files and often deleted.
> Does someone has any experience of the troubleshooting I will have to face ?

Should be fine. Try to avoid filename renames as that causes a DHT miss
penalty.

My advice : first build a test setup.

-- Hans
-- 
Hans Lambermont | Senior Architect
(t) +31407370104 (w) www.shapeways.com


[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux