Hi Hans ! Do you think that an hexadecimal way could do fine ? a/b/c/d/e/f/0/1/2... That way : 16 subfolders per directory. Will that be fine (performance wise) ? Many thanks ! Regards, Alain > Date: Wed, 8 May 2013 10:10:21 +0200 > From: hans at shapeways.com > To: deco33 at hotmail.fr > CC: gluster-users at gluster.org > Subject: Re: How many subfolders in parent folders ? > > Hi Alain, > > alain meunier wrote on 20130507: > > I will have a lot (many hundreds of millions) of subdirectories inside a directory, named with md5 hashes. > > I thought of a subdividivision abcd/efgh/... or maybe a/b/c/... > > But it will create a bunch of subfolders. > > Will gluster handler those files without problems ? > > Yes, but slow. Very slow IMO :-/ To give you an idea : a recursive walk > through my Distributed-Replicate 14 x 2 filesystem (2M directories, max > 100 per subdirectory) takes several days. > > > One last thing : hardlinks will be involved to point the files and often deleted. > > Does someone has any experience of the troubleshooting I will have to face ? > > Should be fine. Try to avoid filename renames as that causes a DHT miss > penalty. > > My advice : first build a test setup. > > -- Hans > -- > Hans Lambermont | Senior Architect > (t) +31407370104 (w) www.shapeways.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130508/d8d2de45/attachment.html>
