Thanks Pranith. We have tried two organizations with the same results. The levels we use currently are like this (indicates number of dirs at this level): -dir-top --runs (1) ---id (1) ----cam (4) -----date (1) ------h (6) -------m (60) --------s (60) We have also used a flatter structure too. I find that the client machine's glusterfs process uses 100% of the core processing. When I kill the ls (or find, mv, etc) command, usage returns to baseline on the client and daemons. -Jon On Nov 1, 2012 10:52 PM, "Pranith Kumar Karampuri" <pkarampu at redhat.com> wrote: > Jonathan, > Could you give us the directory structure details, so that we can try > to re-create the issue. I am assuming each file is about 300kb. Please give > us the depth of the directory structure and how many directories in each > level. > > Thanks > Pranith > ----- Original Message ----- > From: "Jonathan Lefman" <jonathan.lefman at essess.com> > To: gluster-users at gluster.org > Sent: Friday, November 2, 2012 5:33:21 AM > Subject: Very slow directory listing and high CPU usage on > replicated volume > > > Hi all, > > > I am having problems with painfully slow directory listings on a freshly > created replicated volume. The configuration is as follows: 2 nodes with 3 > replicated drives each. The total volume capacity is 5.6T. We would like to > expand the storage capacity much more, but first we need to figure this > problem out. > > > Soon after loading up about 100 MB of small files (about 300kb each), the > drive usage is at 1.1T. I am not sure if this to be expected. The main > problem is that directory listing (ls or find) takes a very long time. The > CPU usage on the nodes is high for each of the glusterfsd processes - 3 on > each machine 54%, 43%, and 25% per core is an example of the usage. Memory > is very low for each process. It is incredibly difficult to diagnose this > issue. We have wiped previous gluster installs, all directories, and mount > points as well as reformatting the disks. Each drive is formatted with ext4. > > > Has anyone had a similar result? Any ideas on how to debug this one? > > > Thank you, > > > Jon > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://supercolony.gluster.org/mailman/listinfo/gluster-users > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20121101/7bde7310/attachment.html>
