Re: GFS and Small Files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



Hi,

On Wed, Apr 29, 2009 at 08:35, William L. Maltby
<CentOS4Bill@xxxxxxxxxxxx> wrote:
> One thing to keep in mind is that ls must sort the file list.

Not only sorting, but usually "ls" ends up trying to find out if the
file is a directory, which uses a "stat" syscall for each of the
files.

This is always expensive on remote filesystems (e.g., NFS) and I would
expect it to be also the case in GFS.

Besides the "-U" option of "ls", you might want to look at other
options that would make "ls" not try to "stat" each file. I'm not sure
exactly which of them, but one I can think of is disabling color
(since color will trigger a "stat" to find if the file is a
directory).

In general, having directories with a huge number of files tends to be
a bad idea, you will most likely have performance bottlenecks with
specific filesystems or tools. If possible, try to change the
application to create two or three levels of directories using a hash
on the filename and then creating directories with a small number of
files on each of them.

HTH,
Filipe
_______________________________________________
CentOS mailing list
CentOS@xxxxxxxxxx
http://lists.centos.org/mailman/listinfo/centos

[Index of Archives]     [CentOS]     [CentOS Announce]     [CentOS Development]     [CentOS ARM Devel]     [CentOS Docs]     [CentOS Virtualization]     [Carrier Grade Linux]     [Linux Media]     [Asterisk]     [DCCP]     [Netdev]     [Xorg]     [Linux USB]
  Powered by Linux