Re: copying small files to glusterfs is slow

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Trung,

I'm no Gluster developer but I might have some insights on your problem.

The big question is: how do you copy those files? Are you doing one at a
time (like a "cp" would)? If so, them that's the answer. From now on I'm
taking a 'yes' as the answer.

Copying one file means open, write, write, write, write and close. Every
operation suffers the network latency delay. That latency is around 0.2ms if
you are using a TCP/IP interconnect but it can go as down as 0.007ms if you
use the not-so-cheap infiniband interconnect.

Doing some cheap linear interpolation, it seems that, in your test system,
the time necessary to create a tiny file is 19 milliseconds. So, when you
say you needed 20 ms to create a 5kb file, 95% is file creation and 5% is
writing. But, when you are creating a 1MB in 100 ms, only 19% of it is file
creation. The writing speed I got here from the interpolation is 106mbps,
witch is close to the expected value.

I'm working in a project here that will have a few millions of small files.
But, in my case all the files are written (or read) in a different thread,
so, no matter how fast I need to be in files per second, I just need more
threads and more Gluster servers.

Best regards,
Daniel

On 5/28/07, Trung Le Thanh <trung_bmt@xxxxxxxxx> wrote:

Hi all,

I am new to glusterfs. During the last week I did some
testing and found that glusterfs took quite a long
time to deal with small files.

My configuration:
3 server, 1 clien linked by a 100Mbps switch
Centos 5
Glusterfs 1.2.3
Fuse 2.6.5
The server and client vol files are the same as in the
wiki

The speed of copying 5kb files is:    ~2000kbit/s
The speed of copying 50kb files is:  ~10000kbit/s
The speed of copying 300kb files is: ~40000kbit/s
The speed of copying 600kb files is: ~70000kbit/s
The speed of copying 1MB files is:   ~85000kbit/s

The above values are average speed resulted from
copying 4000 files of each size.

I also tested with different sizes of the aggregate
value of write-behind and I saw no improvement when
dealing with small files. Can anyone help me on this?

Thank you very much
Best regards,
Trung



_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxx
http://lists.nongnu.org/mailman/listinfo/gluster-devel



[Index of Archives]     [Gluster Users]     [Ceph Users]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux