Re: Slow ceph fs performance

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



greg@xxxxxxxxxxx said:
> >
>         The rados benchmark was run on one of the OSD
> machines.  Read and write results looked like this (the
> objects size was just the default, which seems to be 4kB):
> Actually, that's 4MB. ;) 

	Oops! My plea is that I was the victim of a 
man page bug:

       bench seconds mode [ -b objsize ] [ -t threads ]
              Benchmark  for  seconds.  The  mode  can  be  write or read. The
              default object size is 4 KB, and the default number of simulated
              threads (parallel writes) is 16.


> Can you run # rados bench -p pbench 900 write -t 256
> -b 4096 and see what that gets? It'll run 256 simultaneous 4KB writes. (You
> can also vary the number of simultaneous writes and see if that impacts it.)

	Here's the new benchmark output:

 Total time run:         900.880070
Total writes made:      537187
Write size:             4096
Bandwidth (MB/sec):     2.329 

Stddev Bandwidth:       2.57691
Max bandwidth (MB/sec): 12.6055
Min bandwidth (MB/sec): 0
Average Latency:        0.429315
Stddev Latency:         0.891734
Max latency:            19.7647
Min latency:            0.016743
	

> However, my suspicion is that you're limited by metadata throughput here. How
> large are your files? There might be some MDS or client tunables we can
> adjust, but rsync's workload is a known weak spot for CephFS. -Greg 

	The file size is generally small.  Here's the distribution:

http://ayesha.phys.virginia.edu/~bryan/filesize.png

The mean is about 2.5 MB.

						Bryan

-- 
========================================================================
Bryan Wright              |"If you take cranberries and stew them like 
Physics Department        | applesauce, they taste much more like prunes
University of Virginia    | than rhubarb does."  --  Groucho 
Charlottesville, VA  22901|			
(434) 924-7218            |         bryan@xxxxxxxxxxxx
========================================================================

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux