Hi Roman,
Is this with the ceph fuse client or the ceph kernel module?
Its not surprising that the local file system (/home) is so much faster
than a mounted ceph volume, especially the first time the directory tree
is traversed (metadata results are cached at the client to improve
performance). Try running the same find command on the ceph volume and
see if the cached results at the client improve performance at all.
In order to understand what the performance of ceph should be capable of
doing with your deployment for this specific workload, you should run
iperf between two nodes to get an idea of your latency limits.
Also, I noticed that the real timings you listed for ceph and /home are
offset by exactly 17 minutes (user and sys are identical). Was that a
copy/paste error, by chance?
-sam
On 10/29/2012 09:01 AM, Roman Alekseev wrote:
Hi,
Kindly guide me how to improve performance on the cluster which consist
of 5 dedicated servers:
- ceph.conf: http://pastebin.com/hT3qEhUF
- file system on all drives is ext4
- mount options "user_xattr"
- each server has :
CPU:Intel® Xeon® Processor E5335(8M Cache, 2.00 GHz, 1333 MHz FSB) x2
MEM: 4Gb DDR2
- 1Gb network
Simple test:
mounted as ceph
root@client1:/mnt/mycephfs# time find . | wc -l
83932
real 17m55.399s
user 0m0.152s
sys 0m1.528s
on 1 HDD:
root@client1:/home# time find . | wc -l
83932
real 0m55.399s
user 0m0.152s
sys 0m1.528s
Please help me to find out the issue. Thanks.
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
