On Wednesday, March 21, 2012 at 8:11 AM, madhusudhana wrote: > Hi all, > > Will increase in the number of OSD will increase performance in ceph > cluster? I mean to say, if ceph cluster is giving ~70MB/s with 3 OSD, > will increasing OSD number to 5, increase the performance to ~100MB/s ? It should, assuming similar OSD hardware. > I have re-created my entire ceph cluster with CentOS 6.2 and kernel > 3.2.11 and ceph 0.43. When i ran some tests, the results are far > better than my first cluster results. But, they are not yet near to > a normal linux machine or a entry level NetApp storage box. I have tested > with cp, rsync, rm and dd commands. The performance what I am getting from > ceph are not satisfactory (test with my actual work load is even > worse). For instance, rsync of 10G directory took 84 mins to complete > in Linux machine (in NetApp it was 89mins). But, the same rsync took like > 10hrs to copy 10G of data. 1GB/hour is certainly not good! Does the rsync cover folders with very large numbers of files in them (tens of thousands)? These are currently slow on the MDS, although we can give you some features to enable that should remove that as a factor. (As with the rest of the filesystem, they're not production ready, but we'll probably turn them on by default as soon as we get the time to run more large-scale tests with them.) If you aren't using very large folders, then either your MDS is CPU constrained, your OSDs aren't fast enough, or something horrible is happening. How much CPU does the ceph-mds process use up while running the rsync? What client are you doing the rsync from? (Kernel or ceph-fuse?) What are the results of "rados -p data bench write 60"? How much memory are your daemons using? -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
