|
On 2018-06-12 01:01, Jialin Liu wrote:
Hello Ceph Community,
I used libradosstriper api to test the striping feature, it doesn't seem to improve the performance at all, can anyone advise what's wrong with my settings:
The rados object store testbed at my center has
osd: 48
oss: 4
monitor:2
pg number: 1024
replicated size: 3
I then used 1 processes 1 thread to do the test, by varying a few settings:
- stripe count from 1 to 48,
- and object size from 1MB to 128 MB (with stripe size 1MB, stripe size needs to be smaller than the rados object size),
- and file size from 100MB to 1.6GB,
The peak bandwidth among all tests is only 130MB/s, no difference in different tests.
I suspect that the IO got serialized in the rados layer, with some uncertain evidence in the libradosstriper source code (note the for loop):
...
·Striper::file_to_extents(cct(), format.c_str(), &l, off, len, 0, extents);
·for (vector<ObjectExtent>::iterator p = extents.begin(); p != extents.end(); ++p) {
r = m_ioCtx.aio_write(p->oid.name, rados_completion, oid_bl,
p->length, p->offset);
}
...
Could you please correct me if I misused or misunderstood any things? Thanks much.
Best,
Jialin
NERSC/LBNL
The striper source is correct, the for loop does aio writes so there is no serialization blocking.
/Maged
|
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
