Am 10.09.2015 um 16:26 schrieb Haomai Wang:
Actually we can reach 700us per 4k write IO for single io depth(2 copy, E52650, 10Gib, intel s3700). So I think 400 read iops shouldn't be a unbridgeable problem. CPU is critical for ssd backend, so what's your cpu model?
Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz mostly running in turbo boost mode with 3.6 - 3.7 Ghz serving 6 SSDs. Stefan
On Thu, Sep 10, 2015 at 9:48 PM, Jan Schermer <jan@xxxxxxxxxxx <mailto:jan@xxxxxxxxxxx>> wrote: It's certainly not a problem with DRBD (yeah, it's something completely different but it's used for all kinds of workloads including things like replicated tablespaces for databases). It won't be a problem with VSAN (again, a bit different, but most people just want something like that) It surely won't be a problem with e.g. ScaleIO which should be comparable to Ceph. Latency on the network can be very low (0.05ms on my 10GbE). Latency on good SSDs is 2 orders of magnitute lower (as low as 0.00005 ms). Linux is pretty good nowadays at waking up threads and pushing the work. Multiply those numbers by whatever factor and it's still just a fraction of the 0.5ms needed. The problem is quite frankly slow OSD code and the only solution now is to keep the data closer to the VM. Jan > On 10 Sep 2015, at 15:38, Gregory Farnum <gfarnum@xxxxxxxxxx <mailto:gfarnum@xxxxxxxxxx>> wrote: > > On Thu, Sep 10, 2015 at 2:34 PM, Stefan Priebe - Profihost AG > <s.priebe@xxxxxxxxxxxx <mailto:s.priebe@xxxxxxxxxxxx>> wrote: >> Hi, >> >> while we're happy running ceph firefly in production and also reach >> enough 4k read iop/s for multithreaded apps (around 23 000) with qemu 2.2.1. >> >> We've now a customer having a single threaded application needing around >> 2000 iop/s but we don't go above 600 iop/s in this case. >> >> Any tuning hints for this case? > > If the application really wants 2000 sync IOPS to disk without any > parallelism, I don't think any network storage system is likely to > satisfy him — that's only half a millisecond per IO. 600 IOPS is about > the limit of what the OSD can do right now (in terms of per-op > speeds), and although there is some work being done to improve that > it's not going to be in a released codebase for a while. > > Or perhaps I misunderstood the question? > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx <mailto:ceph-users@xxxxxxxxxxxxxx> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Best Regards, Wheat _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
