Hi Eric, we are also very interested in your dmclock work in ceph. May I ask if this branch wip_dmclock_clients has both clients and OSD support for dmclock? we want to fully test this implementations. thanks for your help. Sheng On Tue, Jun 13, 2017 at 7:14 AM, J. Eric Ivancich <ivancich@xxxxxxxxxx> wrote: > > On 06/02/2017 03:36 AM, Lijie wrote: > > Hi Eric, > > > > Our team has developed QOS feature on ceph using the dmclock library > > from community. We treat a rbd as a dmclock client instead of pool as . > > We tested our code and the result is confusing . > > > > > > > > Testing environment: single server with 16 cores , RAM of 32G, 8 > > non-systerm disks,each runs one OSD. We set > > osd_op_num_threads_per_shard=2, osd_op_num_shards=5 on each OSD. The > > Size of our RBD is 100G. The qos params of RBD is （r:1K, p:100, l:2k）, > > rbd_cache = false , allow_limit_break = false. > > > > > > > > Conclusion: we only got 1500 IOPS while the system serves much more > > than the limit value 2000. > > > > We used fio and adjusted osd_op_num_shards .And we found that iops runs > > bigger along as the growth of osd_op_num_shards. Finally it can break > > the limit value .( fio iops = 2300 while osd_op_num_shards = 20) . > > > > > > > > So would you mind to provide your test environment and result about > > dmclock library? And we would be appreciated if you offer some other > > direction . It will really be a great help for us. > > > > Are you using src/common/mClockPriorityQueue.h? Because it makes sure > allow_limit_break is true and asserts that as long as the dmclock queue > is not empty, it will retrieve a request when pull_request is called > (assert(pr.is_retn());). If allow_limit_break is false, that assert can > fail. > > I'm guessing you've created your own interface to the dmclock library, > because of the issue outlined above. > > The dmclock library has a simulator built on top of it, which you can > use to design various scenarios and see how it works. > > You can run it, for example, by: > > $ git clone git@xxxxxxxxxx:ceph/dmclock.git > $ cd dmclock/ > $ mkdir build > $ cd build > $ cmake .. > $ make dmclock-sims > $ sim/dmc_sim -c ../sim/dmc_sim_100th.conf > > where the file dmc_sim_100th.conf describes the scenario being simulated. > > I'm not fully clear I understand your approach. If you can describe it > in more detail, I'll try to address any questions. > > Eric > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
