Hi Sage, The primary QoS issue we see with OpenStack users is wanting to guarantee minimum IOPS to each Cinder-mounted RBD volume as a way to guarantee the health of well-mannered workloads against badly-behaving ones. As an OpenStack Administrator, I want to guarantee a minimum number of IOPS to each Cinder volume to prevent any tenant from interfering with another. The number of IOPS may vary per volume, but in many cases a "standard" and "high" number would probably suffice. The guarantee is more important than the granularity. This is something impacting users at today's Ceph performance level. Looking at the future, once Bluestore becomes the default, there will also be latency requirements from the crowd that wants to run databases with RBD backends — both low latency and low jitter in the latency, but rather than applying that to all volumes, it will be only to select ones backing RDBMs. Well, at least in the case of a general purpose cluster. My hunch is that Enterprise users that want hard-QoS guarantees will accept that a capacity planning exercise is necessary, software can only allocate existing capacity, not create more. Community users may value more some "fairness" in distributing existing resources instead. Just a hunch at this point. Best -F _________________________________________ -- "You must try until your brain hurts —Elon Musk (Federico L. Lucifredi) - federico at redhat.com - GnuPG 0x4A73884C On Fri, Dec 2, 2016 at 2:01 PM, Sage Weil <sweil@xxxxxxxxxx> wrote: > > Hi all, > > We're working on getting infrasture into RADOS to allow for proper > distributed quality-of-service guarantees. The work is based on the > mclock paper published in OSDI'10 > > https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Gulati.pdf > > There are a few ways this can be applied: > > - We can use mclock simply as a better way to prioritize background > activity (scrub, snap trimming, recovery, rebalancing) against client IO. > - We can use d-mclock to set QoS parameters (e.g., min IOPS or > proportional priority/weight) on RADOS pools > - We can use d-mclock to set QoS parameters (e.g., min IOPS) for > individual clients. > > Once the rados capabilities are in place, there will be a significant > amount of effort needed to get all of the APIs in place to configure and > set policy. In order to make sure we build somethign that makes sense, > I'd like to collection a set of user stores that we'd like to support so > that we can make sure we capture everything (or at least the important > things). > > Please add any use-cases that are important to you to this pad: > > http://pad.ceph.com/p/qos-user-stories > > or as a follow-up to this email. > > mClock works in terms of a minimum allocation (of IOPS or bandwidth; they > are sort of reduced into a single unit of work), a maximum (i.e. simple > cap), and a proportional weighting (to allocation any additional capacity > after the minimum allocations are satisfied). It's somewhat flexible in > terms of how we apply it to specific clients, classes of clients, or types > of work (e.g., recovery). How we put it all together really depends on > what kinds of things we need to accomplish (e.g., do we need to support a > guaranteed level of service shared across a specific set of N different > clients, or only individual clients?). > > Thanks! > sage > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |