With the caveat that inclusion of tiring v2 is new to the discussion (and leaving out discussion of CephFS), support for a storage class concept in RGW supported by lifecycle transitions has been discussed for quite a while, and is in the upstream road map (Trello). I've been looking for more concrete upstream discussion to link in this thread, but pending that, I'd like to emphasize that the concept is general and shouldn't need to depend on tieringv2, though for a subset of scenarios it could, once tieringv2 concretely exists. I don't think most scenarios depend on tieringv2 though. Most discussion I am familiar with has emphasized using an RGW mechanism to move data between pools, or between cloud tiers (a feature that is already being worked on). I think the idea of sharing a common framework of policy and lifecycle transitions between different strategies is interesting. Matt On Mon, Apr 2, 2018 at 6:19 PM, Robin H. Johnson <robbat2@xxxxxxxxxx> wrote: > On Mon, Apr 02, 2018 at 10:58:03AM -0700, Gregory Farnum wrote: >> Hmm, it sounds like you're interested in extending the RADOS >> cache-tier functionality for this. That is definitely a mistake; we >> have been backing off support for that over the past several releases. >> Sage has a plan for some "tiering v2" infrastructure (that integrates >> with SK Telecom's dedupe work) which might fit with this but I don't >> think it has any kind of timeline for completion. > Varada & others (Danny al-Graf, Dan @ CERN): > I was thinking of the intersection of these FlipCart ideas and tiering > v2 for RADOS for some of what we discussed at various parts of > Cephalocon: > > At the core of it, it's a bottom-most rung for tiering, that puts just > the data of an RADOS object somewhere EXTERNAL to Ceph. The metadata esp > OMAP would remain in Ceph. > > Various thoughts: > - External tier might be an extra copy of the data or just the ONLY copy > of the data. > - Possible external tiers: exports to disk, export to tape, export to > another cluster or cloud > - RGW: Exposed via S3-Glacier API functions to trigger a copy being > brought back from external tier to disk tier. > - CephFS: revitalizing the Hierarchal Storage Management calls like IRIS > XFS to push data of single files out to external (w/ ioctls to trigger > tier transitions). > - Both the RGW & CephFS piece need a means to queue & process > transitions asynchronously in both directions: > - RGW lifecycle says 'this object IS old enough to transition to lower > storage class now' > - RGW lifecycle doesn't say if the transition has actually happened > yet. > - RGW needs to be aware of the v2-tiering, and able to do explicit > transitions of objects between tiers. > > -- > Robin Hugh Johnson > Gentoo Linux: Dev, Infra Lead, Foundation Treasurer > E-Mail : robbat2@xxxxxxxxxx > GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85 > GnuPG FP : 7D0B3CEB E9B85B1F 825BCECF EE05E6F6 A48F6136 > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-821-5101 fax. 734-769-8938 cel. 734-216-5309 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
