Unfortunately, nothing like this exists in RADOS. It can't really -- scaling is inimical to the sort of data collation you seem to be looking for. If you use librados, you need to maintain all your own metadata. RGW has done a lot of work to support these features; depending on what you need you may be able to make do by implementing your own centralized metadata system in other objects (omap might be useful); if not you'll need to set up a more extensive system. -Greg On Tue, Mar 2, 2021 at 1:12 PM Cary FitzHugh <cary.fitzhugh@xxxxxxxxx> wrote: > > Phooey. :) > > Do you know of any notification subsystems in libRADOS that might be > useful? > > Will have to think on this... > > Thanks > > On Tue, Mar 2, 2021 at 4:05 PM Matt Benjamin <mbenjami@xxxxxxxxxx> wrote: > > > Right. The elastic search integration--or something custom you could > > base on s3 bucket notifications--would both be working with events > > generated in RGW. > > > > Matt > > > > On Tue, Mar 2, 2021 at 3:55 PM Cary FitzHugh <cary.fitzhugh@xxxxxxxxx> > > wrote: > > > > > > Understood. > > > > > > With the RGW architecture comes more load balancing concerns, more > > moving parts, more tedious (to me) ACLs, less features (append and some > > other things not supported in S3). Was hoping for a solution which didn't > > require us to be hamstrung and only read / write to a pool via the gateway. > > > > > > If the RGW Metadata search was able to "source" it's data from the OSDs > > and sync that way, then I'd be up for setting up a skeleton > > implementation, but it sounds like RGW Metadata is only going to record > > things which are flowing through the gateway. (Is that correct?) > > > > > > > > > > > > > > > On Tue, Mar 2, 2021 at 3:46 PM Matt Benjamin <mbenjami@xxxxxxxxxx> > > wrote: > > >> > > >> Hi Cary, > > >> > > >> As you've said, these are well-developed features of RGW, I think that > > >> would be the way to go, in the Ceph ecosystem. > > >> > > >> Matt > > >> > > >> On Tue, Mar 2, 2021 at 3:41 PM Cary FitzHugh <cary.fitzhugh@xxxxxxxxx> > > wrote: > > >> > > > >> > Hello - > > >> > > > >> > We're trying to use native libRADOS and the only challenge we're > > running > > >> > into is searching metadata. > > >> > > > >> > Using the rgw metadata sync seems to require all data to be pushed > > through > > >> > the rgw, which is not something we're interested in setting up at the > > >> > moment. > > >> > > > >> > Are there hooks or features of libRADOS which could be leveraged to > > enable > > >> > syncing of metadata to an external system (elastic-search / postgres > > / etc)? > > >> > > > >> > Is there a way to listen to a stream of updates to a pool in > > real-time, > > >> > with some guarantees I wouldn't miss things? > > >> > > > >> > Are there any features like this in libRADOS? > > >> > > > >> > Thank you > > >> > _______________________________________________ > > >> > ceph-users mailing list -- ceph-users@xxxxxxx > > >> > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > >> > > > >> > > >> > > >> -- > > >> > > >> Matt Benjamin > > >> Red Hat, Inc. > > >> 315 West Huron Street, Suite 140A > > >> Ann Arbor, Michigan 48103 > > >> > > >> http://www.redhat.com/en/technologies/storage > > >> > > >> tel. 734-821-5101 > > >> fax. 734-769-8938 > > >> cel. 734-216-5309 > > >> > > > > > > -- > > > > Matt Benjamin > > Red Hat, Inc. > > 315 West Huron Street, Suite 140A > > Ann Arbor, Michigan 48103 > > > > http://www.redhat.com/en/technologies/storage > > > > tel. 734-821-5101 > > fax. 734-769-8938 > > cel. 734-216-5309 > > > > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |