Re: Metadata for LibRADOS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Right.  The elastic search integration--or something custom you could
base on s3 bucket notifications--would both be working with events
generated in RGW.

Matt

On Tue, Mar 2, 2021 at 3:55 PM Cary FitzHugh <cary.fitzhugh@xxxxxxxxx> wrote:
>
> Understood.
>
> With the RGW architecture comes more load balancing concerns, more moving parts, more tedious (to me) ACLs, less features (append and some other things not supported in S3).  Was hoping for a solution which didn't require us to be hamstrung and only read / write to a pool via the gateway.
>
> If the RGW Metadata search was able to "source" it's data from the OSDs and sync that way, then I'd be up for setting up a skeleton implementation,  but it sounds like RGW Metadata is only going to record things which are flowing through the gateway.  (Is that correct?)
>
>
>
>
> On Tue, Mar 2, 2021 at 3:46 PM Matt Benjamin <mbenjami@xxxxxxxxxx> wrote:
>>
>> Hi Cary,
>>
>> As you've said, these are well-developed features of RGW, I think that
>> would be the way to go, in the Ceph ecosystem.
>>
>> Matt
>>
>> On Tue, Mar 2, 2021 at 3:41 PM Cary FitzHugh <cary.fitzhugh@xxxxxxxxx> wrote:
>> >
>> > Hello -
>> >
>> > We're trying to use native libRADOS and the only challenge we're running
>> > into is searching metadata.
>> >
>> > Using the rgw metadata sync seems to require all data to be pushed through
>> > the rgw, which is not something we're interested in setting up at the
>> > moment.
>> >
>> > Are there hooks or features of libRADOS which could be leveraged to enable
>> > syncing of metadata to an external system (elastic-search / postgres / etc)?
>> >
>> > Is there a way to listen to a stream of updates to a pool in real-time,
>> > with some guarantees I wouldn't miss things?
>> >
>> > Are there any features like this in libRADOS?
>> >
>> > Thank you
>> > _______________________________________________
>> > ceph-users mailing list -- ceph-users@xxxxxxx
>> > To unsubscribe send an email to ceph-users-leave@xxxxxxx
>> >
>>
>>
>> --
>>
>> Matt Benjamin
>> Red Hat, Inc.
>> 315 West Huron Street, Suite 140A
>> Ann Arbor, Michigan 48103
>>
>> http://www.redhat.com/en/technologies/storage
>>
>> tel.  734-821-5101
>> fax.  734-769-8938
>> cel.  734-216-5309
>>


-- 

Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux