On Tue, Apr 2, 2019 at 7:11 PM Maged Mokhtar <mmokhtar@xxxxxxxxxxx> wrote: > > > Hi all, > > would it be useful when building/re-building rbd object map as well as > snap diff map to have osd side processing multiple objects instead of > having to test 1 object at a time which is very slow. Technically, it tests up to "rbd_concurrent_management_ops" (defaults to 10) objects at a time. > One approach to speed things would be to extend the pgnls operation > filter to accept an object prefix + ability to access a snap context, > currently the filter interface only supports xattr. Searching potentially hundreds of millions of objects on the OSDs to find a very small percentage of related RBD data objects doesn't seem too efficient to me at first glance. You also wouldn't necessarily want to block the OSD's PG thread searching through and filtering tons of object metadata since that would impact other unrelated IO. To reduce the impact, you would want to limit the listing to querying some fixed amount of objects, which given the expected low percentage of hits, would result in 0 or very few actual results per query. > Another approach would be to add a new pg op that does hit testing, the > client would pass an object prefix + a vector of object_no and optional > to/from snap ids. The client would pre-assign the objects to pgs and > would send say 1k objects per op at a time. Sorting the objects in > reverse-hash may further speed the rocksdb lookups. > > Would either of these approaches be useful ? any gotchas ? > > /Maged -- Jason
