(resending to list, sorry tv) On Tue, Jan 31, 2012 at 5:02 PM, Tommi Virtanen <tommi.virtanen@xxxxxxxxxxxxx> wrote: > > On Tue, Jan 31, 2012 at 16:33, Sage Weil <sage@xxxxxxxxxxxx> wrote: > > Currently rgw logs objects it wants to delete after some period of time, > > and an radosgw-admin command comes back later to process the log. It > > works, but is currently slow (one sync op at a time). > > > > A better approach would be to mark objects for later removal, and have the > > OSD do it in some more efficient way. wip-objs-expire has a client side > > (librados) interface for this. > > Is there some reason why this would be significantly more performant > when done by the OSD itself? It seems like the deletion times can be > bucketed by time nicely, then each bucket just contains a set of ids > -- a good fit for the map data type -- and the client for running this > deletion just streams the bucket contents over and issues delete > messages for everything. What makes that inherently slow? Random access to random cold objects is generally slower than doing the operations on a single pg. E.g., if doing it as part of the scrub, then objects are accessed anyway and are hopefully cached. > > > Should this be generalized to saying "do these osd ops at time X" instead > > of "delete at time X". Then it could setxattr, remove, call into a class, > > whatever. > > That sounds like a really complex API, for quite marginal gain. I do agree that for the sake of a garbage collection it's an overkill, however, > > To make my point even clearer: point me to another data store that has > that idiom. I can see its use, and even if not, I'm sure that there would be users who would need it. I don't think there's any relevance to the question whether there's any data store that implements that. Yehuda -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
