On Fri, 4 Nov 2016, Somnath Roy wrote: > Ohh I see what you are saying, you are saying rather than merging two > files completely, take the first file and check for updates for in the > 2nd and flush only the 1st..May be we should take 1st one and check for > updates for the min_write_buffer_number_to_merge and flush the 1st > one..Likewise for others.. Exactly :) Could probably get fancy by adding options like min_write_buffer_to_merge_against or something so that we merge N + M and write N, but I don't think it matters too much. Either way, we're spending some extra CPU examining the next buffer, but we save on IO, and probably eventually save on CPU in the long term once you factor in compaction. sage > > -----Original Message----- > From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Somnath Roy > Sent: Friday, November 04, 2016 10:07 AM > To: Sage Weil > Cc: ceph-devel@xxxxxxxxxxxxxxx; g.marripudi@xxxxxxxxxxx > Subject: RE: rocksdb wal proposed improvement > > Sage, > My understanding was rocksdb is already doing this kind of merge before flush. I thought It will always go through min_write_buffer_number_to_merge memTable files and merge the updates, but, it will not be able to merge if same key update came beyond min_write_buffer_number_to_merge files. Am I missing anything ? > > Thanks & Regards > Somnath > > -----Original Message----- > From: Sage Weil [mailto:sweil@xxxxxxxxxx] > Sent: Friday, November 04, 2016 9:06 AM > To: Somnath Roy > Cc: ceph-devel@xxxxxxxxxxxxxxx; g.marripudi@xxxxxxxxxxx > Subject: rocksdb wal proposed improvement > > Here's a quick write-up of what I think we should improve in rocksdb: > > http://pad.ceph.com/p/rocksdb-wal-improvement > > sage > PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
