Mark, If you need, we can create a patch and send it you. Please let us know. Mana On 6/8/16, 6:12 PM, "Mark Nelson" <mnelson@xxxxxxxxxx> wrote: >ugh, that might explain why increasing the rocksdb compaction threads >and max concurrent compactions didn't help me at all today. > >I'll try to take a look at: > >rocksdb::DB::Open in RocksDBStore::do_open opt.IncreaseParallelism(16); >opt.OptimizeLevelStyleCompaction(512 * 1024 * 1024); > >Thanks, >Mark > >On 06/08/2016 08:08 PM, Somnath Roy wrote: >> Great, thanks..I was checking rocksdb tree.. >> Also, one thing I observed that rocksdb::GetOptionsFromString is >>expecting options to be separated by ';' and not by ',' like we are >>doing in bluestore_rocksdb_options. >> >> >>https://github.com/facebook/rocksdb/blob/533cda90cec8588fe13832833cd2c288 >>f3ae786f/util/options_test.cc#L601 >> >>https://github.com/facebook/rocksdb/blob/533cda90cec8588fe13832833cd2c288 >>f3ae786f/util/options_helper.cc#L709 >> >> It seems none of the options we are passing is setting correctly in >>rocksdb ? >> I will add some print to verify. >> >> Thanks & Regards >> Somnath >> >> -----Original Message----- >> From: Jianjian Huo [mailto:jianjian.huo@xxxxxxxxxxx] >> Sent: Wednesday, June 08, 2016 5:50 PM >> To: Somnath Roy; Allen Samuels; Manavalan Krishnan; Mark Nelson; Ceph >>Development >> Subject: RE: RocksDB tuning >> >> >> On Wed, Jun 8, 2016 at 5:47 PM, Jianjian Huo <jianjian.huo@xxxxxxxxxxx> >>wrote: >>> >>> >>> -----Original Message----- >>> From: Somnath Roy [mailto:Somnath.Roy@xxxxxxxxxxx] >>> Sent: Wednesday, June 08, 2016 5:39 PM >>> To: Jianjian Huo; Allen Samuels; Manavalan Krishnan; Mark Nelson; Ceph >>> Development >>> Subject: RE: RocksDB tuning >>> >>> Jianjian, >>> Couldn't find options like compaction_threads/flusher_threads in >>>rocksdb tree. Max_* option can be done by options. >>> It seems it is increasing through env->SetBackgroundThreads(). Am I >>>missing anything ? >> >> It's handled by Ceph kv and then passed to rocksdb. See below >> https://github.com/ceph/ceph/blob/master/src/kv/RocksDBStore.cc#L148 >>> >>> Thanks & Regards >>> Somnath >>> >>> -----Original Message----- >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Jianjian Huo >>> Sent: Wednesday, June 08, 2016 5:31 PM >>> To: Allen Samuels; Manavalan Krishnan; Mark Nelson; Ceph Development >>> Subject: RE: RocksDB tuning >>> >>> To just increase compaction and flusher threads, I think we can also >>>configure rocksdb with below settings as well? >>> >>> bluestore_rocksdb_options = >>> "...compaction_threads=, flusher_threads=, >>>max_background_compactions=, max_background_flushes=..." >>> >>> Regards, >>> Jianjian >>> >>>> -----Original Message----- >>>> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Allen Samuels >>>> Sent: Wednesday, June 08, 2016 4:53 PM >>>> To: Manavalan Krishnan; Mark Nelson; Ceph Development >>>> Subject: RE: RocksDB tuning >>>> >>>> Let's make a patch that creates actual Ceph parameters for these >>>>things so that we don't have to edit the source code in the future. >>>> >>>> >>>> Allen Samuels >>>> SanDisk |a Western Digital brand >>>> 2880 Junction Avenue, San Jose, CA 95134 >>>> T: +1 408 801 7030| M: +1 408 780 6416 allen.samuels@xxxxxxxxxxx >>>> >>>> >>>> -----Original Message----- >>>> From: ceph-devel-owner@xxxxxxxxxxxxxxx [mailto:ceph-devel- >>>> owner@xxxxxxxxxxxxxxx] On Behalf Of Manavalan Krishnan >>>> Sent: Wednesday, June 08, 2016 3:10 PM >>>> To: Mark Nelson <mnelson@xxxxxxxxxx>; Ceph Development <ceph- >>>> devel@xxxxxxxxxxxxxxx> >>>> Subject: RocksDB tuning >>>> >>>> Hi Mark >>>> >>>> Here are the tunings that we used to avoid the IOPs choppiness caused >>>> by rocksdb compaction. >>>> >>>> We need to add the following options in src/kv/RocksDBStore.cc before >>>> rocksdb::DB::Open in RocksDBStore::do_open >>>>opt.IncreaseParallelism(16); >>>> opt.OptimizeLevelStyleCompaction(512 * 1024 * 1024); >>>> >>>> >>>> >>>> Thanks >>>> Mana >>>> >>>> >>>>> >>>> >>>> PLEASE NOTE: The information contained in this electronic mail >>>> message is intended only for the use of the designated recipient(s) >>>> named above. If the reader of this message is not the intended >>>> recipient, you are hereby notified that you have received this >>>> message in error and that any review, dissemination, distribution, or >>>> copying of this message is strictly prohibited. If you have received >>>> this communication in error, please notify the sender by telephone or >>>> e-mail (as shown above) immediately and destroy any and all copies of >>>> this message in your possession (whether hard copies or >>>>electronically stored copies). >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo >>>> info at http://vger.kernel.org/majordomo-info.html >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo >>> info at http://vger.kernel.org/majordomo-info.html >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo >>> info at http://vger.kernel.org/majordomo-info.html >>> PLEASE NOTE: The information contained in this electronic mail message >>>is intended only for the use of the designated recipient(s) named >>>above. If the reader of this message is not the intended recipient, you >>>are hereby notified that you have received this message in error and >>>that any review, dissemination, distribution, or copying of this >>>message is strictly prohibited. If you have received this communication >>>in error, please notify the sender by telephone or e-mail (as shown >>>above) immediately and destroy any and all copies of this message in >>>your possession (whether hard copies or electronically stored copies). >>> PLEASE NOTE: The information contained in this electronic mail message is intended only for the use of the designated recipient(s) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify the sender by telephone or e-mail (as shown above) immediately and destroy any and all copies of this message in your possession (whether hard copies or electronically stored copies). -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
