Hi Casey, Got it. I was directed to the old code base. By the way, Since the testing case was used to exercise all of object stores. Strongly recommend to change the name from fio_ceph_filestore.cc to fio_ceph_objectstore.cc . And the code in fio_ceph_filestore.cc should be refactored to reflect that the whole objectstore will be supported by fio_ceph_objectstore.cc. what you think? Let me know if you need any help from my side. Regards, James -----Original Message----- From: Casey Bodley [mailto:cbodley@xxxxxxxxx] Sent: Thursday, July 09, 2015 12:32 PM To: James (Fei) Liu-SSI Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx Subject: Re: About Fio backend with ObjectStore API Hi James, Are you looking at the code from https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore? It uses ObjectStore::create() instead of new FileStore(). This allows us to exercise all of the object stores with the same code. Casey On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI <james.liu@xxxxxxxxxxxxxxx> wrote: > Hi Casey, > Here is the code in the fio_ceph_filestore.cc. Basically, it creates a filestore as backend engine for IO exercises. If we got to send IO commands to KeyValue Store or Newstore, we got to change the code accordingly, right? I did not see any other files like fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In my humble opinion, we might need to create other two fio engines for keyvaluestore and newstore if we want to exercise these two, right? > > Regards, > James > > static int fio_ceph_filestore_init(struct thread_data *td) > 209 { > 210 vector<const char*> args; > 211 struct ceph_filestore_data *ceph_filestore_data = (struct ceph_filestore_data *) td->io_ops->data; > 212 ObjectStore::Transaction ft; > 213 > > 214 global_init(NULL, args, CEPH_ENTITY_TYPE_OSD, CODE_ENVIRONMENT_UTILITY, 0); > 215 //g_conf->journal_dio = false; > 216 common_init_finish(g_ceph_context); > 217 //g_ceph_context->_conf->set_val("debug_filestore", "20"); > 218 //g_ceph_context->_conf->set_val("debug_throttle", "20"); > 219 g_ceph_context->_conf->apply_changes(NULL); > 220 > > 221 ceph_filestore_data->osd_path = strdup("/mnt/fio_ceph_filestore.XXXXXXX"); > 222 ceph_filestore_data->journal_path = strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX"); > 223 > > 224 if (!mkdtemp(ceph_filestore_data->osd_path)) { > 225 cout << "mkdtemp failed: " << strerror(errno) << std::endl; > 226 return 1; > 227 } > 228 //mktemp(ceph_filestore_data->journal_path); // NOSPC issue > 229 > > 230 ObjectStore *fs = new FileStore(ceph_filestore_data->osd_path, ceph_filestore_data->journal_path); > 231 ceph_filestore_data->fs = fs; > 232 > > 233 if (fs->mkfs() < 0) { > 234 cout << "mkfs failed" << std::endl; > 235 goto failed; > 236 } > 237 > 238 if (fs->mount() < 0) { > 239 cout << "mount failed" << std::endl; > 240 goto failed; > 241 } > 242 > > 243 ft.create_collection(coll_t()); > 244 fs->apply_transaction(ft); > 245 > > 246 > > 247 return 0; > 248 > > 249 failed: > 250 return 1; > 251 > > 252 } > -----Original Message----- > From: Casey Bodley [mailto:cbodley@xxxxxxxxx] > Sent: Thursday, July 09, 2015 9:19 AM > To: James (Fei) Liu-SSI > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx > Subject: Re: About Fio backend with ObjectStore API > > Hi James, > > In the job file src/test/filestore.fio, you can modify the line > "objectstore=filestore" to use any objectstore type supported by the > ObjectStore::create() factory. > > Casey > > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI <james.liu@xxxxxxxxxxxxxxx> wrote: >> Hi Casey, >> Quick questions, The code in the trunk only cover the test for filestore. I was wondering do you have any plan to cover the test for kvstore and newstore? >> >> Thanks, >> James >> >> -----Original Message----- >> From: ceph-devel-owner@xxxxxxxxxxxxxxx >> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of James (Fei) >> Liu-SSI >> Sent: Tuesday, June 30, 2015 2:19 PM >> To: Casey Bodley >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >> Subject: RE: About Fio backend with ObjectStore API >> >> Hi Casey, >> >> Thanks a lot. >> >> Regards, >> James >> >> -----Original Message----- >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >> Sent: Tuesday, June 30, 2015 2:16 PM >> To: James (Fei) Liu-SSI >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >> Subject: Re: About Fio backend with ObjectStore API >> >> Hi, >> >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance >> Analysis: fio and RBD" at >> https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analy >> s is_fio_rbd.html, they also mentioned a fio engine that linked >> directly into ceph's FileStore. I was able to find Daniel's branch on >> github at https://github.com/gollub/ceph/tree/fio_filestore_v2, and did some more work on it at the time. >> >> I just rebased that work onto the latest ceph master branch, and pushed to our github at https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore. You can find the source in src/test/fio_ceph_filestore.cc, and run fio with the provided example fio job file in src/test/filestore.fio. >> >> I didn't have a chance to confirm that it builds with automake, but >> the cmake version built for me. I'm happy to help if you run into >> problems, Casey >> >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI <james.liu@xxxxxxxxxxxxxxx> wrote: >>> Hi Haomai, >>> What are you trying to ask is to benchmark local objectstore(like kvstore/filestore/newstore) locally with FIO(ObjectStore engine)? You want to purely compare the performance locally for these objectstores, right? >>> >>> Regards, >>> James >>> >>> -----Original Message----- >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Haomai Wang >>> Sent: Tuesday, June 30, 2015 9:06 AM >>> To: ceph-devel@xxxxxxxxxxxxxxx >>> Subject: About Fio backend with ObjectStore API >>> >>> Hi all, >>> >>> Long long ago, is there someone said about fio backend with Ceph ObjectStore API? So we could use the existing mature fio facility to benchmark ceph objectstore. >>> >>> -- >>> Best Regards, >>> >>> Wheat >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo >>> info at http://vger.kernel.org/majordomo-info.html >> {.n + +% lzwm b 맲 r yǩ ׯzX ܨ} Ơz &j:+v zZ+ +zf h ~ i z w ? & )ߢ f ��.n��������+%������w��{.n����z��u���ܨ}���Ơz�j:+v�����w����ޙ��&�)ߡ�a����z�ޗ���ݢj��w�f