Hi Haomai, I was able to run this after a couple changes to the filestore.fio job file. Two of the config options were using the wrong names. I pushed a fix for the job file, as well as a patch that renames everything from filestore to objectstore (thanks James), to https://github.com/linuxbox2/linuxbox-ceph/commits/fio-objectstore. I found that the read support doesn't appear to work anymore, so give "rw=write" a try. And because it does a mkfs(), make sure you're pointing it to an empty xfs directory with the "directory=" option. Casey On Tue, Jul 14, 2015 at 2:45 AM, Haomai Wang <haomaiwang@xxxxxxxxx> wrote: > Anyone who have successfully ran the fio with this external io engine > ceph_objectstore? > > It's strange that it alwasys hit segment fault when entering > "_fio_setup_ceph_filestore_data", gdb tells "td->io_ops" is NULL but > when I up the stack, the "td->io_ops" is not null. Maybe it's related > to dlopen? > > On Fri, Jul 10, 2015 at 3:51 PM, Haomai Wang <haomaiwang@xxxxxxxxx> wrote: >> I have rebased the branch with master, and push it to ceph upstream >> repo. https://github.com/ceph/ceph/compare/fio-objectstore?expand=1 >> >> Plz let me know if who is working on this. Otherwise, I would like to >> improve this to be merge ready. >> >> On Fri, Jul 10, 2015 at 4:26 AM, Matt W. Benjamin <matt@xxxxxxxxxxxx> wrote: >>> That makes sense. >>> >>> Matt >>> >>> ----- "James (Fei) Liu-SSI" <james.liu@xxxxxxxxxxxxxxx> wrote: >>> >>>> Hi Casey, >>>> Got it. I was directed to the old code base. By the way, Since the >>>> testing case was used to exercise all of object stores. Strongly >>>> recommend to change the name from fio_ceph_filestore.cc to >>>> fio_ceph_objectstore.cc . And the code in fio_ceph_filestore.cc should >>>> be refactored to reflect that the whole objectstore will be supported >>>> by fio_ceph_objectstore.cc. what you think? >>>> >>>> Let me know if you need any help from my side. >>>> >>>> >>>> Regards, >>>> James >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >>>> Sent: Thursday, July 09, 2015 12:32 PM >>>> To: James (Fei) Liu-SSI >>>> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>>> Subject: Re: About Fio backend with ObjectStore API >>>> >>>> Hi James, >>>> >>>> Are you looking at the code from >>>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore? It >>>> uses ObjectStore::create() instead of new FileStore(). This allows us >>>> to exercise all of the object stores with the same code. >>>> >>>> Casey >>>> >>>> On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI >>>> <james.liu@xxxxxxxxxxxxxxx> wrote: >>>> > Hi Casey, >>>> > Here is the code in the fio_ceph_filestore.cc. Basically, it >>>> creates a filestore as backend engine for IO exercises. If we got to >>>> send IO commands to KeyValue Store or Newstore, we got to change the >>>> code accordingly, right? I did not see any other files like >>>> fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In my humble >>>> opinion, we might need to create other two fio engines for >>>> keyvaluestore and newstore if we want to exercise these two, right? >>>> > >>>> > Regards, >>>> > James >>>> > >>>> > static int fio_ceph_filestore_init(struct thread_data *td) >>>> > 209 { >>>> > 210 vector<const char*> args; >>>> > 211 struct ceph_filestore_data *ceph_filestore_data = (struct >>>> ceph_filestore_data *) td->io_ops->data; >>>> > 212 ObjectStore::Transaction ft; >>>> > 213 >>>> > >>>> > 214 global_init(NULL, args, CEPH_ENTITY_TYPE_OSD, >>>> CODE_ENVIRONMENT_UTILITY, 0); >>>> > 215 //g_conf->journal_dio = false; >>>> > 216 common_init_finish(g_ceph_context); >>>> > 217 //g_ceph_context->_conf->set_val("debug_filestore", "20"); >>>> > 218 //g_ceph_context->_conf->set_val("debug_throttle", "20"); >>>> > 219 g_ceph_context->_conf->apply_changes(NULL); >>>> > 220 >>>> > >>>> > 221 ceph_filestore_data->osd_path = >>>> strdup("/mnt/fio_ceph_filestore.XXXXXXX"); >>>> > 222 ceph_filestore_data->journal_path = >>>> strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX"); >>>> > 223 >>>> > >>>> > 224 if (!mkdtemp(ceph_filestore_data->osd_path)) { >>>> > 225 cout << "mkdtemp failed: " << strerror(errno) << >>>> std::endl; >>>> > 226 return 1; >>>> > 227 } >>>> > 228 //mktemp(ceph_filestore_data->journal_path); // NOSPC issue >>>> > 229 >>>> > >>>> > 230 ObjectStore *fs = new >>>> FileStore(ceph_filestore_data->osd_path, >>>> ceph_filestore_data->journal_path); >>>> > 231 ceph_filestore_data->fs = fs; >>>> > 232 >>>> > >>>> > 233 if (fs->mkfs() < 0) { >>>> > 234 cout << "mkfs failed" << std::endl; >>>> > 235 goto failed; >>>> > 236 } >>>> > 237 >>>> > 238 if (fs->mount() < 0) { >>>> > 239 cout << "mount failed" << std::endl; >>>> > 240 goto failed; >>>> > 241 } >>>> > 242 >>>> > >>>> > 243 ft.create_collection(coll_t()); >>>> > 244 fs->apply_transaction(ft); >>>> > 245 >>>> > >>>> > 246 >>>> > >>>> > 247 return 0; >>>> > 248 >>>> > >>>> > 249 failed: >>>> > 250 return 1; >>>> > 251 >>>> > >>>> > 252 } >>>> > -----Original Message----- >>>> > From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >>>> > Sent: Thursday, July 09, 2015 9:19 AM >>>> > To: James (Fei) Liu-SSI >>>> > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>>> > Subject: Re: About Fio backend with ObjectStore API >>>> > >>>> > Hi James, >>>> > >>>> > In the job file src/test/filestore.fio, you can modify the line >>>> > "objectstore=filestore" to use any objectstore type supported by >>>> the >>>> > ObjectStore::create() factory. >>>> > >>>> > Casey >>>> > >>>> > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI >>>> <james.liu@xxxxxxxxxxxxxxx> wrote: >>>> >> Hi Casey, >>>> >> Quick questions, The code in the trunk only cover the test for >>>> filestore. I was wondering do you have any plan to cover the test for >>>> kvstore and newstore? >>>> >> >>>> >> Thanks, >>>> >> James >>>> >> >>>> >> -----Original Message----- >>>> >> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>>> >> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of James (Fei) >>>> >>>> >> Liu-SSI >>>> >> Sent: Tuesday, June 30, 2015 2:19 PM >>>> >> To: Casey Bodley >>>> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>>> >> Subject: RE: About Fio backend with ObjectStore API >>>> >> >>>> >> Hi Casey, >>>> >> >>>> >> Thanks a lot. >>>> >> >>>> >> Regards, >>>> >> James >>>> >> >>>> >> -----Original Message----- >>>> >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >>>> >> Sent: Tuesday, June 30, 2015 2:16 PM >>>> >> To: James (Fei) Liu-SSI >>>> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>>> >> Subject: Re: About Fio backend with ObjectStore API >>>> >> >>>> >> Hi, >>>> >> >>>> >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance >>>> >> Analysis: fio and RBD" at >>>> >> >>>> https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analy >>>> >> s is_fio_rbd.html, they also mentioned a fio engine that linked >>>> >> directly into ceph's FileStore. I was able to find Daniel's branch >>>> on >>>> >> github at https://github.com/gollub/ceph/tree/fio_filestore_v2, and >>>> did some more work on it at the time. >>>> >> >>>> >> I just rebased that work onto the latest ceph master branch, and >>>> pushed to our github at >>>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore. You >>>> can find the source in src/test/fio_ceph_filestore.cc, and run fio >>>> with the provided example fio job file in src/test/filestore.fio. >>>> >> >>>> >> I didn't have a chance to confirm that it builds with automake, but >>>> >>>> >> the cmake version built for me. I'm happy to help if you run into >>>> >> problems, Casey >>>> >> >>>> >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI >>>> <james.liu@xxxxxxxxxxxxxxx> wrote: >>>> >>> Hi Haomai, >>>> >>> What are you trying to ask is to benchmark local >>>> objectstore(like kvstore/filestore/newstore) locally with >>>> FIO(ObjectStore engine)? You want to purely compare the performance >>>> locally for these objectstores, right? >>>> >>> >>>> >>> Regards, >>>> >>> James >>>> >>> >>>> >>> -----Original Message----- >>>> >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>>> >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Haomai >>>> Wang >>>> >>> Sent: Tuesday, June 30, 2015 9:06 AM >>>> >>> To: ceph-devel@xxxxxxxxxxxxxxx >>>> >>> Subject: About Fio backend with ObjectStore API >>>> >>> >>>> >>> Hi all, >>>> >>> >>>> >>> Long long ago, is there someone said about fio backend with Ceph >>>> ObjectStore API? So we could use the existing mature fio facility to >>>> benchmark ceph objectstore. >>>> >>> >>>> >>> -- >>>> >>> Best Regards, >>>> >>> >>>> >>> Wheat >>>> >>> -- >>>> >>> To unsubscribe from this list: send the line "unsubscribe >>>> ceph-devel" >>>> >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More >>>> majordomo >>>> >>> info at http://vger.kernel.org/majordomo-info.html >>>> >> {.n + +% lzwm b 맲 r yǩ ׯzX ܨ} Ơz &j:+v zZ+ >>>> +zf h ~ i z w ? & )ߢ f >>>> N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"�� >>> >>> -- >>> Matt Benjamin >>> CohortFS, LLC. >>> 315 West Huron Street, Suite 140A >>> Ann Arbor, Michigan 48103 >>> >>> http://cohortfs.com >>> >>> tel. 734-761-4689 >>> fax. 734-769-8938 >>> cel. 734-216-5309 >> >> >> >> -- >> Best Regards, >> >> Wheat > > > > -- > Best Regards, > > Wheat -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
