Anyone who have successfully ran the fio with this external io engine ceph_objectstore? It's strange that it alwasys hit segment fault when entering "_fio_setup_ceph_filestore_data", gdb tells "td->io_ops" is NULL but when I up the stack, the "td->io_ops" is not null. Maybe it's related to dlopen? On Fri, Jul 10, 2015 at 3:51 PM, Haomai Wang <haomaiwang@xxxxxxxxx> wrote: > I have rebased the branch with master, and push it to ceph upstream > repo. https://github.com/ceph/ceph/compare/fio-objectstore?expand=1 > > Plz let me know if who is working on this. Otherwise, I would like to > improve this to be merge ready. > > On Fri, Jul 10, 2015 at 4:26 AM, Matt W. Benjamin <matt@xxxxxxxxxxxx> wrote: >> That makes sense. >> >> Matt >> >> ----- "James (Fei) Liu-SSI" <james.liu@xxxxxxxxxxxxxxx> wrote: >> >>> Hi Casey, >>> Got it. I was directed to the old code base. By the way, Since the >>> testing case was used to exercise all of object stores. Strongly >>> recommend to change the name from fio_ceph_filestore.cc to >>> fio_ceph_objectstore.cc . And the code in fio_ceph_filestore.cc should >>> be refactored to reflect that the whole objectstore will be supported >>> by fio_ceph_objectstore.cc. what you think? >>> >>> Let me know if you need any help from my side. >>> >>> >>> Regards, >>> James >>> >>> >>> >>> -----Original Message----- >>> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >>> Sent: Thursday, July 09, 2015 12:32 PM >>> To: James (Fei) Liu-SSI >>> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>> Subject: Re: About Fio backend with ObjectStore API >>> >>> Hi James, >>> >>> Are you looking at the code from >>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore? It >>> uses ObjectStore::create() instead of new FileStore(). This allows us >>> to exercise all of the object stores with the same code. >>> >>> Casey >>> >>> On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI >>> <james.liu@xxxxxxxxxxxxxxx> wrote: >>> > Hi Casey, >>> > Here is the code in the fio_ceph_filestore.cc. Basically, it >>> creates a filestore as backend engine for IO exercises. If we got to >>> send IO commands to KeyValue Store or Newstore, we got to change the >>> code accordingly, right? I did not see any other files like >>> fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In my humble >>> opinion, we might need to create other two fio engines for >>> keyvaluestore and newstore if we want to exercise these two, right? >>> > >>> > Regards, >>> > James >>> > >>> > static int fio_ceph_filestore_init(struct thread_data *td) >>> > 209 { >>> > 210 vector<const char*> args; >>> > 211 struct ceph_filestore_data *ceph_filestore_data = (struct >>> ceph_filestore_data *) td->io_ops->data; >>> > 212 ObjectStore::Transaction ft; >>> > 213 >>> > >>> > 214 global_init(NULL, args, CEPH_ENTITY_TYPE_OSD, >>> CODE_ENVIRONMENT_UTILITY, 0); >>> > 215 //g_conf->journal_dio = false; >>> > 216 common_init_finish(g_ceph_context); >>> > 217 //g_ceph_context->_conf->set_val("debug_filestore", "20"); >>> > 218 //g_ceph_context->_conf->set_val("debug_throttle", "20"); >>> > 219 g_ceph_context->_conf->apply_changes(NULL); >>> > 220 >>> > >>> > 221 ceph_filestore_data->osd_path = >>> strdup("/mnt/fio_ceph_filestore.XXXXXXX"); >>> > 222 ceph_filestore_data->journal_path = >>> strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX"); >>> > 223 >>> > >>> > 224 if (!mkdtemp(ceph_filestore_data->osd_path)) { >>> > 225 cout << "mkdtemp failed: " << strerror(errno) << >>> std::endl; >>> > 226 return 1; >>> > 227 } >>> > 228 //mktemp(ceph_filestore_data->journal_path); // NOSPC issue >>> > 229 >>> > >>> > 230 ObjectStore *fs = new >>> FileStore(ceph_filestore_data->osd_path, >>> ceph_filestore_data->journal_path); >>> > 231 ceph_filestore_data->fs = fs; >>> > 232 >>> > >>> > 233 if (fs->mkfs() < 0) { >>> > 234 cout << "mkfs failed" << std::endl; >>> > 235 goto failed; >>> > 236 } >>> > 237 >>> > 238 if (fs->mount() < 0) { >>> > 239 cout << "mount failed" << std::endl; >>> > 240 goto failed; >>> > 241 } >>> > 242 >>> > >>> > 243 ft.create_collection(coll_t()); >>> > 244 fs->apply_transaction(ft); >>> > 245 >>> > >>> > 246 >>> > >>> > 247 return 0; >>> > 248 >>> > >>> > 249 failed: >>> > 250 return 1; >>> > 251 >>> > >>> > 252 } >>> > -----Original Message----- >>> > From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >>> > Sent: Thursday, July 09, 2015 9:19 AM >>> > To: James (Fei) Liu-SSI >>> > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>> > Subject: Re: About Fio backend with ObjectStore API >>> > >>> > Hi James, >>> > >>> > In the job file src/test/filestore.fio, you can modify the line >>> > "objectstore=filestore" to use any objectstore type supported by >>> the >>> > ObjectStore::create() factory. >>> > >>> > Casey >>> > >>> > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI >>> <james.liu@xxxxxxxxxxxxxxx> wrote: >>> >> Hi Casey, >>> >> Quick questions, The code in the trunk only cover the test for >>> filestore. I was wondering do you have any plan to cover the test for >>> kvstore and newstore? >>> >> >>> >> Thanks, >>> >> James >>> >> >>> >> -----Original Message----- >>> >> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>> >> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of James (Fei) >>> >>> >> Liu-SSI >>> >> Sent: Tuesday, June 30, 2015 2:19 PM >>> >> To: Casey Bodley >>> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>> >> Subject: RE: About Fio backend with ObjectStore API >>> >> >>> >> Hi Casey, >>> >> >>> >> Thanks a lot. >>> >> >>> >> Regards, >>> >> James >>> >> >>> >> -----Original Message----- >>> >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >>> >> Sent: Tuesday, June 30, 2015 2:16 PM >>> >> To: James (Fei) Liu-SSI >>> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >>> >> Subject: Re: About Fio backend with ObjectStore API >>> >> >>> >> Hi, >>> >> >>> >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance >>> >> Analysis: fio and RBD" at >>> >> >>> https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analy >>> >> s is_fio_rbd.html, they also mentioned a fio engine that linked >>> >> directly into ceph's FileStore. I was able to find Daniel's branch >>> on >>> >> github at https://github.com/gollub/ceph/tree/fio_filestore_v2, and >>> did some more work on it at the time. >>> >> >>> >> I just rebased that work onto the latest ceph master branch, and >>> pushed to our github at >>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore. You >>> can find the source in src/test/fio_ceph_filestore.cc, and run fio >>> with the provided example fio job file in src/test/filestore.fio. >>> >> >>> >> I didn't have a chance to confirm that it builds with automake, but >>> >>> >> the cmake version built for me. I'm happy to help if you run into >>> >> problems, Casey >>> >> >>> >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI >>> <james.liu@xxxxxxxxxxxxxxx> wrote: >>> >>> Hi Haomai, >>> >>> What are you trying to ask is to benchmark local >>> objectstore(like kvstore/filestore/newstore) locally with >>> FIO(ObjectStore engine)? You want to purely compare the performance >>> locally for these objectstores, right? >>> >>> >>> >>> Regards, >>> >>> James >>> >>> >>> >>> -----Original Message----- >>> >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx >>> >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Haomai >>> Wang >>> >>> Sent: Tuesday, June 30, 2015 9:06 AM >>> >>> To: ceph-devel@xxxxxxxxxxxxxxx >>> >>> Subject: About Fio backend with ObjectStore API >>> >>> >>> >>> Hi all, >>> >>> >>> >>> Long long ago, is there someone said about fio backend with Ceph >>> ObjectStore API? So we could use the existing mature fio facility to >>> benchmark ceph objectstore. >>> >>> >>> >>> -- >>> >>> Best Regards, >>> >>> >>> >>> Wheat >>> >>> -- >>> >>> To unsubscribe from this list: send the line "unsubscribe >>> ceph-devel" >>> >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More >>> majordomo >>> >>> info at http://vger.kernel.org/majordomo-info.html >>> >> {.n + +% lzwm b 맲 r yǩ ׯzX ܨ} Ơz &j:+v zZ+ >>> +zf h ~ i z w ? & )ߢ f >>> N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"�� >> >> -- >> Matt Benjamin >> CohortFS, LLC. >> 315 West Huron Street, Suite 140A >> Ann Arbor, Michigan 48103 >> >> http://cohortfs.com >> >> tel. 734-761-4689 >> fax. 734-769-8938 >> cel. 734-216-5309 > > > > -- > Best Regards, > > Wheat -- Best Regards, Wheat -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
