I have rebased the branch with master, and push it to ceph upstream repo. https://github.com/ceph/ceph/compare/fio-objectstore?expand=1 Plz let me know if who is working on this. Otherwise, I would like to improve this to be merge ready. On Fri, Jul 10, 2015 at 4:26 AM, Matt W. Benjamin <matt@xxxxxxxxxxxx> wrote: > That makes sense. > > Matt > > ----- "James (Fei) Liu-SSI" <james.liu@xxxxxxxxxxxxxxx> wrote: > >> Hi Casey, >> Got it. I was directed to the old code base. By the way, Since the >> testing case was used to exercise all of object stores. Strongly >> recommend to change the name from fio_ceph_filestore.cc to >> fio_ceph_objectstore.cc . And the code in fio_ceph_filestore.cc should >> be refactored to reflect that the whole objectstore will be supported >> by fio_ceph_objectstore.cc. what you think? >> >> Let me know if you need any help from my side. >> >> >> Regards, >> James >> >> >> >> -----Original Message----- >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >> Sent: Thursday, July 09, 2015 12:32 PM >> To: James (Fei) Liu-SSI >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >> Subject: Re: About Fio backend with ObjectStore API >> >> Hi James, >> >> Are you looking at the code from >> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore? It >> uses ObjectStore::create() instead of new FileStore(). This allows us >> to exercise all of the object stores with the same code. >> >> Casey >> >> On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI >> <james.liu@xxxxxxxxxxxxxxx> wrote: >> > Hi Casey, >> > Here is the code in the fio_ceph_filestore.cc. Basically, it >> creates a filestore as backend engine for IO exercises. If we got to >> send IO commands to KeyValue Store or Newstore, we got to change the >> code accordingly, right? I did not see any other files like >> fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In my humble >> opinion, we might need to create other two fio engines for >> keyvaluestore and newstore if we want to exercise these two, right? >> > >> > Regards, >> > James >> > >> > static int fio_ceph_filestore_init(struct thread_data *td) >> > 209 { >> > 210 vector<const char*> args; >> > 211 struct ceph_filestore_data *ceph_filestore_data = (struct >> ceph_filestore_data *) td->io_ops->data; >> > 212 ObjectStore::Transaction ft; >> > 213 >> > >> > 214 global_init(NULL, args, CEPH_ENTITY_TYPE_OSD, >> CODE_ENVIRONMENT_UTILITY, 0); >> > 215 //g_conf->journal_dio = false; >> > 216 common_init_finish(g_ceph_context); >> > 217 //g_ceph_context->_conf->set_val("debug_filestore", "20"); >> > 218 //g_ceph_context->_conf->set_val("debug_throttle", "20"); >> > 219 g_ceph_context->_conf->apply_changes(NULL); >> > 220 >> > >> > 221 ceph_filestore_data->osd_path = >> strdup("/mnt/fio_ceph_filestore.XXXXXXX"); >> > 222 ceph_filestore_data->journal_path = >> strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX"); >> > 223 >> > >> > 224 if (!mkdtemp(ceph_filestore_data->osd_path)) { >> > 225 cout << "mkdtemp failed: " << strerror(errno) << >> std::endl; >> > 226 return 1; >> > 227 } >> > 228 //mktemp(ceph_filestore_data->journal_path); // NOSPC issue >> > 229 >> > >> > 230 ObjectStore *fs = new >> FileStore(ceph_filestore_data->osd_path, >> ceph_filestore_data->journal_path); >> > 231 ceph_filestore_data->fs = fs; >> > 232 >> > >> > 233 if (fs->mkfs() < 0) { >> > 234 cout << "mkfs failed" << std::endl; >> > 235 goto failed; >> > 236 } >> > 237 >> > 238 if (fs->mount() < 0) { >> > 239 cout << "mount failed" << std::endl; >> > 240 goto failed; >> > 241 } >> > 242 >> > >> > 243 ft.create_collection(coll_t()); >> > 244 fs->apply_transaction(ft); >> > 245 >> > >> > 246 >> > >> > 247 return 0; >> > 248 >> > >> > 249 failed: >> > 250 return 1; >> > 251 >> > >> > 252 } >> > -----Original Message----- >> > From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >> > Sent: Thursday, July 09, 2015 9:19 AM >> > To: James (Fei) Liu-SSI >> > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >> > Subject: Re: About Fio backend with ObjectStore API >> > >> > Hi James, >> > >> > In the job file src/test/filestore.fio, you can modify the line >> > "objectstore=filestore" to use any objectstore type supported by >> the >> > ObjectStore::create() factory. >> > >> > Casey >> > >> > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI >> <james.liu@xxxxxxxxxxxxxxx> wrote: >> >> Hi Casey, >> >> Quick questions, The code in the trunk only cover the test for >> filestore. I was wondering do you have any plan to cover the test for >> kvstore and newstore? >> >> >> >> Thanks, >> >> James >> >> >> >> -----Original Message----- >> >> From: ceph-devel-owner@xxxxxxxxxxxxxxx >> >> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of James (Fei) >> >> >> Liu-SSI >> >> Sent: Tuesday, June 30, 2015 2:19 PM >> >> To: Casey Bodley >> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >> >> Subject: RE: About Fio backend with ObjectStore API >> >> >> >> Hi Casey, >> >> >> >> Thanks a lot. >> >> >> >> Regards, >> >> James >> >> >> >> -----Original Message----- >> >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] >> >> Sent: Tuesday, June 30, 2015 2:16 PM >> >> To: James (Fei) Liu-SSI >> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx >> >> Subject: Re: About Fio backend with ObjectStore API >> >> >> >> Hi, >> >> >> >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance >> >> Analysis: fio and RBD" at >> >> >> https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analy >> >> s is_fio_rbd.html, they also mentioned a fio engine that linked >> >> directly into ceph's FileStore. I was able to find Daniel's branch >> on >> >> github at https://github.com/gollub/ceph/tree/fio_filestore_v2, and >> did some more work on it at the time. >> >> >> >> I just rebased that work onto the latest ceph master branch, and >> pushed to our github at >> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore. You >> can find the source in src/test/fio_ceph_filestore.cc, and run fio >> with the provided example fio job file in src/test/filestore.fio. >> >> >> >> I didn't have a chance to confirm that it builds with automake, but >> >> >> the cmake version built for me. I'm happy to help if you run into >> >> problems, Casey >> >> >> >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI >> <james.liu@xxxxxxxxxxxxxxx> wrote: >> >>> Hi Haomai, >> >>> What are you trying to ask is to benchmark local >> objectstore(like kvstore/filestore/newstore) locally with >> FIO(ObjectStore engine)? You want to purely compare the performance >> locally for these objectstores, right? >> >>> >> >>> Regards, >> >>> James >> >>> >> >>> -----Original Message----- >> >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx >> >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Haomai >> Wang >> >>> Sent: Tuesday, June 30, 2015 9:06 AM >> >>> To: ceph-devel@xxxxxxxxxxxxxxx >> >>> Subject: About Fio backend with ObjectStore API >> >>> >> >>> Hi all, >> >>> >> >>> Long long ago, is there someone said about fio backend with Ceph >> ObjectStore API? So we could use the existing mature fio facility to >> benchmark ceph objectstore. >> >>> >> >>> -- >> >>> Best Regards, >> >>> >> >>> Wheat >> >>> -- >> >>> To unsubscribe from this list: send the line "unsubscribe >> ceph-devel" >> >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More >> majordomo >> >>> info at http://vger.kernel.org/majordomo-info.html >> >> {.n + +% lzwm b 맲 r yǩ ׯzX ܨ} Ơz &j:+v zZ+ >> +zf h ~ i z w ? & )ߢ f >> N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"�� > > -- > Matt Benjamin > CohortFS, LLC. > 315 West Huron Street, Suite 140A > Ann Arbor, Michigan 48103 > > http://cohortfs.com > > tel. 734-761-4689 > fax. 734-769-8938 > cel. 734-216-5309 -- Best Regards, Wheat -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
