That makes sense. Matt ----- "James (Fei) Liu-SSI" <james.liu@xxxxxxxxxxxxxxx> wrote: > Hi Casey, > Got it. I was directed to the old code base. By the way, Since the > testing case was used to exercise all of object stores. Strongly > recommend to change the name from fio_ceph_filestore.cc to > fio_ceph_objectstore.cc . And the code in fio_ceph_filestore.cc should > be refactored to reflect that the whole objectstore will be supported > by fio_ceph_objectstore.cc. what you think? > > Let me know if you need any help from my side. > > > Regards, > James > > > > -----Original Message----- > From: Casey Bodley [mailto:cbodley@xxxxxxxxx] > Sent: Thursday, July 09, 2015 12:32 PM > To: James (Fei) Liu-SSI > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx > Subject: Re: About Fio backend with ObjectStore API > > Hi James, > > Are you looking at the code from > https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore? It > uses ObjectStore::create() instead of new FileStore(). This allows us > to exercise all of the object stores with the same code. > > Casey > > On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI > <james.liu@xxxxxxxxxxxxxxx> wrote: > > Hi Casey, > > Here is the code in the fio_ceph_filestore.cc. Basically, it > creates a filestore as backend engine for IO exercises. If we got to > send IO commands to KeyValue Store or Newstore, we got to change the > code accordingly, right? I did not see any other files like > fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In my humble > opinion, we might need to create other two fio engines for > keyvaluestore and newstore if we want to exercise these two, right? > > > > Regards, > > James > > > > static int fio_ceph_filestore_init(struct thread_data *td) > > 209 { > > 210 vector<const char*> args; > > 211 struct ceph_filestore_data *ceph_filestore_data = (struct > ceph_filestore_data *) td->io_ops->data; > > 212 ObjectStore::Transaction ft; > > 213 > > > > 214 global_init(NULL, args, CEPH_ENTITY_TYPE_OSD, > CODE_ENVIRONMENT_UTILITY, 0); > > 215 //g_conf->journal_dio = false; > > 216 common_init_finish(g_ceph_context); > > 217 //g_ceph_context->_conf->set_val("debug_filestore", "20"); > > 218 //g_ceph_context->_conf->set_val("debug_throttle", "20"); > > 219 g_ceph_context->_conf->apply_changes(NULL); > > 220 > > > > 221 ceph_filestore_data->osd_path = > strdup("/mnt/fio_ceph_filestore.XXXXXXX"); > > 222 ceph_filestore_data->journal_path = > strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX"); > > 223 > > > > 224 if (!mkdtemp(ceph_filestore_data->osd_path)) { > > 225 cout << "mkdtemp failed: " << strerror(errno) << > std::endl; > > 226 return 1; > > 227 } > > 228 //mktemp(ceph_filestore_data->journal_path); // NOSPC issue > > 229 > > > > 230 ObjectStore *fs = new > FileStore(ceph_filestore_data->osd_path, > ceph_filestore_data->journal_path); > > 231 ceph_filestore_data->fs = fs; > > 232 > > > > 233 if (fs->mkfs() < 0) { > > 234 cout << "mkfs failed" << std::endl; > > 235 goto failed; > > 236 } > > 237 > > 238 if (fs->mount() < 0) { > > 239 cout << "mount failed" << std::endl; > > 240 goto failed; > > 241 } > > 242 > > > > 243 ft.create_collection(coll_t()); > > 244 fs->apply_transaction(ft); > > 245 > > > > 246 > > > > 247 return 0; > > 248 > > > > 249 failed: > > 250 return 1; > > 251 > > > > 252 } > > -----Original Message----- > > From: Casey Bodley [mailto:cbodley@xxxxxxxxx] > > Sent: Thursday, July 09, 2015 9:19 AM > > To: James (Fei) Liu-SSI > > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx > > Subject: Re: About Fio backend with ObjectStore API > > > > Hi James, > > > > In the job file src/test/filestore.fio, you can modify the line > > "objectstore=filestore" to use any objectstore type supported by > the > > ObjectStore::create() factory. > > > > Casey > > > > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI > <james.liu@xxxxxxxxxxxxxxx> wrote: > >> Hi Casey, > >> Quick questions, The code in the trunk only cover the test for > filestore. I was wondering do you have any plan to cover the test for > kvstore and newstore? > >> > >> Thanks, > >> James > >> > >> -----Original Message----- > >> From: ceph-devel-owner@xxxxxxxxxxxxxxx > >> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of James (Fei) > > >> Liu-SSI > >> Sent: Tuesday, June 30, 2015 2:19 PM > >> To: Casey Bodley > >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx > >> Subject: RE: About Fio backend with ObjectStore API > >> > >> Hi Casey, > >> > >> Thanks a lot. > >> > >> Regards, > >> James > >> > >> -----Original Message----- > >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx] > >> Sent: Tuesday, June 30, 2015 2:16 PM > >> To: James (Fei) Liu-SSI > >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx > >> Subject: Re: About Fio backend with ObjectStore API > >> > >> Hi, > >> > >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance > >> Analysis: fio and RBD" at > >> > https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-analy > >> s is_fio_rbd.html, they also mentioned a fio engine that linked > >> directly into ceph's FileStore. I was able to find Daniel's branch > on > >> github at https://github.com/gollub/ceph/tree/fio_filestore_v2, and > did some more work on it at the time. > >> > >> I just rebased that work onto the latest ceph master branch, and > pushed to our github at > https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore. You > can find the source in src/test/fio_ceph_filestore.cc, and run fio > with the provided example fio job file in src/test/filestore.fio. > >> > >> I didn't have a chance to confirm that it builds with automake, but > > >> the cmake version built for me. I'm happy to help if you run into > >> problems, Casey > >> > >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI > <james.liu@xxxxxxxxxxxxxxx> wrote: > >>> Hi Haomai, > >>> What are you trying to ask is to benchmark local > objectstore(like kvstore/filestore/newstore) locally with > FIO(ObjectStore engine)? You want to purely compare the performance > locally for these objectstores, right? > >>> > >>> Regards, > >>> James > >>> > >>> -----Original Message----- > >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx > >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Haomai > Wang > >>> Sent: Tuesday, June 30, 2015 9:06 AM > >>> To: ceph-devel@xxxxxxxxxxxxxxx > >>> Subject: About Fio backend with ObjectStore API > >>> > >>> Hi all, > >>> > >>> Long long ago, is there someone said about fio backend with Ceph > ObjectStore API? So we could use the existing mature fio facility to > benchmark ceph objectstore. > >>> > >>> -- > >>> Best Regards, > >>> > >>> Wheat > >>> -- > >>> To unsubscribe from this list: send the line "unsubscribe > ceph-devel" > >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More > majordomo > >>> info at http://vger.kernel.org/majordomo-info.html > >> {.n + +% lzwm b 맲 r yǩ ׯzX ܨ} Ơz &j:+v zZ+ > +zf h ~ i z w ? & )ߢ f > N�����r��y���b�X��ǧv�^�){.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"�� -- Matt Benjamin CohortFS, LLC. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://cohortfs.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
