Re: About Fio backend with ObjectStore API

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi James,

I'm sorry for not following up on that segfault, but I wasn't ever able to reproduce it. I used it recently for memstore testing without any problems. I wonder if there's a problem with the autotools build? I've only tested it with cmake. When I find some time, I'll rebase it on master and do another round of testing.

Casey

----- Original Message -----
> From: "James (Fei) Liu-SSI" <james.liu@xxxxxxxxxxxxxxx>
> To: "Haomai Wang" <haomaiwang@xxxxxxxxx>, "Casey Bodley" <cbodley@xxxxxxxxxx>
> Cc: "Casey Bodley" <cbodley@xxxxxxxxx>, "Matt W. Benjamin" <matt@xxxxxxxxxxxx>, ceph-devel@xxxxxxxxxxxxxxx
> Sent: Wednesday, September 2, 2015 8:06:14 PM
> Subject: RE: About Fio backend with ObjectStore API
> 
> Hi Haomai and Case,
>      Do you have any fixes for that segfault?
> 
>     Thanks,
>     James
> 
> -----Original Message-----
> From: Haomai Wang [mailto:haomaiwang@xxxxxxxxx]
> Sent: Wednesday, July 22, 2015 6:07 PM
> To: Casey Bodley
> Cc: Casey Bodley; Matt W. Benjamin; James (Fei) Liu-SSI;
> ceph-devel@xxxxxxxxxxxxxxx
> Subject: Re: About Fio backend with ObjectStore API
> 
> no special
> 
> [global]
> #logging
> #write_iops_log=write_iops_log
> #write_bw_log=write_bw_log
> #write_lat_log=write_lat_log
> ioengine=./ceph-int/src/.libs/libfio_ceph_objectstore.so
> invalidate=0 # mandatory
> rw=write
> #bs=4k
> 
> [filestore]
> iodepth=1
> # create a journaled filestore
> objectstore=filestore
> directory=./osd/
> filestore_journal=./osd/journal
> 
> On Thu, Jul 23, 2015 at 4:56 AM, Casey Bodley <cbodley@xxxxxxxxxx> wrote:
> > Hi Haomai,
> >
> > Sorry for the late response, I was out of the office. I'm afraid I haven't
> > run into that segfault. The io_ops should be set at the very beginning
> > when it calls get_ioengine(). All I can suggest is that you verify that
> > your job file is pointing to the correct fio_ceph_objectstore.so. If
> > you've made any other interesting changes to the job file, could you share
> > it here?
> >
> > Casey
> >
> > ----- Original Message -----
> > From: "Haomai Wang" <haomaiwang@xxxxxxxxx>
> > To: "Casey Bodley" <cbodley@xxxxxxxxx>
> > Cc: "Matt W. Benjamin" <matt@xxxxxxxxxxxx>, "James (Fei) Liu-SSI"
> > <james.liu@xxxxxxxxxxxxxxx>, ceph-devel@xxxxxxxxxxxxxxx
> > Sent: Tuesday, July 21, 2015 7:50:32 AM
> > Subject: Re: About Fio backend with ObjectStore API
> >
> > Hi Casey,
> >
> > I check your commits and know what you fixed. I cherry-picked your new
> > commits but I still met the same problem.
> >
> > """
> > It's strange that it alwasys hit segment fault when entering
> > "_fio_setup_ceph_filestore_data", gdb tells "td->io_ops" is NULL but
> > when I up the stack, the "td->io_ops" is not null. Maybe it's related
> > to dlopen?
> > """
> >
> > Do you have any hint about this?
> >
> > On Thu, Jul 16, 2015 at 5:23 AM, Casey Bodley <cbodley@xxxxxxxxx> wrote:
> >> Hi Haomai,
> >>
> >> I was able to run this after a couple changes to the filestore.fio
> >> job file. Two of the config options were using the wrong names. I
> >> pushed a fix for the job file, as well as a patch that renames
> >> everything from filestore to objectstore (thanks James), to
> >> https://github.com/linuxbox2/linuxbox-ceph/commits/fio-objectstore.
> >>
> >> I found that the read support doesn't appear to work anymore, so give
> >> "rw=write" a try. And because it does a mkfs(), make sure you're
> >> pointing it to an empty xfs directory with the "directory=" option.
> >>
> >> Casey
> >>
> >> On Tue, Jul 14, 2015 at 2:45 AM, Haomai Wang <haomaiwang@xxxxxxxxx> wrote:
> >>> Anyone who have successfully ran the fio with this external io
> >>> engine ceph_objectstore?
> >>>
> >>> It's strange that it alwasys hit segment fault when entering
> >>> "_fio_setup_ceph_filestore_data", gdb tells "td->io_ops" is NULL but
> >>> when I up the stack, the "td->io_ops" is not null. Maybe it's
> >>> related to dlopen?
> >>>
> >>> On Fri, Jul 10, 2015 at 3:51 PM, Haomai Wang <haomaiwang@xxxxxxxxx>
> >>> wrote:
> >>>> I have rebased the branch with master, and push it to ceph upstream
> >>>> repo. https://github.com/ceph/ceph/compare/fio-objectstore?expand=1
> >>>>
> >>>> Plz let me know if who is working on this. Otherwise, I would like
> >>>> to improve this to be merge ready.
> >>>>
> >>>> On Fri, Jul 10, 2015 at 4:26 AM, Matt W. Benjamin <matt@xxxxxxxxxxxx>
> >>>> wrote:
> >>>>> That makes sense.
> >>>>>
> >>>>> Matt
> >>>>>
> >>>>> ----- "James (Fei) Liu-SSI" <james.liu@xxxxxxxxxxxxxxx> wrote:
> >>>>>
> >>>>>> Hi Casey,
> >>>>>>   Got it. I was directed to the old code base. By the way, Since
> >>>>>> the testing case was used to exercise all of object stores.
> >>>>>> Strongly recommend to change the name from fio_ceph_filestore.cc
> >>>>>> to fio_ceph_objectstore.cc . And the code in
> >>>>>> fio_ceph_filestore.cc should be refactored to reflect that the
> >>>>>> whole objectstore will be supported by fio_ceph_objectstore.cc. what
> >>>>>> you think?
> >>>>>>
> >>>>>> Let me know if you need any help from my side.
> >>>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>> James
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Casey Bodley [mailto:cbodley@xxxxxxxxx]
> >>>>>> Sent: Thursday, July 09, 2015 12:32 PM
> >>>>>> To: James (Fei) Liu-SSI
> >>>>>> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx
> >>>>>> Subject: Re: About Fio backend with ObjectStore API
> >>>>>>
> >>>>>> Hi James,
> >>>>>>
> >>>>>> Are you looking at the code from
> >>>>>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore?
> >>>>>> It uses ObjectStore::create() instead of new FileStore(). This
> >>>>>> allows us to exercise all of the object stores with the same code.
> >>>>>>
> >>>>>> Casey
> >>>>>>
> >>>>>> On Thu, Jul 9, 2015 at 2:01 PM, James (Fei) Liu-SSI
> >>>>>> <james.liu@xxxxxxxxxxxxxxx> wrote:
> >>>>>> > Hi Casey,
> >>>>>> >   Here is the code in the fio_ceph_filestore.cc. Basically, it
> >>>>>> creates a filestore as backend engine for IO exercises. If we got
> >>>>>> to send IO commands to KeyValue Store or Newstore, we got to
> >>>>>> change the code accordingly, right?  I did not see any other
> >>>>>> files like fio_ceph_keyvaluestore.cc or fio_ceph_newstore.cc. In
> >>>>>> my humble opinion, we might need to create other two fio engines
> >>>>>> for keyvaluestore and newstore if we want to exercise these two,
> >>>>>> right?
> >>>>>> >
> >>>>>> > Regards,
> >>>>>> > James
> >>>>>> >
> >>>>>> > static int fio_ceph_filestore_init(struct thread_data *td)
> >>>>>> > 209 {
> >>>>>> > 210     vector<const char*> args;
> >>>>>> > 211     struct ceph_filestore_data *ceph_filestore_data = (struct
> >>>>>> ceph_filestore_data *) td->io_ops->data;
> >>>>>> > 212     ObjectStore::Transaction ft;
> >>>>>> > 213
> >>>>>> >
> >>>>>> > 214     global_init(NULL, args, CEPH_ENTITY_TYPE_OSD,
> >>>>>> CODE_ENVIRONMENT_UTILITY, 0);
> >>>>>> > 215     //g_conf->journal_dio = false;
> >>>>>> > 216     common_init_finish(g_ceph_context);
> >>>>>> > 217     //g_ceph_context->_conf->set_val("debug_filestore", "20");
> >>>>>> > 218     //g_ceph_context->_conf->set_val("debug_throttle", "20");
> >>>>>> > 219     g_ceph_context->_conf->apply_changes(NULL);
> >>>>>> > 220
> >>>>>> >
> >>>>>> > 221     ceph_filestore_data->osd_path =
> >>>>>> strdup("/mnt/fio_ceph_filestore.XXXXXXX");
> >>>>>> > 222     ceph_filestore_data->journal_path =
> >>>>>> strdup("/var/lib/ceph/osd/journal-ram/fio_ceph_filestore.XXXXXXX"
> >>>>>> );
> >>>>>> > 223
> >>>>>> >
> >>>>>> > 224     if (!mkdtemp(ceph_filestore_data->osd_path)) {
> >>>>>> > 225             cout << "mkdtemp failed: " << strerror(errno) <<
> >>>>>> std::endl;
> >>>>>> > 226             return 1;
> >>>>>> > 227     }
> >>>>>> > 228     //mktemp(ceph_filestore_data->journal_path); // NOSPC issue
> >>>>>> > 229
> >>>>>> >
> >>>>>> > 230     ObjectStore *fs = new
> >>>>>> FileStore(ceph_filestore_data->osd_path,
> >>>>>> ceph_filestore_data->journal_path);
> >>>>>> > 231     ceph_filestore_data->fs = fs;
> >>>>>> > 232
> >>>>>> >
> >>>>>> > 233     if (fs->mkfs() < 0) {
> >>>>>> > 234             cout << "mkfs failed" << std::endl;
> >>>>>> > 235             goto failed;
> >>>>>> > 236     }
> >>>>>> > 237
> >>>>>> > 238     if (fs->mount() < 0) {
> >>>>>> > 239             cout << "mount failed" << std::endl;
> >>>>>> > 240             goto failed;
> >>>>>> > 241     }
> >>>>>> > 242
> >>>>>> >
> >>>>>> > 243     ft.create_collection(coll_t());
> >>>>>> > 244     fs->apply_transaction(ft);
> >>>>>> > 245
> >>>>>> >
> >>>>>> > 246
> >>>>>> >
> >>>>>> > 247     return 0;
> >>>>>> > 248
> >>>>>> >
> >>>>>> > 249 failed:
> >>>>>> > 250     return 1;
> >>>>>> > 251
> >>>>>> >
> >>>>>> > 252 }
> >>>>>> > -----Original Message-----
> >>>>>> > From: Casey Bodley [mailto:cbodley@xxxxxxxxx]
> >>>>>> > Sent: Thursday, July 09, 2015 9:19 AM
> >>>>>> > To: James (Fei) Liu-SSI
> >>>>>> > Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx
> >>>>>> > Subject: Re: About Fio backend with ObjectStore API
> >>>>>> >
> >>>>>> > Hi James,
> >>>>>> >
> >>>>>> > In the job file src/test/filestore.fio, you can modify the line
> >>>>>> > "objectstore=filestore" to use any objectstore type supported
> >>>>>> > by
> >>>>>> the
> >>>>>> > ObjectStore::create() factory.
> >>>>>> >
> >>>>>> > Casey
> >>>>>> >
> >>>>>> > On Wed, Jul 8, 2015 at 8:02 PM, James (Fei) Liu-SSI
> >>>>>> <james.liu@xxxxxxxxxxxxxxx> wrote:
> >>>>>> >> Hi Casey,
> >>>>>> >>   Quick questions, The code in the trunk only cover the test
> >>>>>> >> for
> >>>>>> filestore. I was wondering do you have any plan to cover the test
> >>>>>> for kvstore and newstore?
> >>>>>> >>
> >>>>>> >>   Thanks,
> >>>>>> >>   James
> >>>>>> >>
> >>>>>> >> -----Original Message-----
> >>>>>> >> From: ceph-devel-owner@xxxxxxxxxxxxxxx
> >>>>>> >> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of James
> >>>>>> >> (Fei)
> >>>>>>
> >>>>>> >> Liu-SSI
> >>>>>> >> Sent: Tuesday, June 30, 2015 2:19 PM
> >>>>>> >> To: Casey Bodley
> >>>>>> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx
> >>>>>> >> Subject: RE: About Fio backend with ObjectStore API
> >>>>>> >>
> >>>>>> >> Hi Casey,
> >>>>>> >>
> >>>>>> >>   Thanks a lot.
> >>>>>> >>
> >>>>>> >>   Regards,
> >>>>>> >>   James
> >>>>>> >>
> >>>>>> >> -----Original Message-----
> >>>>>> >> From: Casey Bodley [mailto:cbodley@xxxxxxxxx]
> >>>>>> >> Sent: Tuesday, June 30, 2015 2:16 PM
> >>>>>> >> To: James (Fei) Liu-SSI
> >>>>>> >> Cc: Haomai Wang; ceph-devel@xxxxxxxxxxxxxxx
> >>>>>> >> Subject: Re: About Fio backend with ObjectStore API
> >>>>>> >>
> >>>>>> >> Hi,
> >>>>>> >>
> >>>>>> >> When Danny Al-Gaaf & Daniel Gollub published "Ceph Performance
> >>>>>> >> Analysis: fio and RBD" at
> >>>>>> >>
> >>>>>> https://telekomcloud.github.io/ceph/2014/02/26/ceph-performance-a
> >>>>>> naly
> >>>>>> >> s is_fio_rbd.html, they also mentioned a fio engine that
> >>>>>> >> linked directly into ceph's FileStore. I was able to find
> >>>>>> >> Daniel's branch
> >>>>>> on
> >>>>>> >> github at
> >>>>>> >> https://github.com/gollub/ceph/tree/fio_filestore_v2, and
> >>>>>> did some more work on it at the time.
> >>>>>> >>
> >>>>>> >> I just rebased that work onto the latest ceph master branch,
> >>>>>> >> and
> >>>>>> pushed to our github at
> >>>>>> https://github.com/linuxbox2/linuxbox-ceph/tree/fio-objectstore.
> >>>>>> You can find the source in src/test/fio_ceph_filestore.cc, and
> >>>>>> run fio with the provided example fio job file in
> >>>>>> src/test/filestore.fio.
> >>>>>> >>
> >>>>>> >> I didn't have a chance to confirm that it builds with
> >>>>>> >> automake, but
> >>>>>>
> >>>>>> >> the cmake version built for me. I'm happy to help if you run
> >>>>>> >> into problems, Casey
> >>>>>> >>
> >>>>>> >> On Tue, Jun 30, 2015 at 2:31 PM, James (Fei) Liu-SSI
> >>>>>> <james.liu@xxxxxxxxxxxxxxx> wrote:
> >>>>>> >>> Hi Haomai,
> >>>>>> >>>   What are you trying to ask is to benchmark local
> >>>>>> objectstore(like kvstore/filestore/newstore) locally with
> >>>>>> FIO(ObjectStore engine)? You want to purely compare the
> >>>>>> performance locally for these objectstores, right?
> >>>>>> >>>
> >>>>>> >>>   Regards,
> >>>>>> >>>   James
> >>>>>> >>>
> >>>>>> >>> -----Original Message-----
> >>>>>> >>> From: ceph-devel-owner@xxxxxxxxxxxxxxx
> >>>>>> >>> [mailto:ceph-devel-owner@xxxxxxxxxxxxxxx] On Behalf Of Haomai
> >>>>>> Wang
> >>>>>> >>> Sent: Tuesday, June 30, 2015 9:06 AM
> >>>>>> >>> To: ceph-devel@xxxxxxxxxxxxxxx
> >>>>>> >>> Subject: About Fio backend with ObjectStore API
> >>>>>> >>>
> >>>>>> >>> Hi all,
> >>>>>> >>>
> >>>>>> >>> Long long ago, is there someone said about fio backend with
> >>>>>> >>> Ceph
> >>>>>> ObjectStore API? So we could use the existing mature fio facility
> >>>>>> to benchmark ceph objectstore.
> >>>>>> >>>
> >>>>>> >>> --
> >>>>>> >>> Best Regards,
> >>>>>> >>>
> >>>>>> >>> Wheat
> >>>>>> >>> --
> >>>>>> >>> To unsubscribe from this list: send the line "unsubscribe
> >>>>>> ceph-devel"
> >>>>>> >>> in the body of a message to majordomo@xxxxxxxxxxxxxxx More
> >>>>>> majordomo
> >>>>>> >>> info at  http://vger.kernel.org/majordomo-info.html
> >>>>>> >>   {.n +       +%  lzwm  b 맲  r  yǩ ׯzX     ܨ}   Ơz &j:+v        zZ+
> >>>>>>  +zf   h   ~    i   z   w   ?    & )ߢ f
> >>>>>> N     r  y   b X  ǧv ^ )޺{.n +   z ]z   {ay ʇڙ ,j  f   h   z  w
> >>>>>>     j:+v   w j m        zZ+  ݢj"
> >>>>>
> >>>>> --
> >>>>> Matt Benjamin
> >>>>> CohortFS, LLC.
> >>>>> 315 West Huron Street, Suite 140A
> >>>>> Ann Arbor, Michigan 48103
> >>>>>
> >>>>> http://cohortfs.com
> >>>>>
> >>>>> tel.  734-761-4689
> >>>>> fax.  734-769-8938
> >>>>> cel.  734-216-5309
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>>
> >>>> Wheat
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards,
> >>>
> >>> Wheat
> >
> >
> >
> > --
> > Best Regards,
> >
> > Wheat
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> > in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo
> > info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> 
> --
> Best Regards,
> 
> Wheat
> N�����r��y���b�X��ǧv�^�)޺{.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w������j:+v���w�j�m��������zZ+��ݢj"��
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html



[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux