I just re-read the documentation… It looks like its a proposed feature that is in development. I will have to adjust my test in consequence in that case. Any one out there have any ideas when this will be implemented? Or what the plans look like as of right now? On 2014-03-15, 1:17 PM, "Karol Kozubal" <Karol.Kozubal@xxxxxxxxx> wrote: >How are the SSDs going to be in writeback? Is that the new caching pool >Feature? > >I am not sure what version implemented this, but it is documented here >(https://ceph.com/docs/master/dev/cache-pool/). >I will be using the latest stable release for my next batch of testing, >right now I am on 0.67.4 and I will be moving towards the 0.72.x branch. > >As for the IOPS, it would be a total cluster IO throughput estimate based >on an application that would be reading/writing to more than 60 rbd >volumes. > > > > > >On 2014-03-15, 1:11 PM, "Wido den Hollander" <wido@xxxxxxxx> wrote: > >>On 03/15/2014 05:40 PM, Karol Kozubal wrote: >>> Hi Wido, >>> >>> I will have some new hardware for running tests in the next two weeks >>>or >>> so and will report my findings once I get a chance to run some tests. I >>> will disable writeback on the target side as I will be attempting to >>> configure an ssd caching pool of 24 ssd's with writeback for the main >>>pool >>> with 360 disks with a 5 osd spinners to 1 ssd journal ratio. I will be >> >>How are the SSDs going to be in writeback? Is that the new caching pool >>feature? >> >>> running everything through 10Gig SFP+ Ethernet interfaces with a >>>dedicated >>> cluster network interface, dedicated public ceph interface and a >>>separate >>> iscsi network also with 10 gig interfaces for the target machines. >>> >> >>That seems like a good network. >> >>> I am ideally looking for a 20,000 to 60,000 IOPS from this system if I >>>can >>> get the caching pool configuration right. The application has a 30ms >>>max >>> latency requirement for the storage. >>> >> >>20.000 to 60.000 is a big difference. But the only way you are going to >>achieve that is by doing a lot of parellel I/O. Ceph doesn't excel in >>single threads doing a lot of I/O. >> >>So if you have multiple RBD devices on which you are doing the I/O it >>shouldn't be that much of a problem. >> >>Just spread out the I/O. Scale horizontal instead of vertical. >> >>> In my current tests I have only spinners with SAS 10K disks, 4.2ms >>>write >>> latency on the disks with separate journaling on SAS 15K disks with a >>> 3.3ms write latency. With 20 OSDs and 4 Journals I am only concerned >>>with >>> the overall operation apply latency that I have been seeing (1-6ms idle >>>is >>> normal, but up to 60-170ms for a moderate workload using rbd >>>bench-write) >>> however I am on a network where I am bound to 1500 mtu and I will get >>>to >>> test jumbo frames with the next setup in addition to the ssd¹s. I >>>suspect >>> the overall performance will be good in the new test setup and I am >>> curious to see what my tests will yield. >>> >>> Thanks for the response! >>> >>> Karol >>> >>> >>> >>> On 2014-03-15, 12:18 PM, "Wido den Hollander" <wido@xxxxxxxx> wrote: >>> >>>> On 03/15/2014 04:11 PM, Karol Kozubal wrote: >>>>> Hi Everyone, >>>>> >>>>> I am just wondering if any of you are running a ceph cluster with an >>>>> iSCSI target front end? I know this isn¹t available out of the box, >>>>> unfortunately in one particular use case we are looking at providing >>>>> iSCSI access and it's a necessity. I am liking the idea of having rbd >>>>> devices serving block level storage to the iSCSI Target servers while >>>>> providing a unified backed for native rbd access by openstack and >>>>> various application servers. On multiple levels this would reduce the >>>>> complexity of our SAN environment and move us away from expensive >>>>> proprietary solutions that don¹t scale out. >>>>> >>>>> If any of you have deployed any HA iSCSI Targets backed by rbd I >>>>>would >>>>> really appreciate your feedback and any thoughts. >>>>> >>>> >>>> I haven't used it in production, but a couple of things which come to >>>> mind: >>>> >>>> - Use TGT so you can run it all in userspace backed by librbd >>>> - Do not use writeback caching on the targets >>>> >>>> You could use multipathing if you don't use writeback caching. Use >>>> writeback would also cause data loss/corruption in case of multiple >>>> targets. >>>> >>>> It will probably just work with TGT, but I don't know anything about >>>>the >>>> performance. >>>> >>>>> Karol >>>>> >>>>> >>>>> _______________________________________________ >>>>> ceph-users mailing list >>>>> ceph-users@xxxxxxxxxxxxxx >>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>> >>>> >>>> >>>> -- >>>> Wido den Hollander >>>> 42on B.V. >>>> >>>> Phone: +31 (0)20 700 9902 >>>> Skype: contact42on >>>> _______________________________________________ >>>> ceph-users mailing list >>>> ceph-users@xxxxxxxxxxxxxx >>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >> >> >>-- >>Wido den Hollander >>42on B.V. >> >>Phone: +31 (0)20 700 9902 >>Skype: contact42on >>_______________________________________________ >>ceph-users mailing list >>ceph-users@xxxxxxxxxxxxxx >>http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >_______________________________________________ >ceph-users mailing list >ceph-users@xxxxxxxxxxxxxx >http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |