> from the responses I've gotten, it looks like there's no viable option to use
> fibre channel as an interconnect between the nodes of the cluster.
> Would it be worth while development effort to establish a block protocol
> between the nodes so that something like fibre channel could be used to
> communicate internally? Unless I'm waaay wrong (And I'm seldom *that*
> wrong), it would not be worth the effort. I won't even feature request it.
> Looks like I'll have to look into infiniband or CE, and possibly migrate away
> from Fibre Channel, even though it kinda just works, and therefore I really
> like it :(
I would think even conceptually it would be a mess - FC as a peer to peer network fabric might be useful (in many ways I like it a lot better than Ethernet), but you would have to develop an entire transport protocol over it (the normal SCSI model would be useless) for Ceph and then write that in to replace any of the network code in the existing Ceph code base.
A lot of work for something that is probably easier done swapping your FC HBAs for 10G NICs or IB HBAs.
>
> On Thu, Apr 21, 2016 at 11:06 PM, Schlacta, Christ <aarcane@xxxxxxxxxxx>
> wrote:
> > My primary motivations are:
> > Most of my systems that I want to use with ceph already have fibre
> > Chantel cards and infrastructure, and more infrastructure is
> > incredibly cheap compared to infiniband or {1,4}0gbe cards and
> > infrastructure Most of my systems are expansion slot constrained, and
> > I'd be forced to pick one or the other anyway.
> >
> > On Thu, Apr 21, 2016 at 9:28 PM, Paul Evans <pe@xxxxxxxxxxxx> wrote:
> >> In today’s world, OSDs communicate via IP and only IP*. Some
> >> FiberChannel switches and HBAs support IP-over-FC, but it’s about
> >> 0.000002% of the FC deployments.
> >> Therefore, one could technically use FC, but it does’t appear to
> >> offer enough benefit to OSD operations to justify the unique architecture.
> >>
> >> What is your motivation to leverage FC behind OSDs?
> >>
> >> -Paul
> >>
> >> *Ceph on native Infiniband may be available some day, but it seems
> >> impractical with the current releases. IP-over-IB is also known to work.
> >>
> >>
> >> On Apr 21, 2016, at 8:12 PM, Schlacta, Christ <aarcane@xxxxxxxxxxx>
> wrote:
> >>
> >> Is it possible? Can I use fibre channel to interconnect my ceph OSDs?
> >> Intuition tells me it should be possible, yet experience (Mostly with
> >> fibre channel) tells me no. I don't know enough about how ceph works
> >> to know for sure. All my googling returns results about using ceph
> >> as a BACKEND for exporting fibre channel LUNs, which is, sadly, not
> >> what I'm looking for at the moment.
> >>
> >>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Confidentiality: This email and any attachments are confidential and may be subject to copyright, legal or some other professional privilege. They are intended solely for the attention and use of the named addressee(s). They may only be copied, distributed or disclosed with the consent of the copyright owner. If you have received this email by mistake or by breach of the confidentiality clause, please notify the sender immediately by return email and delete or destroy all copies of the email. Any confidentiality, privilege or copyright is not waived or lost because this email has been sent to you by mistake.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
