Burkhard, I woke up with the same conclusion - LACP load balancing can break down when the traffic traverses a router since the IP headers have the router as the destination address and thus the Ethernet header has the same to MAC addresses. (I think that in a pure layer 2 fabric the MAC addresses vary enough to produce reasonable - not perfect - LACP load balancing.) Then add VXLAN, SDN, and other newer networking technologies and it all gets even more confusing. But I always come back to the starter cluster, likely a proof of concept demonstration, that might be built with left-over parts. Networking is frequently an afterthought. In this case node-level traffic management - weighted fair queueing - could make all the difference. -Dave -- Dave Hall Binghamton University kdhall@xxxxxxxxxxxxxx On Tue, Mar 16, 2021 at 4:20 AM Burkhard Linke < Burkhard.Linke@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> wrote: > Hi, > > On 16.03.21 03:40, Dave Hall wrote: > > Andrew, > > > > I agree that the choice of hash function is important for LACP. My > > thinking has always been to stay down in layers 2 and 3. With enough > > hosts it seems likely that traffic would be split close to evenly. > > Heads or tails - 50% of the time you're right. TCP ports should also > > be nearly equally split, but listening ports could introduce some > > asymmetry. > > > Just a comment on the hashing methods. LACP specs does not include > layer3+4, so running it is somewhat outside of the spec. > > The main reason for it being present it the fact that LACP load > balancing does not work well in case of routing. If all your clients are > in a different network reachable via a gateway, all your traffic will be > directed to the MAC address of the gateway. As a result all that traffic > will use a single link only. > > Also keep in mind that these hashing methods only affect the traffic the > originate from the corresponding system. In case of a ceph host only the > traffic sent from the host is controlled by it; the traffic from the > switch to the host uses the switch's hashing setting. > > > We use layer 3+4 hashing on all baremetal hosts (including ceph hosts) > and all switches, and traffic is roughly evenly distributed between the > links. > > > Regards, > > Burkhard > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |