Re: [ceph-users] RGW Beast frontend and ipv6 options

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



After discussing with Casey, I'd like to propose some clarifications to this.

First, we do not treat EAFNOSUPPORT as a non-fatal error. Any other error binding is fatal, but that one we warn and continue.

Second, we treat "port=<port>" as expanding to "endpoint=0.0.0.0:<port>, endpoint=[::<port>]".

Then, we just process the set of endpoints properly. This should, I believe, result in simple, straight-forward code, and easily understandable semantics, and should make it simple for orchetrators.

This would make 1 and 2 below fallout naturally. 3 is modified so that we only use configured endpoints, but port= is now implicit endpoint configuration.

Daniel

On 5/2/19 10:08 AM, Daniel Gryniewicz wrote:
Based on past experience with this issue in other projects, I would propose this:

1. By default (rgw frontends=beast), we should bind to both IPv4 and IPv6, if available.

2. Just specifying port (rgw frontends=beast port=8000) should apply to both IPv4 and IPv6, if available.

3. If the user provides endpoint config, we should use only that endpoint config.  For example, if they provide only v4 addresses, we should only bind to v4.

This should all be independent of the bindv6only setting; that is, we should specifically bind our v4 and v6 addresses, and not depend on the system to automatically bind v4 when binding v6.

In the case of 1 or 2, if the system has disabled either v4 or v6, this should not be an error, as long as one of the two binds works.  In the case of 3, we should error out if any configured endpoint cannot be bound.

This should allow an orchestrator to confidently install a system, knowing what will happen, without needing to know or manipulate the bindv6only flag.

As for what happens if you specify an endpoint and a port, I don't have a strong opinion.  I see 2 reasonable possibilites:

a. Make it an error

b. Treat a port in this case as an endpoint of 0.0.0.0:port (v4-only)

Daniel

On 4/26/19 4:49 AM, Abhishek Lekshmanan wrote:

Currently RGW's beast frontend supports ipv6 via the endpoint
configurable. The port option will bind to ipv4 _only_.

http://docs.ceph.com/docs/master/radosgw/frontends/#options

Since many Linux systems may default the sysconfig net.ipv6.bindv6only
flag to true, it usually means that specifying a v6 endpoint will bind
to both v4 and v6. But this also means that deployment systems must be
aware of this while configuring depending on whether both v4 and v6
endpoints need to work or not. Specifying both a v4 and v6 endpoint or a
port (v4) and endpoint with the same v6 port will currently lead to a
failure as the system would've already bound the v6 port to both v4 and
v6. This leaves us with a few options.

1. Keep the implicit behaviour as it is, document this, as systems are
already aware of sysconfig flags and will expect that at a v6 endpoint
will bind to both v4 and v6.

2. Be explicit with endpoints & configuration, Beast itself overrides
the socket option to bind both v4 and v6, which means that v6 endpoint
will bind to v6 *only* and binding to a v4 will need an explicit
specification. (there is a pr in progress for this:
https://github.com/ceph/ceph/pull/27270)

Any more suggestions on how systems handle this are also welcome.

--
Abhishek
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com






[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux