GlusterFS SSL behaviour change breaks openstack Manila GlusterFS Native driver

[Deepak Shetty <dpkshetty@xxxxxxxxx>]

Hi Jeff

  The recent commit (548547b2e) with subject "transport: fix default behavior for SSL authorization" changes the behaviour of GlusterFS SSL in a way that causes to break the Manila usecase.

  I am willing to change on the Manila side, but before I do, i wanted to quickly put my thoughts on the above change you made and ask some Qs.

Before the 548547b2e patch:

1) To enable glusterfs in SSL mode, I used to enable client.ssl and server.ssl options

This ensured that no clients (tenants in openstack lingo) with right certificate can access/mount the gluster volume, because default behaviour was AUTH_REJECT

2) As part of Manila allow acces to share API - i used to enable ssl.auth-allow <CN>, this made the tenant with the right certificate mount the volume and others couldn't

3) As part of Manila deny access to share API - I used to remove (reset) ssl.auth-allow, this ensured that the  tenant couldn't mount, even if he has the right certificate, at the same time ensuring that GlusterFS volume is SSLenabled (bcos of client and server .ssl being on)

After the 548547b2e patch:

1) Refer to #1 above. This isn't true anymore, because not having the ssl.auth-allow means anyone with the certificate can access, because default behaviour changed to AUTH_DONT_CARE (iiuc)

2) Refer to #2 above. This remains same

3) Refer to #3 above. Now removing ssl.auth-allow doesn't work anymore, because removing it means the tenants can still connect (if they try to) and deny access is broken - thus my Manila native driver is broken.

Q1) So, given the behaviour, is there a way to keep the volume SSL enabled and yet deny access to clients ? (which was earlier possible via ssl.auth-allow's old behaviour)

Now, if i want to comply with the new behaviour, as part of deny access ...

    3a) I need to remove all the 3 options (ssl.auth-allow, client.ssl, server.ssl) and add a auth.reject * (because we don't have ssl.auth-reject option support) to ensure the volume cannot be mounted by tenants/clients, tho' local clients (trusted ones) should be able to connect. This effectively means I disable SSL and use IP based auth.reject *, but havign SSL based volume was the requirement to begin with, so disablign SSL seems like compromising security here.

    -- or --

    3b) I can also do as ... remove ssl.auth-allow, and disable one of client/server.ssl option, but then trusted clients (incl quotad etc) also  won't be able to connect

   -- or --

    3c) As part of deny access i keep client/server.ssl on & change ssl.auth-allow <CN> where CN=some randomly generated UUID and hope no one has the cert with that UUID as CN, this won't work for trusted clients too

--or --

    3d) As part of deny access, i keep client/server.ssl on & ensure that ssl.auth-allow is a list of CNs where CN1 = gluster server's CN, CN2 = randomly generated UUID. Now local trusted clients should be able to connect, so server administration isn't broken and external clients cannot connect. This effectively means whenever i modify ssl.auth-allow from my Manila driver,  i ensure that keep CN1 as-is and add/remove CN2 based on allow/deny access behaviour needed.

  -- or --

    3e) Have support for ssl.auth-reject option in GlusterFS. As part of deny access remove ssl.auth-allow, keep client.ssl, server.ssl options,  & do ssl.auth-reject *, but this will break trusted clients again! But this is in contrast to the auth.reject * behaviour where trusted clients are able to connect. Because git commit 548547b2e aimed to make ssl's auth-allow behaviour consistent with non-ssl behaviour, here too when we add ssl.auth-reject support, we should figure a way to allow trusted clients to be able to connect as long as client/server.ssl on is present.

Looking at the options above ...

1) #3e seems logical to me, but isn't implemented yet.

2) #3d has the addnl overhead of maintaining the list of CNs as part of ssl.auth-allow get/set, which isn't ideal (compared to #3e) , but given the status quo , seems to be the only way forward.

3) #3a can be done too, but is defintely not clean way, and is more of a workaround than solution

On a side note:
    Option #3d also means Manila code does a get first , appends either random UUID or correct CN (depending on deny/allow access) and then sets it. This also means the Manila code knows what the server CN is and potentially can create a SSL cert with that CN and since we trust the client CA, it can connect to server as server itself, but I believe we can safely assume that Manila servcies run inside openstack nodes which are part of the deployement hence safe enuf not to be hacked this way :).

Thoughts ?

thanx,

deepak

_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel