Re: Send bucket policy to OPA server

Casey Bodley <cbodley@xxxxxxxxxx> · Wed, 29 Jan 2020 14:43:36 -0500

On 1/28/20 2:45 PM, Matthias Muench wrote:
Hi,
I think making Ceph special to what the rest of the clients in the 
world would expect would be a bit off the idea of providing S3 like 
service.
To my understanding, setting OPA to be the source of truth would 
introduce latency (based on Casey’s comments) and will not allow to 
set policies (based on Seena).
The first one brings us towards harder latency and especially 
depending on extern systems resource capability (assume central 
resource as the idea is and therefor not necessarily really “in reach” 
within an acceptable latency, routing in addition, etc.). The second 
one says simply that this would break any existing compatibility with 
clients and use cases. To me it looks not that good to loose on both ends.

Agreed. Even if one has to opt-in to this broken s3 compatiblity, I'm 
skeptical that users will find this to be a compelling target for their 
applications.

The existing prototype of OPA integration sends this authorization 
request to OPA -in addition to- radosgw's own authorization logic, where 
we consult any of our user/bucket policies or ACLs that apply. In this 
model, OPA is not the only source of truth. It just has the opportunity 
to deny access that we would otherwise grant, so it doesn't require that 
we break compatibility with any S3 features that conflict with OPA's 
view of policy.

Were we to change this so that OPA was the only source of truth, then 
we'd be left with two bad options: either reject all requests to modify 
policy and break existing applications, or send all policy/ACL 
information to OPA and require every OPA policy script to implement 
s3-compatible enforcement of them. I also don't see any benefit to this 
model - why, if an client wants to use s3 policy to restrict a certain 
access, would OPA want to override that and grant access instead?

I could live more with the latency issue but wouldn’t like it.
For the second, I can understand the idea of having simplification for 
auditing the access but I’m not that convinced to take the burden of 
being “the special” one that nobody wants to work with. So, I would 
love to see the full fledged support of setting the policy by clients, 
no matter what the result would be in terms of implementing it to 
interact with OPA. Instead, having an additional requirement to 
implement additional handling to set policies different from what S3 
actually provides would require special clients first and secondly an 
additional path to OPA with all the additional burden to tweak 
security to allow this path to OPA. I feel that the first wouldn’t 
happen (special clients) and the second in practice not either because 
of security constraints by the OPA admin folks.

G,
-matt

——————————————————
Matthias Muench
Senior Specialist Solution Architect
EMEA Storage Specialist
matthias.muench@xxxxxxxxxx <mailto:mmuench@xxxxxxxxxx>
Phone: +49-160-92654111 <tel:+49-160-92654111>

Red Hat GmbH
Werner-von-Siemens-Ring 14 <x-apple-data-detectors://2/1>
85630 Grasbrunn <x-apple-data-detectors://2/1>
Germany <x-apple-data-detectors://2/1>
_______________________________________________________________________
Red Hat GmbH, http://www.de.redhat.com <http://de.redhat.com/> · 
Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen
HRB 153243 · Managing Directors: Charles Cachera, Michael O'Neill, Tom 
Savage, Eric Shander

On Jan 28, 2020, at 15:02, Seena Fallah <seenafallah@xxxxxxxxx> wrote:

Amazon AWS S3 has two type of policies. One from bucket policy and 
one form IAM. I think it could be better to have two policies models 
in Ceph one from bucket policy and one form OPA if its enable.
So if you are okay we can change the PR to make bucket policy enabled 
when OPA is enabled, too. Because now bucket policies not working 
when OPA integration is enabled.

On Tue, Jan 28, 2020 at 2:57 AM Seena Fallah <seenafallah@xxxxxxxxx 
<mailto:seenafallah@xxxxxxxxx>> wrote:

    Matt When OPA integration is enabled S3 policies doesn’t work! If
    you want them to be worked we should bypass S3 policies to OPA
    for being applied and worked.
    Here we have conflict in OPA integration with S3 policies!

    On Tue, Jan 28, 2020 at 2:52 AM Matt Benjamin
    <mbenjami@xxxxxxxxxx <mailto:mbenjami@xxxxxxxxxx>> wrote:

        My take so far is that this is not a bug, and I'd like not to
        introduce special-case logic to override or suppress
        processing of
        native policy.

        Matt

        On Mon, Jan 27, 2020 at 5:24 PM Seena Fallah
        <seenafallah@xxxxxxxxx <mailto:seenafallah@xxxxxxxxx>> wrote:
        >
        > I think it's very good that Ceph export its authorization
        and we could have external source of truth with it. S3
        policies can transport to OPA and updates by users set/del
        policies.
        > But now we have conflict with OPA integration and S3
        policies which is set when OPA integration is enabled, aren't
        work.
        >
        > Can you all please help to fix this bug?
        >
        > On Fri, Jan 24, 2020 at 1:05 PM Seena Fallah
        <seenafallah@xxxxxxxxx <mailto:seenafallah@xxxxxxxxx>> wrote:
        >>
        >> Hi all.
        >>
        >> Any updates here?
        >>
        >> On Tue, Jan 21, 2020 at 2:50 AM Seena Fallah
        <seenafallah@xxxxxxxxx <mailto:seenafallah@xxxxxxxxx>> wrote:
        >>>
        >>> OPA can be used in companies that uses many services like
        k8s, Ceph,... and want to have one central point for
        authorizing users so they can maintenance their access for
        each user on each service for example and etc. It’s just a
        use case and so it’s really good to have it. I think this is
        the biggest use case for having OPA in products that gets an
        option to centralize authorizing for all types of services.
        >>>
        >>> Performance for this model is issue like having keystone
        with Ceph. So I think it’s based on users that active this
        integrations at all.
        >>>
        >>> The model for writing policies to radosgw isn’t really
        good I think because of the reason above if this accrued
        there is always two copies of policies and it doesn’t sounds
        good for maintaining.
        >>>
        >>> If bucket policy disable, s3 clients like boto3 and etc
        will not work for setting polices but I think when someone is
        enabling OPA for authorizing it will also have an API for
        his/her OPA server to set/del policies and they can call
        these APIs to set/del policies.
        >>> And for extensions like PublicAccessBlock, it will
        disable because OPA is just authorizing requests and Ceph
        doesn’t authorize any request when OPA integration is enabled
        so OPA should handle any incoming policies were made by S3
        policies. So it doesn’t make conflicts and if OPA integration
        is enabled it won’t work as we return 405 on each set/del
        policies requests and if OPA is disabled users can use this
        policies.
        >>>
        >>>
        >>> On Tue, Jan 21, 2020 at 2:05 AM Casey Bodley
        <cbodley@xxxxxxxxxx <mailto:cbodley@xxxxxxxxxx>> wrote:
        >>>>
        >>>> I am a big fan of the IAM policy documents, both because
        of the
        >>>> flexibility and expressiveness they provide, and because
        they're in a
        >>>> format that all of our s3 clients understand.
        >>>>
        >>>> I'm not familiar enough with OPA to know what extra
        capabilities it
        >>>> offers that IAM policy cannot, but I have serious
        concerns about the
        >>>> performance and scalability of a model where radosgw has
        to send
        >>>> blocking RPCs to OPA in order to authorize each and
        every request.
        >>>>
        >>>> On the other hand, consider a model where a Policy Agent
        exercises its
        >>>> control over authorization by writing IAM documents to
        radosgw, which we
        >>>> use to cheaply authorize requests out of our metadata
        cache. I would
        >>>> imagine that this model could cover a lot of interesting
        use cases,
        >>>> without breaking support for existing s3 applications
        that rely on
        >>>> bucket policy - as the proposal to reject
        PutBucketPolicy requests would.
        >>>>
        >>>> Is this something that OPA could feasibly do?
        >>>>
        >>>> For use cases that aren't supported by the existing
        policy grammar,
        >>>> we're open to maintaining extensions to these documents.
        We already
        >>>> implement a number of s3 extensions [1][2] that are
        easily accessible
        >>>> via python/boto and the aws cli.
        >>>>
        >>>> But a model where radosgw outsources authorization
        entirely is a hard
        >>>> sell, because it conflicts with feature development
        going forward. One
        >>>> example would be support for PublicAccessBlock [3],
        where radosgw needs
        >>>> full visibility into policy to detect cases where public
        access would be
        >>>> granted.
        >>>>
        >>>> [1]
        >>>>
        https://docs.ceph.com/docs/master/radosgw/s3/python/#using-s3-api-extensions
        >>>>
        >>>> [2]
        >>>>
        https://github.com/ceph/ceph/blob/master/examples/boto3/service-2.sdk-extras.json
        >>>>
        >>>> [3] https://github.com/ceph/ceph/pull/30033
        >>>>
        >>>> On 1/20/20 12:21 PM, Seena Fallah wrote:
        >>>> > I’m also agree with you Matt that it will free us from
        complexity of
        >>>> > handling S3 policy or Swift ACL if we save the current
        state of OPA.
        >>>> > But if we want use this state of OPA we should act for
        S3 policy and
        >>>> > Swift ACL that if user is setting them it shouldn’t be
        allowed and
        >>>> > return user that you can’t set them! Because now when
        OPA integration
        >>>> > is enabled and user set bucket policy it returns
        success but actually
        >>>> > it doesn’t work!
        >>>> >
        >>>> > What are your thoughts?
        >>>> >
        >>>> > On Sun, Jan 19, 2020 at 12:33 AM Seena Fallah
        <seenafallah@xxxxxxxxx <mailto:seenafallah@xxxxxxxxx>
        >>>> > <mailto:seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx>>> wrote:
        >>>> >
        >>>> >     I think the other problem caused when OPA
        integration is enabled
        >>>> >     and we set bucket policy is when user wants to get
        his/her bucket
        >>>> >     policy. Some policies are set through OPA (for
        example in OPA
        >>>> >     rules we have user A that has access to user B
        bucket so OPA
        >>>> >     return true on authorizing request and it acts
        like bucket policy)
        >>>> >     and some through bucket policy (s3 clients
        command). So when user
        >>>> >     is getting his/her bucket policy what data should
        we return? The
        >>>> >     policies that are set through bucket policy or OPA
        rules for that
        >>>> >     bucket?
        >>>> >
        >>>> >     I fact I think OPA rules are not static and will
        change in time
        >>>> >     and so there should be a client interface for that
        OPA server that
        >>>> >     users could change their rules for their buckets
        (giving access to
        >>>> >     put, get, ... to someone else and etc.). So if the
        client exists
        >>>> >     there is no need to bucket policy and we can make
        it disable (by
        >>>> >     returning 405) when OPA integration is enabled (I
        repeat that
        >>>> >     still now in Ceph latest version when OPA
        integration is enabled
        >>>> >     bucket policies aren’t work!) because the policies
        that are set
        >>>> >     with bucket policy can be check with OPA, too.
        >>>> >
        >>>> >     What’s your opinion?
        >>>> >
        >>>> >     On Fri, Jan 17, 2020 at 9:40 PM Seena Fallah
        >>>> >     <seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx> <mailto:seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx>>> wrote:
        >>>> >
        >>>> >         I think when OPA integration is enabled the
        source of truth
        >>>> >         for authorizing should be OPA (it is right now
        in Ceph and all
        >>>> >         requests are authorizing with OPA and Ceph
        doesn’t authorize
        >>>> >         any request by it self).
        >>>> >         When user is using bucket policy feature
        he/she wants to get
        >>>> >         access to someone else so when he/she is the
        bucket owner,
        >>>> >         he/she can perform this action and we should
        apply this policy
        >>>> >         for him/her. If we want policies just update
        within OPA
        >>>> >         server/client and S3 clients (s3cmd, aws, ...)
        don’t edit
        >>>> >         policies, we should reply to them that
        set/delpolicy isn’t
        >>>> >         allowed from here (return 405 for example;
        just for saying
        >>>> >         that the request that user send isn’t successful).
        >>>> >
        >>>> >         Yes we can have some process and
        simplification before sending
        >>>> >         it to OPA but the s3 policy has a general
        structure so OPA
        >>>> >         server can decode it by it self.
        >>>> >
        >>>> >         On Fri, Jan 17, 2020 at 9:16 PM Matt Benjamin
        >>>> >         <mbenjami@xxxxxxxxxx
        <mailto:mbenjami@xxxxxxxxxx> <mailto:mbenjami@xxxxxxxxxx
        <mailto:mbenjami@xxxxxxxxxx>>> wrote:
        >>>> >
        >>>> >             The larger question, I think, is what OPA
        is supposed to
        >>>> >             do with it.
        >>>> >             The larger question I think it asks is
        whether OPA or Ceph
        >>>> >             owns a
        >>>> >             particular dimension of policy--or,
        perhaps, which owns
        >>>> >             policy for
        >>>> >             what portions of the namespace (at any
        particular point in
        >>>> >             time).
        >>>> >
        >>>> >             Without any new interaction, when OPA is
        configured, OPA
        >>>> >             can make a
        >>>> >             direct authorization decision with all
        available
        >>>> >             information for
        >>>> >             Ceph/RGW, notwithstanding any S3 or Swift
        ACL or S3 policy
        >>>> >             that might
        >>>> >             exist--including any that might have been
        stored prior to
        >>>> >             turning on
        >>>> >             this proposed feature to push policy
        documents to OPA.  This
        >>>> >             overriding property of the OPA integration
        when in use
        >>>> >             frees us from a
        >>>> >             lot of complexity regarding which system
        is the source of
        >>>> >             truth, and
        >>>> >             for what.
        >>>> >
        >>>> >             I can see value in more sophisticated
        integration that
        >>>> >             mutually
        >>>> >             comprehends policy--but I'm having trouble
        with "send
        >>>> >             policy documents
        >>>> >             to OPA, maybe it will do something with them."
        >>>> >
        >>>> >             Matt
        >>>> >
        >>>> >             On Fri, Jan 17, 2020 at 12:01 PM Seena Fallah
        >>>> >             <seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx> <mailto:seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx>>> wrote:
        >>>> >             >
        >>>> >             > Hello Ash
        >>>> >             >
        >>>> >             > With bucket policy user A can get access
        to user B for
        >>>> >             putting object on bucket C. So if this
        policy sent to Ceph
        >>>> >             and OPA integration is enabled it will be
        discard because
        >>>> >             this policy isn’t sent to OPA server to be
        updated.
        >>>> >             > Here is a documentation for bucket policy:
        >>>> >             >
        https://docs.ceph.com/docs/master/radosgw/bucketpolicy/
        >>>> >             >
        >>>> >             > With this PR when user set bucket
        policy, the data of
        >>>> >             that policy will sent to OPA server to be
        applied and so
        >>>> >             OPA can get access to user that gets
        access to bucket via
        >>>> >             bucket policy.
        >>>> >             >
        >>>> >             > On Fri, Jan 17, 2020 at 8:24 PM Ash Narkar
        >>>> >             <ash@xxxxxxxxx <mailto:ash@xxxxxxxxx>
        <mailto:ash@xxxxxxxxx <mailto:ash@xxxxxxxxx>>> wrote:
        >>>> >             >>
        >>>> >             >> Hello Seena,
        >>>> >             >>
        >>>> >             >> The OPA integration is with the RGW and
        the intent is
        >>>> >             to check if an authenticated user is
        allowed to perform a
        >>>> >             particular action on a particular
        resource. For example,
        >>>> >             can Bob delete a bucket based on some
        attribute like his
        >>>> >             location. I am not familiar with the
        internals of Ceph's
        >>>> >             bucket policy command. It would be great
        to get some
        >>>> >             context here and discuss if the bucket
        policy can be
        >>>> >             authorized with OPA which is the intent of
        your PR I believe.
        >>>> >             >>
        >>>> >             >> Thanks
        >>>> >             >> Ash
        >>>> >             >>
        >>>> >             >> On Fri, Jan 17, 2020 at 6:33 AM Seena
        Fallah
        >>>> >             <seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx> <mailto:seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx>>> wrote:
        >>>> >             >>>
        >>>> >             >>> So when OPA integration is enabled the
        bucket policy
        >>>> >             from users will not work!
        >>>> >             >>> I think it’s about Ceph architecture
        not OPA because
        >>>> >             OPA is for authorizing the requests and
        bucket policy is
        >>>> >             one of the authorizing methods that OPA
        should support.
        >>>> >             >>>
        >>>> >             >>> On Fri, Jan 17, 2020 at 5:56 PM Matt
        Benjamin
        >>>> >             <mbenjami@xxxxxxxxxx
        <mailto:mbenjami@xxxxxxxxxx> <mailto:mbenjami@xxxxxxxxxx
        <mailto:mbenjami@xxxxxxxxxx>>> wrote:
        >>>> >             >>>>
        >>>> >             >>>> Hi Seena,
        >>>> >             >>>>
        >>>> >             >>>> As I wrote in a comment on your PR,
        my current
        >>>> >             intuition is that what
        >>>> >             >>>> you're doing here isn't consistent
        with the original
        >>>> >             intent of the OPA
        >>>> >             >>>> integration we currently have, nor
        with the OPA model
        >>>> >             in general.
        >>>> >             >>>>
        >>>> >             >>>> That said, I'd really like some
        feedback from OPA
        >>>> >             architects, CC'd.
        >>>> >             >>>>
        >>>> >             >>>> regards,
        >>>> >             >>>>
        >>>> >             >>>> Matt
        >>>> >             >>>>
        >>>> >             >>>> On Thu, Jan 16, 2020 at 5:04 AM Seena
        Fallah
        >>>> >             <seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx> <mailto:seenafallah@xxxxxxxxx
        <mailto:seenafallah@xxxxxxxxx>>> wrote:
        >>>> >             >>>> >
        >>>> >             >>>> > Hi all. In OPA integration from
        Ceph there is no
        >>>> >             integration for bucket policy.
        >>>> >             >>>> > When user is setting bucket policy
        to his/her
        >>>> >             bucket the OPA server won't get who get's
        access to that
        >>>> >             bucket so after that if the request is
        coming from the
        >>>> >             user (that gets access to that bucket via
        bucket policy)
        >>>> >             to access that bucket (PUT, GET,...), OPA
        will reject that
        >>>> >             because of no data in database.
        >>>> >             >>>> > I have create a pull request for
        this problem so if
        >>>> >             user creates a bucket policy for his/her
        bucket, the
        >>>> >             policy data will send to OPA server to be
        update on the
        >>>> >             database.
        >>>> >             >>>> > I think the main idea of having OPA
        is to have all
        >>>> >             authorization in OPA and Ceph don't
        authorize any request
        >>>> >             by it self.
        >>>> >             >>>> > Here is the pull request and I
        would be thankful to
        >>>> >             hear about your comments.
        >>>> >             >>>> > https://github.com/ceph/ceph/pull/32294
        >>>> >             >>>> > Thanks.
        >>>> >             >>>> >
        _______________________________________________
        >>>> >             >>>> > Dev mailing list -- dev@xxxxxxx
        <mailto:dev@xxxxxxx> <mailto:dev@xxxxxxx <mailto:dev@xxxxxxx>>
        >>>> >             >>>> > To unsubscribe send an email to
        dev-leave@xxxxxxx <mailto:dev-leave@xxxxxxx>
        >>>> >             <mailto:dev-leave@xxxxxxx
        <mailto:dev-leave@xxxxxxx>>
        >>>> >             >>>>
        >>>> >             >>>>
        >>>> >             >>>>
        >>>> >             >>>> --
        >>>> >             >>>>
        >>>> >             >>>> Matt Benjamin
        >>>> >             >>>> Red Hat, Inc.
        >>>> >             >>>> 315 West Huron Street, Suite 140A
        >>>> >             >>>> Ann Arbor, Michigan 48103
        >>>> >             >>>>
        >>>> >             >>>>
        http://www.redhat.com/en/technologies/storage
        >>>> >             >>>>
        >>>> >             >>>> tel.  734-821-5101
        >>>> >             >>>> fax.  734-769-8938
        >>>> >             >>>> cel.  734-216-5309
        >>>> >             >>>>
        >>>> >
        >>>> >
        >>>> >             --
        >>>> >
        >>>> >             Matt Benjamin
        >>>> >             Red Hat, Inc.
        >>>> >             315 West Huron Street, Suite 140A
        >>>> >             Ann Arbor, Michigan 48103
        >>>> >
        >>>> > http://www.redhat.com/en/technologies/storage
        >>>> >
        >>>> >             tel. 734-821-5101
        >>>> >             fax. 734-769-8938
        >>>> >             cel. 734-216-5309
        >>>> >
        >>>> >
        >>>> > _______________________________________________
        >>>> > Dev mailing list -- dev@xxxxxxx <mailto:dev@xxxxxxx>
        >>>> > To unsubscribe send an email to dev-leave@xxxxxxx
        <mailto:dev-leave@xxxxxxx>
        >>>> _______________________________________________
        >>>> Dev mailing list -- dev@xxxxxxx <mailto:dev@xxxxxxx>
        >>>> To unsubscribe send an email to dev-leave@xxxxxxx
        <mailto:dev-leave@xxxxxxx>

        -- 

        Matt Benjamin
        Red Hat, Inc.
        315 West Huron Street, Suite 140A
        Ann Arbor, Michigan 48103

        http://www.redhat.com/en/technologies/storage

        tel.  734-821-5101
        fax.  734-769-8938
        cel.  734-216-5309

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx