Hi,
When you list the roles, the Condition element of the trust policy in the
role doesn't seem quite right:
"Condition": {
> "StringEquals": {
> "localhost:8080/auth/realms/demo:myclient
<http://10.0.26.1:8080/auth/realms/demo:myclient>": "account"
> }
But what you have mentioned in the policy_document just above is correct:
"Condition":{"StringEquals":{"localhost:8080/auth/realms/demo:app_id":"account"}}
Is the value of 'aud' field in the access token that you generated, set to
"account"?
Another thing to check would be to see that the clientid (myclient) that
you have set in clientIdList as part of create_openid_connect_provider()
call, matches with the value of either clientId or client_id field in the
access token.
Or you can also check rgw logs and see what error is being logged for
AssumeRoleWithWebIdentity.
Thanks,
Pritha
On Sat, Mar 19, 2022 at 12:21 AM Seth Cagampang <seth.cagampang@xxxxxxxxxxx>
wrote:
> Hello,
>
>
>
> It seems like Pritha is the Ceph RGW expert in this forum. I am currently
> trying to integrate CephRGW object storage with KeyCloak as the OIDC
> provider. I am running ceph version 16.2.7 Pacific stable.
>
>
>
> At this point, I am just trying to get a POC working with the python
> scripts provided in the example in these docs <
> https://docs.ceph.com/en/latest/radosgw/STS/#sts-configuration> . Here are
> some step by step instructions on how I set up the ceph cluster and
> KeyCloak server:
>
>
>
> *Set up keycloak server*:
>
> 1. Create new Realm 'demo'
>
> 2. Create 'testuser' and add credentials. Verify that I am able to login to
> the realm using the new credentials.
>
> 3. Create a client 'myclient' and set Access Type as 'confidential' to
> generate client secret
>
> 4. Add a keycloak-oidc provider using the client credentials.
>
> 5. On the client set 'Authorization Enabled' to ON and 'Service Accounts
> Enabled' to ON.
>
>
>
> We should now be able to get the access tokens from the OIDC provider. To
> do this I used the sample curl calls from these docs <
> https://docs.ceph.com/en/latest/radosgw/keycloak/#setting-up-keycloak>
> which I put into scripts:
>
> access_token.sh
>
> #!/bin/bash
>
> KC_REALM=demo
>
> KC_CLIENT=myclient
>
> KC_CLIENT_SECRET=620b31fa-****-****-****-************
>
> KC_SERVER=localhost:8080 <http://10.0.26.1:8080/>
>
> KC_CONTEXT=auth
>
>
>
> # Request Tokens for credentials
>
> KC_RESPONSE=$( \
>
> curl -k -v -X POST \
>
> -H "Content-Type: application/x-www-form-urlencoded" \
>
> -d "scope=openid" \
>
> -d "grant_type=client_credentials" \
>
> -d "client_id=$KC_CLIENT" \
>
> -d "client_secret=$KC_CLIENT_SECRET" \
>
> "http://
> $KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token"
> \
>
> | jq .
>
> )
>
>
>
> KC_ACCESS_TOKEN=$(echo $KC_RESPONSE| jq -r .access_token)
>
> echo $KC_RESPONSE | jq .
>
> echo $KC_ACCESS_TOKEN
>
>
>
> Using this script I am able to get the access token for later usage and it
> has been verified that we are able to get the access token from the key
> cloak OIDC.
>
>
>
> *Set up Ceph Cluster w/ RGW*:
>
> 1. Create Ceph Cluster with OSD's and journals. Create an S3 object storage
> pool and then create an RGW on the cluster manager node.
>
> 2. Enable sts in the gateway config in /etc/ceph/ceph.conf as seen in the
> example from the docs <
> https://docs.ceph.com/en/latest/radosgw/keycloak/#setting-up-keycloak> :
>
> > [client.radosgw.gateway_name]
>
> > rgw sts key = abcdefghijklmnop
>
> > rgw s3 auth use sts = true
>
> 3. Create test users to be used in the test application python script.
>
> > radosgw-admin --uid TESTER --display-name "testuser" --access_key TESTER
> --secret test123 user create
> > radosgw-admin caps add --uid="TESTER" --caps="oidc-provider=*"
> > radosgw-admin caps add --uid="TESTER" --caps="roles=*"
> >
> > radosgw-admin --uid TESTER1 --display-name "testuser1" --access_key
> TESTER1 --secret test321 user create
> > radosgw-admin caps add --uid="TESTER1" --caps="roles=*"
>
> 4. We need to generate thumbprints of the OIDC provider. I used the docs
> here <https://docs.ceph.com/en/latest/radosgw/STS/#sts-configuration> to
> write a script to generate the thumbprints:
>
> # Get the 'x5c' from this response to turn into an IDP-cert
>
> KEY1_RESPONSE=$(curl -k -v \
>
> -X GET \
>
> -H "Content-Type: application/x-www-form-urlencoded" \
>
> "http://localhost:8080/auth/realms/demo/protocol/openid-connect/certs
> "
> \
>
> | jq -r .keys[0].x5c)
>
>
>
> KEY2_RESPONSE=$(curl -k -v \
>
> -X GET \
>
> -H "Content-Type: application/x-www-form-urlencoded" \
>
> "http://localhost:8080/auth/realms/demo/protocol/openid-connect/certs
> "
> \
>
> | jq -r .keys[1].x5c)
>
>
>
> echo
>
> echo "Assembling Certificates...."
>
>
>
> # Assemble Cert1
>
> echo '-----BEGIN CERTIFICATE-----' > certificate1.crt
>
> echo $(echo $KEY1_RESPONSE) | sed
> 's/^.//;s/.$//;s/^.//;s/.$//;s/^.//;s/.$//' >> certificate1.crt
>
> echo '-----END CERTIFICATE-----' >> certificate1.crt
>
> echo $(cat certificate1.crt)
>
>
>
> # Assemble Cert2
>
> echo '-----BEGIN CERTIFICATE-----' > certificate2.crt
>
> echo $(echo $KEY2_RESPONSE) | sed
> 's/^.//;s/.$//;s/^.//;s/.$//;s/^.//;s/.$//' >> certificate2.crt
>
> echo '-----END CERTIFICATE-----' >> certificate2.crt
>
> echo $(cat certificate2.crt)
>
>
>
> echo
>
> echo "Generating thumbprints...."
>
> # Create Thumbprint for both certs
>
> PRETHUMBPRINT1=$(openssl x509 -in certificate1.crt -fingerprint -noout)
>
> PRETHUMBPRINT2=$(openssl x509 -in certificate2.crt -fingerprint -noout)
>
>
>
> PRETHUMBPRINT1=$(echo $PRETHUMBPRINT1 | awk '{ print substr($0, 18) }')
>
> PRETHUMBPRINT2=$(echo $PRETHUMBPRINT2 | awk '{ print substr($0, 18) }')
>
>
>
> echo "${PRETHUMBPRINT1//:}"
>
> echo "${PRETHUMBPRINT2//:}"
>
> I copied and pasted these thumbprints into the example application python
> script to perform the create_open_id_connect_provider() for the
> 'iam_client'.
>
> 5. Next I filled out the missing information in the example application
> script:
>
> #!/usr/bin/python3
>
> import boto3
>
>
>
> iam_client = boto3.client('iam',
>
> aws_access_key_id="TESTER",
>
> aws_secret_access_key="test123",
>
> endpoint_url="http://10.x.x.x:7480";, #<----Ceph RGW endpoint - using
> http for proof of concept
>
> region_name=''
>
> )
>
>
>
> oidc_response = iam_client.create_open_id_connect_provider(
>
> Url="http://localhost:8080/auth/realms/demo";,
>
> ClientIDList=[
>
> "myclient"
>
> ],
>
> ThumbprintList=[
>
> "E43DBA95FC202A9773F3F542F12CF2A831FC7A6F",
>
> "CE0E06206F6F5670E2BC725BD1557D177B3708BF"
>
> ]
>
> )
>
>
>
> policy_document =
>
> '''{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Federated":["arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/demo"]},"Action":["sts:AssumeRoleWithWebIdentity"],"Condition":{"StringEquals":{"localhost:8080/auth/realms/demo:app_id":"account"}}}]}'''
>
> role_response = iam_client.create_role(
>
> AssumeRolePolicyDocument=policy_document,
>
> Path='/',
>
> RoleName='S3Access',
>
> )
>
>
>
> role_policy =
>
> '''{"Version":"2012-10-17","Statement":{"Effect":"Allow","Action":"s3:*","Resource":"arn:aws:s3:::*"}}'''
>
> response = iam_client.put_role_policy(
>
> RoleName='S3Access',
>
> PolicyName='Policy1',
>
> PolicyDocument=role_policy
>
> )
>
> #TESTER1
>
> sts_client = boto3.client('sts',
>
> aws_access_key_id="TESTER1",
>
> aws_secret_access_key="test321",
>
> endpoint_url="http://10.x.x.x:7480";,
>
> region_name='',
>
> )
>
>
>
> response = sts_client.assume_role_with_web_identity(
>
> RoleArn=get_response['Role']['Arn'],
>
> RoleSessionName='Bob',
>
> DurationSeconds=3600,
>
> WebIdentityToken="inserted-access-token" #<---- access token from step
> 5 in keycloak setup
>
> )
>
>
>
> s3client = boto3.client('s3',
>
> aws_access_key_id = response['Credentials']['AccessKeyId'],
>
> aws_secret_access_key = response['Credentials']['SecretAccessKey'],
>
> aws_session_token = response['Credentials']['SessionToken'],
>
> endpoint_url="http://10.x.x.x:7480";,
>
> region_name='',
>
> )
>
>
>
> bucket_name = 'my-bucket'
>
> s3bucket = s3client.create_bucket(Bucket=bucket_name)
>
> resp = s3client.list_buckets()
>
>
>
> print(resp)
>
>
>
> Currently, This script cannot run to completion. I find that it throws a
> fatal error when trying to run 'sts_client.assume_role_with_web_identity()'
> with the exception: "An error occurred (Unknown) when calling the
> AssumeRoleWithWebIdentity operation: Unknown" .
>
>
>
> I am able to verify that the OIDC provider and role has been created on the
> RGW: >root@terminal# aws --endpoint=http://10.x.x.x:7480/ iam
> list-open-id-connect-providers --region=""
> >{
> > "OpenIDConnectProviderList": [
> > {
> > "Arn":
> "arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/demo"
> > }
> > ]
> >}
> >root@terminal# aws --endpoint=http://10.x.x.x:7480/ iam
> get-open-id-connect-provider
>
> --open-id-connect-provider-arn="arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/demo"
> --region=""
> >{
> > "Url": "http://localhost:8080/auth/realms/demo";,
> > "ClientIDList": [
> > "myclient"
> > ],
> > "ThumbprintList": [
> > "E43DBA95FC202A9773F3F542F12CF2A831FC7A6F",
> > "CE0E06206F6F5670E2BC725BD1557D177B3708BF"
> > ],
> > "CreateDate": "2022-03-15T17:40:17.572000+00:00"
> >}
>
> >root@terminal# aws --endpoint=http://10.x.x.x:7480/ iam list-roles
> --region=""
>
> >{
> > "Roles": [
> > {
> > "Path": "/",
> > "RoleName": "S3Access",
> > "RoleId": "7329b54e-40a2-4476-ae68-52a72b43376c",
> > "Arn": "arn:aws:iam:::role/S3Access",
> > "CreateDate": "2022-03-14T21:30:11.750000+00:00",
> > "AssumeRolePolicyDocument": {
> > "Version": "2012-10-17",
> > "Statement": [
> > {
> > "Effect": "Allow",
> > "Principal": {
> > "Federated": [
> > "arn:aws:iam:::oidc-provider/localhost
> :8080/auth/realms/demo <http://10.0.26.1:8080/auth/realms/demo>"
> > ]
> > },
> > "Action": [
> > "sts:AssumeRoleWithWebIdentity"
> > ],
> > "Condition": {
> > "StringEquals": {
> > "localhost:8080/auth/realms/demo:myclient
> <http://10.0.26.1:8080/auth/realms/demo:myclient>": "account"
> > }
> > }
> > }
> > ]
> > },
> > "MaxSessionDuration": 3600
> > }
> > ]
> >}
>
>
>
> I must be missing something here. Any advice you might have for me would be
> greatly appreciated.
>
>
>
> Thank you
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx
> To unsubscribe send an email to ceph-users-leave@xxxxxxx
>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
