Hi,
I'm working on it too, use this settings into ceph.conf to setup debug logs
and check what you get:
[client.radosgw.gateway_name]
debug ms = 1
debug rgw = 20
Check also if settings are applied correctly from config file into RGW with
this command:
radosgw-admin --show-config
As suggested also by Pritha, check if in the field "aud" of you token are
present the value "account".
Best
Simone
-----Messaggio originale-----
Da: Pritha Srivastava <prsrivas@xxxxxxxxxx>
Inviato: sabato 19 marzo 2022 05:24
A: Seth Cagampang <seth.cagampang@xxxxxxxxxxx>
Cc: ceph-users <ceph-users@xxxxxxx>
Oggetto: Re: Ceph RADOSGW with Keycloak ODIC
Hi,
When you list the roles, the Condition element of the trust policy in the
role doesn't seem quite right:
"Condition": {
> "StringEquals": {
>
> "localhost:8080/auth/realms/demo:myclient
<http://10.0.26.1:8080/auth/realms/demo:myclient>": "account"
> }
But what you have mentioned in the policy_document just above is correct:
"Condition":{"StringEquals":{"localhost:8080/auth/realms/demo:app_id":"accou
nt"}}
Is the value of 'aud' field in the access token that you generated, set to
"account"?
Another thing to check would be to see that the clientid (myclient) that you
have set in clientIdList as part of create_openid_connect_provider() call,
matches with the value of either clientId or client_id field in the access
token.
Or you can also check rgw logs and see what error is being logged for
AssumeRoleWithWebIdentity.
Thanks,
Pritha
On Sat, Mar 19, 2022 at 12:21 AM Seth Cagampang <seth.cagampang@xxxxxxxxxxx>
wrote:
> Hello,
>
>
>
> It seems like Pritha is the Ceph RGW expert in this forum. I am
> currently trying to integrate CephRGW object storage with KeyCloak as
> the OIDC provider. I am running ceph version 16.2.7 Pacific stable.
>
>
>
> At this point, I am just trying to get a POC working with the python
> scripts provided in the example in these docs <
> https://docs.ceph.com/en/latest/radosgw/STS/#sts-configuration> . Here
> are some step by step instructions on how I set up the ceph cluster
> and KeyCloak server:
>
>
>
> *Set up keycloak server*:
>
> 1. Create new Realm 'demo'
>
> 2. Create 'testuser' and add credentials. Verify that I am able to
> login to the realm using the new credentials.
>
> 3. Create a client 'myclient' and set Access Type as 'confidential' to
> generate client secret
>
> 4. Add a keycloak-oidc provider using the client credentials.
>
> 5. On the client set 'Authorization Enabled' to ON and 'Service
> Accounts Enabled' to ON.
>
>
>
> We should now be able to get the access tokens from the OIDC provider.
> To do this I used the sample curl calls from these docs <
> https://docs.ceph.com/en/latest/radosgw/keycloak/#setting-up-keycloak>
> which I put into scripts:
>
> access_token.sh
>
> #!/bin/bash
>
> KC_REALM=demo
>
> KC_CLIENT=myclient
>
> KC_CLIENT_SECRET=620b31fa-****-****-****-************
>
> KC_SERVER=localhost:8080 <http://10.0.26.1:8080/>
>
> KC_CONTEXT=auth
>
>
>
> # Request Tokens for credentials
>
> KC_RESPONSE=$( \
>
> curl -k -v -X POST \
>
> -H "Content-Type: application/x-www-form-urlencoded" \
>
> -d "scope=openid" \
>
> -d "grant_type=client_credentials" \
>
> -d "client_id=$KC_CLIENT" \
>
> -d "client_secret=$KC_CLIENT_SECRET" \
>
> "http://
> $KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token"
> \
>
> | jq .
>
> )
>
>
>
> KC_ACCESS_TOKEN=$(echo $KC_RESPONSE| jq -r .access_token)
>
> echo $KC_RESPONSE | jq .
>
> echo $KC_ACCESS_TOKEN
>
>
>
> Using this script I am able to get the access token for later usage
> and it has been verified that we are able to get the access token from
> the key cloak OIDC.
>
>
>
> *Set up Ceph Cluster w/ RGW*:
>
> 1. Create Ceph Cluster with OSD's and journals. Create an S3 object
> storage pool and then create an RGW on the cluster manager node.
>
> 2. Enable sts in the gateway config in /etc/ceph/ceph.conf as seen in
> the example from the docs <
> https://docs.ceph.com/en/latest/radosgw/keycloak/#setting-up-keycloak> :
>
> > [client.radosgw.gateway_name]
>
> > rgw sts key = abcdefghijklmnop
>
> > rgw s3 auth use sts = true
>
> 3. Create test users to be used in the test application python script.
>
> > radosgw-admin --uid TESTER --display-name "testuser" --access_key
> > TESTER
> --secret test123 user create
> > radosgw-admin caps add --uid="TESTER" --caps="oidc-provider=*"
> > radosgw-admin caps add --uid="TESTER" --caps="roles=*"
> >
> > radosgw-admin --uid TESTER1 --display-name "testuser1"
> > --access_key
> TESTER1 --secret test321 user create
> > radosgw-admin caps add --uid="TESTER1" --caps="roles=*"
>
> 4. We need to generate thumbprints of the OIDC provider. I used the
> docs here
> <https://docs.ceph.com/en/latest/radosgw/STS/#sts-configuration> to write
a script to generate the thumbprints:
>
> # Get the 'x5c' from this response to turn into an IDP-cert
>
> KEY1_RESPONSE=$(curl -k -v \
>
> -X GET \
>
> -H "Content-Type: application/x-www-form-urlencoded" \
>
>
> "http://localhost:8080/auth/realms/demo/protocol/openid-connect/certs
> "
> \
>
> | jq -r .keys[0].x5c)
>
>
>
> KEY2_RESPONSE=$(curl -k -v \
>
> -X GET \
>
> -H "Content-Type: application/x-www-form-urlencoded" \
>
>
> "http://localhost:8080/auth/realms/demo/protocol/openid-connect/certs
> "
> \
>
> | jq -r .keys[1].x5c)
>
>
>
> echo
>
> echo "Assembling Certificates...."
>
>
>
> # Assemble Cert1
>
> echo '-----BEGIN CERTIFICATE-----' > certificate1.crt
>
> echo $(echo $KEY1_RESPONSE) | sed
> 's/^.//;s/.$//;s/^.//;s/.$//;s/^.//;s/.$//' >> certificate1.crt
>
> echo '-----END CERTIFICATE-----' >> certificate1.crt
>
> echo $(cat certificate1.crt)
>
>
>
> # Assemble Cert2
>
> echo '-----BEGIN CERTIFICATE-----' > certificate2.crt
>
> echo $(echo $KEY2_RESPONSE) | sed
> 's/^.//;s/.$//;s/^.//;s/.$//;s/^.//;s/.$//' >> certificate2.crt
>
> echo '-----END CERTIFICATE-----' >> certificate2.crt
>
> echo $(cat certificate2.crt)
>
>
>
> echo
>
> echo "Generating thumbprints...."
>
> # Create Thumbprint for both certs
>
> PRETHUMBPRINT1=$(openssl x509 -in certificate1.crt -fingerprint
> -noout)
>
> PRETHUMBPRINT2=$(openssl x509 -in certificate2.crt -fingerprint
> -noout)
>
>
>
> PRETHUMBPRINT1=$(echo $PRETHUMBPRINT1 | awk '{ print substr($0, 18)
> }')
>
> PRETHUMBPRINT2=$(echo $PRETHUMBPRINT2 | awk '{ print substr($0, 18)
> }')
>
>
>
> echo "${PRETHUMBPRINT1//:}"
>
> echo "${PRETHUMBPRINT2//:}"
>
> I copied and pasted these thumbprints into the example application
> python script to perform the create_open_id_connect_provider() for the
> 'iam_client'.
>
> 5. Next I filled out the missing information in the example
> application
> script:
>
> #!/usr/bin/python3
>
> import boto3
>
>
>
> iam_client = boto3.client('iam',
>
> aws_access_key_id="TESTER",
>
> aws_secret_access_key="test123",
>
> endpoint_url="http://10.x.x.x:7480";, #<----Ceph RGW endpoint -
> using http for proof of concept
>
> region_name=''
>
> )
>
>
>
> oidc_response = iam_client.create_open_id_connect_provider(
>
> Url="http://localhost:8080/auth/realms/demo";,
>
> ClientIDList=[
>
> "myclient"
>
> ],
>
> ThumbprintList=[
>
> "E43DBA95FC202A9773F3F542F12CF2A831FC7A6F",
>
> "CE0E06206F6F5670E2BC725BD1557D177B3708BF"
>
> ]
>
> )
>
>
>
> policy_document =
>
>
'''{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Fede
rated":["arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/demo"]},"Act
ion":["sts:AssumeRoleWithWebIdentity"],"Condition":{"StringEquals":{"localho
st:8080/auth/realms/demo:app_id":"account"}}}]}'''
>
> role_response = iam_client.create_role(
>
> AssumeRolePolicyDocument=policy_document,
>
> Path='/',
>
> RoleName='S3Access',
>
> )
>
>
>
> role_policy =
>
>
'''{"Version":"2012-10-17","Statement":{"Effect":"Allow","Action":"s3:*","Re
source":"arn:aws:s3:::*"}}'''
>
> response = iam_client.put_role_policy(
>
> RoleName='S3Access',
>
> PolicyName='Policy1',
>
> PolicyDocument=role_policy
>
> )
>
> #TESTER1
>
> sts_client = boto3.client('sts',
>
> aws_access_key_id="TESTER1",
>
> aws_secret_access_key="test321",
>
> endpoint_url="http://10.x.x.x:7480";,
>
> region_name='',
>
> )
>
>
>
> response = sts_client.assume_role_with_web_identity(
>
> RoleArn=get_response['Role']['Arn'],
>
> RoleSessionName='Bob',
>
> DurationSeconds=3600,
>
> WebIdentityToken="inserted-access-token" #<---- access token from
> step
> 5 in keycloak setup
>
> )
>
>
>
> s3client = boto3.client('s3',
>
> aws_access_key_id = response['Credentials']['AccessKeyId'],
>
> aws_secret_access_key =
> response['Credentials']['SecretAccessKey'],
>
> aws_session_token = response['Credentials']['SessionToken'],
>
> endpoint_url="http://10.x.x.x:7480";,
>
> region_name='',
>
> )
>
>
>
> bucket_name = 'my-bucket'
>
> s3bucket = s3client.create_bucket(Bucket=bucket_name)
>
> resp = s3client.list_buckets()
>
>
>
> print(resp)
>
>
>
> Currently, This script cannot run to completion. I find that it throws
> a fatal error when trying to run
'sts_client.assume_role_with_web_identity()'
> with the exception: "An error occurred (Unknown) when calling the
> AssumeRoleWithWebIdentity operation: Unknown" .
>
>
>
> I am able to verify that the OIDC provider and role has been created
> on the
> RGW: >root@terminal# aws --endpoint=http://10.x.x.x:7480/ iam
> list-open-id-connect-providers --region=""
> >{
> > "OpenIDConnectProviderList": [
> > {
> > "Arn":
> "arn:aws:iam:::oidc-provider/localhost:8080/auth/realms/demo"
> > }
> > ]
> >}
> >root@terminal# aws --endpoint=http://10.x.x.x:7480/ iam
> get-open-id-connect-provider
>
>
--open-id-connect-provider-arn="arn:aws:iam:::oidc-provider/localhost:8080/a
uth/realms/demo"
> --region=""
> >{
> > "Url": "http://localhost:8080/auth/realms/demo";,
> > "ClientIDList": [
> > "myclient"
> > ],
> > "ThumbprintList": [
> > "E43DBA95FC202A9773F3F542F12CF2A831FC7A6F",
> > "CE0E06206F6F5670E2BC725BD1557D177B3708BF"
> > ],
> > "CreateDate": "2022-03-15T17:40:17.572000+00:00"
> >}
>
> >root@terminal# aws --endpoint=http://10.x.x.x:7480/ iam list-roles
> --region=""
>
> >{
> > "Roles": [
> > {
> > "Path": "/",
> > "RoleName": "S3Access",
> > "RoleId": "7329b54e-40a2-4476-ae68-52a72b43376c",
> > "Arn": "arn:aws:iam:::role/S3Access",
> > "CreateDate": "2022-03-14T21:30:11.750000+00:00",
> > "AssumeRolePolicyDocument": {
> > "Version": "2012-10-17",
> > "Statement": [
> > {
> > "Effect": "Allow",
> > "Principal": {
> > "Federated": [
> >
> >"arn:aws:iam:::oidc-provider/localhost
> :8080/auth/realms/demo <http://10.0.26.1:8080/auth/realms/demo>"
> > ]
> > },
> > "Action": [
> > "sts:AssumeRoleWithWebIdentity"
> > ],
> > "Condition": {
> > "StringEquals": {
> >
> > "localhost:8080/auth/realms/demo:myclient
> <http://10.0.26.1:8080/auth/realms/demo:myclient>": "account"
> > }
> > }
> > }
> > ]
> > },
> > "MaxSessionDuration": 3600
> > }
> > ]
> >}
>
>
>
> I must be missing something here. Any advice you might have for me
> would be greatly appreciated.
>
>
>
> Thank you
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> email to ceph-users-leave@xxxxxxx
>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email
to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
