Hi,
Changed switch-openrc and verified the project to be "admin".
Unfortunately problems stills.
I think I have configured the Ceph now somehow wrong with command
ceph config set mgr rgw_keystone_url http://controller:5000
It probably should be something like
ceph config set client.radosgw.gateway rgw_keystone_url
http:/controllerc:5000
I am not sure about this though.
I tested configuring these parameters to /etc/ceph/ceph.conf as well, but
not sure if those will affect inside docker containers.
It seems that radosgw won't trigger any communication towards keystone.
Will continue with this.
Thanks,
-Mika
On Thu, Jan 7, 2021 at 3:08 PM Wissem MIMOUNA <
wissem.mimouna@xxxxxxxxxxxxxxxx> wrote:
> The user rgwswift only for radosgw config ( do not use it in your file
> openrc ) use swift user instead . Also , keep the default project to admin
> ( os_project_name ) .
>
>
>
> Rgds
>
>
>
> *De :* Mika Saari <mika.saari@xxxxxxxxx>
> *Envoyé :* jeudi 7 janvier 2021 12:45
> *À :* Wissem MIMOUNA <wissem.mimouna@xxxxxxxxxxxxxxxx>
> *Cc :* ceph-users@xxxxxxx
> *Objet :* Re: Re: Ceph RadosGW & OpenStack swift problem
>
>
>
> Hi,
>
>
>
> Adding below what I tested. Do you see from this what I am doing wrong?
>
>
>
> Thank you very much,
>
> -Mika
>
>
>
> --clip clip--
>
> OPENSTACK SIDE:
> [root@controller ~]# openstack user create --domain default
> --password-prompt rgwswift
> User Password:
> Repeat User Password:
> +---------------------+----------------------------------+
> | Field | Value |
> +---------------------+----------------------------------+
> | domain_id | default |
> | enabled | True |
> | id | 85a86ec5c0264302b0471fd147042e0b |
> | name | rgwswift |
> | options | {} |
> | password_expires_at | None |
> +---------------------+----------------------------------+
> [root@controller ~]# openstack role add --project service --user rgwswift
> admin
>
> CEPH SIDE:
> [root@ceph1 ~]# ceph config set mgr rgw_keystone_accepted_roles "admin,
> _member_, Member, member, creator"
> [root@ceph1 ~]# ceph config set mgr rgw_keystone_admin_user rgwswift
>
> [root@ceph1 ~]# ceph config set mgr rgw_keystone_admin_project service
> [root@ceph1 ~]# ceph orch restart rgw.default.ou
> restart rgw.default.ou.ceph1.gxblht from host 'ceph1'
>
>
> CLIENT SIDE:
> $ . swift-openrc
> Where swift-openrc is like this:
> export OS_PROJECT_DOMAIN_NAME=Default
> export OS_USER_DOMAIN_NAME=Default
> export OS_PROJECT_NAME=service
> export OS_USERNAME=rgwswift
> export OS_PASSWORD=rgwswiftpw
> export OS_AUTH_URL=http://controller:5000/v3
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000_v3&d=DwMFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=oc3C1TP2mMYCukAjjobWV7SPwto-zVeUvBG-JgRS3SI&s=xYsKH127snVkstwVzGM-ha6td0BdcY5-XQxutKOxNto&e=>
> export OS_IDENTITY_API_VERSION=3
> export OS_IMAGE_API_VERSION=2
> $ swift stat --debug
>
> Problem like earlier.
> First the swift client authenticates to the keystone and that works.
> Second it tries to contact radosgw, and that gives 401.
>
> Checked the rgw_process.cc : process_request and seems that there is no
> more debug information in the source. I assume the row int ret =
> client_io->init(g_ceph_context); gives < 0 which causes the process_request
> to return out with abort_early.
>
>
>
> On Thu, Jan 7, 2021 at 1:16 PM Wissem MIMOUNA <
> wissem.mimouna@xxxxxxxxxxxxxxxx> wrote:
>
> Hi,
>
> The radosgw should have a dedicated user (different from you swift user)
> for authentifiation with keystone ( openstack) in the project "service" and
> you should also add the role "_member_" in the rgw_keystone_accepted_roles.
>
> Regards
>
> -----Message d'origine-----
> De : Mika Saari <mika.saari@xxxxxxxxx>
> Envoyé : jeudi 7 janvier 2021 11:35
> À : ceph-users@xxxxxxx
> Objet : Re: Ceph RadosGW & OpenStack swift problem
>
> Hi,
>
> I have added debug_rgw 20 to configuration. When checking docker logs -f
> <radosgw container id> I get this error for my radowgw request (swift post
> test3 --debug)
>
> Would there be a way to get more debug information from radosgw to solve
> this 401 problem ?
>
> Thanks a lot,
> -Mika
>
> --- clip clip ----
> debug 2021-01-07T10:32:42.269+0000 7f1ae111b700 1 ====== starting new
> request req=0x7f1b5b32a6b0 ===== debug 2021-01-07T10:32:42.269+0000
> 7f1ae111b700 1 op->ERRORHANDLER:
> err_no=-1 new_err_no=-1
> debug 2021-01-07T10:32:42.269+0000 7f1ae111b700 1 ====== req done
> req=0x7f1b5b32a6b0 op status=0 http_status=401 latency=0s ====== debug
> 2021-01-07T10:32:42.269+0000 7f1ae111b700 1 beast: 0x7f1b5b32a6b0:
> 10.0.2.10 - - [2021-01-07T10:32:42.269372+0000] "POST
> /swift/v1/AUTH_50f0ce372a4a4ed6a41126852358f097/test3 HTTP/1.1" 401 12 -
> "python-swiftclient-3.9.0" -
> --- clip clip ----
>
>
> On Tue, Jan 5, 2021 at 8:00 PM Mika Saari <mika.saari@xxxxxxxxx> wrote:
>
> > Hi,
> >
> > I am using indeed OpenStack Ussuri release. I changed the "gw swift
> > account in url = true" directly with ceph config set ... command. Also
> > checked that rgw_keystone_accepted_roles is correctly set and not the
> > admin one. Also tested disabling rgw_keystone_verify_ssl.
> >
> > Should radosgw communicate with keystone somehow? I can not see my
> > ceph-cluster requesting anything from keystone through any interface
> > (tcpdump checked this one). I have tested restarting the radosgw with
> > command "ceph orch restart rgw.default.ou" and seems that it brings
> > the container down and up. Not sure though it is enough to bring the
> > settings in use.q
> >
> > Current status is:
> > 1) swift command seems to be able to authenticate with keystone at
> > the very beginning, this is done in the client side.
> > 2) swift command makes a request to radosgw and gets 401
> > INFO:swiftclient:REQ: curl -i <radosgw url
> > here>/swift/v1/AUTH_<some id here>/test3 -X POST -H "X-Auth-Token:
> > here><token " -H "Content-Length: 0"
> > INFO:swiftclient:RESP STATUS: 401 Unauthorized
> >
> > Thanks a lot again,
> > -Mika
> >
> > On Tue, Jan 5, 2021 at 11:19 AM Wissem MIMOUNA <
> > wissem.mimouna@xxxxxxxxxxxxxxxx> wrote:
> >
> >> Hi,
> >>
> >> Which version of OpenStack do you have ? I guess , since Usurri ( or
> >> may be even before ) swift authentification through keystone require
> >> the account in url . You have to add this option in
> >> "/etc/ceph/ceph.conf" , section rgw "rgw swift account in url = true"
> or do it via setting directly
> >> . Also , I noticed you did this ==> 3) ceph config set mgr
> >> rgw_keystone_accepted_admin_roles xxxx || I think , you should use
> >> the option "rgw keystone accepted roles xxxx" instead.
> >>
> >> Regards
> >>
> >> -----Message d'origine-----
> >> De : Mika Saari <mika.saari@xxxxxxxxx> Envoyé : mardi 5 janvier 2021
> >> 10:03 À : ceph-users@xxxxxxx Objet : Ceph RadosGW &
> >> OpenStack swift problem
> >>
> >> Hi,
> >>
> >> Using Ceph 15.2.8 installed with cephadm. Trying to get RadosGW to
> work.
> >> I have managed to get the RadosGW working. I can manage it through a
> >> dashboard and use aws s3 client to create new buckets etc. When
> >> trying to use swift I get errors.
> >>
> >> Not sure how to continue to track the problem here. Any tips are
> >> welcome.
> >>
> >> Thank you very much,
> >> -Mika
> >>
> >> ------- What I have done and what are the results. Some data changed
> >> manually -------
> >> What I have done:
> >> At OpenStack Side:
> >> 1) openstack user create --domain default --password-prompt swift
> >> 2) openstack role add --project service --user swift admin
> >> 3) openstack endpoint create --region RegionOne object-store
> >> public
> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AUTH-5F-25-255C-28project-5Fid-255C-29s&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=-1FtdhjTcNA8jPSUoyoUfsPl5uqTqu4I_ThTOJNLjtg&e=
> >> 4) openstack endpoint create --region RegionOne object-store
> >> internal
> >>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AUTH-5F-25-255C-28project-5Fid-255C-29s&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=-1FtdhjTcNA8jPSUoyoUfsPl5uqTqu4I_ThTOJNLjtg&e=
> >> 5) openstack endpoint create --region RegionOne object-store
> >> admin
> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1&d=
> >> DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Ktt
> >> b6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=bm67b3lMVeLC
> >> 3sNvuyufFCe3AksJgfIgeI8SDorhHMU&e=
> >>
> >> At Ceph side:
> >> 1) ceph config set mgr rgw_keystone_api_version 3
> >> 2) ceph config set mgr rgw_keystone_url
> >>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=lyXWyh-BXrikPWqWM3dcPW4ZofvjiAxnq-nXsjifnEw&e=
> >> 3) ceph config set mgr rgw_keystone_accepted_admin_roles admin
> >> 4) ceph config set mgr rgw_keystone_admin_user swift
> >> 5) ceph config set mgr rgw_keystone_admin_password swift_test
> >> 6) ceph config set mgr rgw_keystone_admin_domain default
> >> 7) ceph config set mgr rgw_keystone_admin_project service
> >> for project I have tested different projects e.g. service and
> >> admin
> >>
> >> Now when testing the API using swift client I get next:
> >> 1) swift post test3 --debug
> >>
> >> DEBUG:keystoneclient.auth.identity.v3.base:Making authentication
> >> request to
> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000
> >> _v3_auth_tokens&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KK
> >> a6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7
> >> eX4&s=-98qpMcc8sdRTdN7AwNPIyGsIK1GaFvi_SC5GtZGUpY&e=
> >> DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
> >> controller:5000
> >> DEBUG:urllib3.connectionpool:http://controller:5000
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000&d=DwMFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=oc3C1TP2mMYCukAjjobWV7SPwto-zVeUvBG-JgRS3SI&s=D3W7JtLCq7AbYLGXj1Tm-RTLE4w95svqucaeAg87aeE&e=>
> "POST
> >> /v3/auth/tokens HTTP/1.1" 201 7032
> >>
> >> . some openstack data here .
> >>
> >> DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
> >> ceph1:80
> >> DEBUG:urllib3.connectionpool:http://ceph1:80
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1-3A80&d=DwMFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=oc3C1TP2mMYCukAjjobWV7SPwto-zVeUvBG-JgRS3SI&s=vfsbb-sSKs_VnT0vrT_MZRnADOCDvRh0208AgDEvLeo&e=>
> "POST
> >> /swift/v1/AUTH_adsfasdfasdfasdfasdfasdf/test3 HTTP/1.1" 401 12
> >> INFO:swiftclient:REQ: curl -i
> >>
> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AU
> >> TH-5Fadsfasdfasdfasdfasdfasdf_test3&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA
> >> &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U
> >> 46oD9d1KMRwdpbF9VLg7eX4&s=g1inMAENxiOpxc4L8FlmbLypegdcQwgH8drm6aoESZ0
> >> &e=
> >> -X POST -H
> >> "X-Auth-Token: <Token would be here>" -H "Content-Length: 0"
> >> INFO:swiftclient:RESP STATUS: 401 Unauthorized
> >>
> >> and finally I get
> >> Container POST failed:
> >>
> >> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AU
> >> TH-5Fadsfasdfasdfasdfasdfasdf_test3&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA
> >> &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U
> >> 46oD9d1KMRwdpbF9VLg7eX4&s=g1inMAENxiOpxc4L8FlmbLypegdcQwgH8drm6aoESZ0
> >> &e=
> >> 401 Unauthorized
> >> b'AccessDenied'
> >> _______________________________________________
> >> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> >> email to ceph-users-leave@xxxxxxx
> >>
> >
> _______________________________________________
> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> email to ceph-users-leave@xxxxxxx
>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
