Hello,
thank you very much. Your debugging helped me a lot finding a solution for my own problem with keystone and radosgw.
Greets Stefan
----- Original Message -----
From: "Mika Saari" <mika.saari@xxxxxxxxx>
To: "ceph-users" <ceph-users@xxxxxxx>
Sent: Friday, 8 January, 2021 08:02:31
Subject: Re: Ceph RadosGW & OpenStack swift problem
Hi all,
Thanks a lot for the tips. I finally found out what was the problem. The
"WHO" field in the "ceph config set" was not correct. I was using
"client.radosgw.gateway" but after using "ceph config dump" I found out
that when radosgw was executed up it had generated "client.rgw.default"
named "WHO" field with some parameters, which was the correct one to use.
After using commands in ceph/radosgw machine, everything started to work:
ceph config set client.rgw.default rgw_keystone_url
http://controller:5000
ceph config set client.rgw.default rgw_keystone_admin_domain default
ceph config set client.rgw.default rgw_keystone_admin_password
rgwswiftpw
ceph config set client.rgw.default rgw_keystone_admin_project service
ceph config set client.rgw.default rgw_keystone_admin_user rgwswift
ceph config set client.rgw.default rgw_keystone_api_version 3
ceph config set client.rgw.default rgw_swift_account_in_url true
After these commands RadosGW started to communicate to keystone, created
accounts to radosgw users and also bucket creation started to work. These
all with OpenStack Ussuri & Ceph Octopus 15.2.8.
Summary would be:
1) Check correct "WHO" with ceph config dump after the radosgw is running
2) Create correct users to openstack
3) Start using
Thanks a lot once more for all the help !
-Mika
On Fri, Jan 8, 2021 at 5:27 AM Tony Liu <tonyliu0592@xxxxxxxxxxx> wrote:
> Is swift service endpoint created in OpenStack?
>
> Tony
> > -----Original Message-----
> > From: Mika Saari <mika.saari@xxxxxxxxx>
> > Sent: Thursday, January 7, 2021 3:45 AM
> > To: Wissem MIMOUNA <wissem.mimouna@xxxxxxxxxxxxxxxx>
> > Cc: ceph-users@xxxxxxx
> > Subject: Re: Ceph RadosGW & OpenStack swift problem
> >
> > Hi,
> >
> > Adding below what I tested. Do you see from this what I am doing wrong?
> >
> > Thank you very much,
> > -Mika
> >
> > --clip clip--
> > OPENSTACK SIDE:
> > [root@controller ~]# openstack user create --domain default --password-
> > prompt rgwswift User Password:
> > Repeat User Password:
> > +---------------------+----------------------------------+
> > | Field | Value |
> > +---------------------+----------------------------------+
> > | domain_id | default |
> > | enabled | True |
> > | id | 85a86ec5c0264302b0471fd147042e0b |
> > | name | rgwswift |
> > | options | {} |
> > | password_expires_at | None |
> > +---------------------+----------------------------------+
> > [root@controller ~]# openstack role add --project service --user
> > rgwswift admin
> >
> > CEPH SIDE:
> > [root@ceph1 ~]# ceph config set mgr rgw_keystone_accepted_roles "admin,
> > _member_, Member, member, creator"
> > [root@ceph1 ~]# ceph config set mgr rgw_keystone_admin_user rgwswift
> > [root@ceph1 ~]# ceph config set mgr rgw_keystone_admin_project service
> > [root@ceph1 ~]# ceph orch restart rgw.default.ou restart
> > rgw.default.ou.ceph1.gxblht from host 'ceph1'
> >
> >
> > CLIENT SIDE:
> > $ . swift-openrc
> > Where swift-openrc is like this:
> > export OS_PROJECT_DOMAIN_NAME=Default
> > export OS_USER_DOMAIN_NAME=Default
> > export OS_PROJECT_NAME=service
> > export OS_USERNAME=rgwswift
> > export OS_PASSWORD=rgwswiftpw
> > export OS_AUTH_URL=http://controller:5000/v3
> > export OS_IDENTITY_API_VERSION=3
> > export OS_IMAGE_API_VERSION=2
> > $ swift stat --debug
> >
> > Problem like earlier.
> > First the swift client authenticates to the keystone and that works.
> > Second it tries to contact radosgw, and that gives 401.
> >
> > Checked the rgw_process.cc : process_request and seems that there is no
> > more debug information in the source. I assume the row int ret =
> > client_io->init(g_ceph_context); gives < 0 which causes the
> > process_request to return out with abort_early.
> >
> > On Thu, Jan 7, 2021 at 1:16 PM Wissem MIMOUNA <
> > wissem.mimouna@xxxxxxxxxxxxxxxx> wrote:
> >
> > > Hi,
> > >
> > > The radosgw should have a dedicated user (different from you swift
> > > user) for authentifiation with keystone ( openstack) in the project
> > > "service" and you should also add the role "_member_" in the
> > rgw_keystone_accepted_roles.
> > >
> > > Regards
> > >
> > > -----Message d'origine-----
> > > De : Mika Saari <mika.saari@xxxxxxxxx> Envoyé : jeudi 7 janvier 2021
> > > 11:35 À : ceph-users@xxxxxxx Objet : Re: Ceph RadosGW &
> > > OpenStack swift problem
> > >
> > > Hi,
> > >
> > > I have added debug_rgw 20 to configuration. When checking docker
> > > logs -f <radosgw container id> I get this error for my radowgw request
> > > (swift post
> > > test3 --debug)
> > >
> > > Would there be a way to get more debug information from radosgw to
> > > solve this 401 problem ?
> > >
> > > Thanks a lot,
> > > -Mika
> > >
> > > --- clip clip ----
> > > debug 2021-01-07T10:32:42.269+0000 7f1ae111b700 1 ====== starting new
> > > request req=0x7f1b5b32a6b0 ===== debug 2021-01-07T10:32:42.269+0000
> > > 7f1ae111b700 1 op->ERRORHANDLER:
> > > err_no=-1 new_err_no=-1
> > > debug 2021-01-07T10:32:42.269+0000 7f1ae111b700 1 ====== req done
> > > req=0x7f1b5b32a6b0 op status=0 http_status=401 latency=0s ====== debug
> > > 2021-01-07T10:32:42.269+0000 7f1ae111b700 1 beast: 0x7f1b5b32a6b0:
> > > 10.0.2.10 - - [2021-01-07T10:32:42.269372+0000] "POST
> > > /swift/v1/AUTH_50f0ce372a4a4ed6a41126852358f097/test3 HTTP/1.1" 401 12
> > > - "python-swiftclient-3.9.0" -
> > > --- clip clip ----
> > >
> > >
> > > On Tue, Jan 5, 2021 at 8:00 PM Mika Saari <mika.saari@xxxxxxxxx>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > I am using indeed OpenStack Ussuri release. I changed the "gw
> > > > swift account in url = true" directly with ceph config set ...
> > > > command. Also checked that rgw_keystone_accepted_roles is correctly
> > > > set and not the admin one. Also tested disabling
> > rgw_keystone_verify_ssl.
> > > >
> > > > Should radosgw communicate with keystone somehow? I can not see my
> > > > ceph-cluster requesting anything from keystone through any interface
> > > > (tcpdump checked this one). I have tested restarting the radosgw
> > > > with command "ceph orch restart rgw.default.ou" and seems that it
> > > > brings the container down and up. Not sure though it is enough to
> > > > bring the settings in use.q
> > > >
> > > > Current status is:
> > > > 1) swift command seems to be able to authenticate with keystone
> > > > at the very beginning, this is done in the client side.
> > > > 2) swift command makes a request to radosgw and gets 401
> > > > INFO:swiftclient:REQ: curl -i <radosgw url
> > > > here>/swift/v1/AUTH_<some id here>/test3 -X POST -H "X-Auth-Token:
> > > > here><token " -H "Content-Length: 0"
> > > > INFO:swiftclient:RESP STATUS: 401 Unauthorized
> > > >
> > > > Thanks a lot again,
> > > > -Mika
> > > >
> > > > On Tue, Jan 5, 2021 at 11:19 AM Wissem MIMOUNA <
> > > > wissem.mimouna@xxxxxxxxxxxxxxxx> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> Which version of OpenStack do you have ? I guess , since Usurri (
> > > >> or may be even before ) swift authentification through keystone
> > > >> require the account in url . You have to add this option in
> > > >> "/etc/ceph/ceph.conf" , section rgw "rgw swift account in url =
> > true"
> > > or do it via setting directly
> > > >> . Also , I noticed you did this ==> 3) ceph config set mgr
> > > >> rgw_keystone_accepted_admin_roles xxxx || I think , you should use
> > > >> the option "rgw keystone accepted roles xxxx" instead.
> > > >>
> > > >> Regards
> > > >>
> > > >> -----Message d'origine-----
> > > >> De : Mika Saari <mika.saari@xxxxxxxxx> Envoyé : mardi 5 janvier
> > > >> 2021
> > > >> 10:03 À : ceph-users@xxxxxxx Objet : Ceph RadosGW &
> > > >> OpenStack swift problem
> > > >>
> > > >> Hi,
> > > >>
> > > >> Using Ceph 15.2.8 installed with cephadm. Trying to get RadosGW
> > > >> to
> > > work.
> > > >> I have managed to get the RadosGW working. I can manage it through
> > > >> a dashboard and use aws s3 client to create new buckets etc. When
> > > >> trying to use swift I get errors.
> > > >>
> > > >> Not sure how to continue to track the problem here. Any tips are
> > > >> welcome.
> > > >>
> > > >> Thank you very much,
> > > >> -Mika
> > > >>
> > > >> ------- What I have done and what are the results. Some data
> > > >> changed manually -------
> > > >> What I have done:
> > > >> At OpenStack Side:
> > > >> 1) openstack user create --domain default --password-prompt
> > swift
> > > >> 2) openstack role add --project service --user swift admin
> > > >> 3) openstack endpoint create --region RegionOne object-store
> > > >> public
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AUT
> > > H-5F-25-255C-28project-5Fid-255C-29s&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA
> > > &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U4
> > > 6oD9d1KMRwdpbF9VLg7eX4&s=-1FtdhjTcNA8jPSUoyoUfsPl5uqTqu4I_ThTOJNLjtg&e
> > > =
> > > >> 4) openstack endpoint create --region RegionOne object-store
> > > >> internal
> > > >>
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AUT
> > > H-5F-25-255C-28project-5Fid-255C-29s&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA
> > > &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U4
> > > 6oD9d1KMRwdpbF9VLg7eX4&s=-1FtdhjTcNA8jPSUoyoUfsPl5uqTqu4I_ThTOJNLjtg&e
> > > =
> > > >> 5) openstack endpoint create --region RegionOne object-store
> > > >> admin
> > > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1&;
> > > >> d=
> > > >> DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9K
> > > >> tt
> > > >> b6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=bm67b3lMVe
> > > >> LC
> > > >> 3sNvuyufFCe3AksJgfIgeI8SDorhHMU&e=
> > > >>
> > > >> At Ceph side:
> > > >> 1) ceph config set mgr rgw_keystone_api_version 3
> > > >> 2) ceph config set mgr rgw_keystone_url
> > > >>
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000&;
> > > d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kt
> > > tb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=lyXWyh-BXrik
> > > PWqWM3dcPW4ZofvjiAxnq-nXsjifnEw&e=
> > > >> 3) ceph config set mgr rgw_keystone_accepted_admin_roles admin
> > > >> 4) ceph config set mgr rgw_keystone_admin_user swift
> > > >> 5) ceph config set mgr rgw_keystone_admin_password swift_test
> > > >> 6) ceph config set mgr rgw_keystone_admin_domain default
> > > >> 7) ceph config set mgr rgw_keystone_admin_project service
> > > >> for project I have tested different projects e.g. service and
> > > >> admin
> > > >>
> > > >> Now when testing the API using swift client I get next:
> > > >> 1) swift post test3 --debug
> > > >>
> > > >> DEBUG:keystoneclient.auth.identity.v3.base:Making authentication
> > > >> request to
> > > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A50
> > > >> 00
> > > >> _v3_auth_tokens&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6
> > > >> KK
> > > >> a6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VL
> > > >> g7 eX4&s=-98qpMcc8sdRTdN7AwNPIyGsIK1GaFvi_SC5GtZGUpY&e=
> > > >> DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
> > > >> controller:5000
> > > >> DEBUG:urllib3.connectionpool:http://controller:5000 "POST
> > > >> /v3/auth/tokens HTTP/1.1" 201 7032
> > > >>
> > > >> . some openstack data here .
> > > >>
> > > >> DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
> > > >> ceph1:80
> > > >> DEBUG:urllib3.connectionpool:http://ceph1:80 "POST
> > > >> /swift/v1/AUTH_adsfasdfasdfasdfasdfasdf/test3 HTTP/1.1" 401 12
> > > >> INFO:swiftclient:REQ: curl -i
> > > >>
> > > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_
> > > >> AU
> > > >> TH-5Fadsfasdfasdfasdfasdfasdf_test3&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZt
> > > >> yA
> > > >> &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw
> > > >> 1U
> > > >> 46oD9d1KMRwdpbF9VLg7eX4&s=g1inMAENxiOpxc4L8FlmbLypegdcQwgH8drm6aoES
> > > >> Z0
> > > >> &e=
> > > >> -X POST -H
> > > >> "X-Auth-Token: <Token would be here>" -H "Content-Length: 0"
> > > >> INFO:swiftclient:RESP STATUS: 401 Unauthorized
> > > >>
> > > >> and finally I get
> > > >> Container POST failed:
> > > >>
> > > >> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_
> > > >> AU
> > > >> TH-5Fadsfasdfasdfasdfasdfasdf_test3&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZt
> > > >> yA
> > > >> &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw
> > > >> 1U
> > > >> 46oD9d1KMRwdpbF9VLg7eX4&s=g1inMAENxiOpxc4L8FlmbLypegdcQwgH8drm6aoES
> > > >> Z0
> > > >> &e=
> > > >> 401 Unauthorized
> > > >> b'AccessDenied'
> > > >> _______________________________________________
> > > >> ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send
> > > >> an email to ceph-users-leave@xxxxxxx
> > > >>
> > > >
> > > _______________________________________________
> > > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> > > email to ceph-users-leave@xxxxxxx
> > >
> > _______________________________________________
> > ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an
> > email to ceph-users-leave@xxxxxxx
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
