Re: Ceph RadosGW & OpenStack swift problem

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



The user rgwswift only for radosgw config ( do not use it in your file openrc ) use swift user instead . Also , keep the default project to admin ( os_project_name ) .

Rgds

De : Mika Saari <mika.saari@xxxxxxxxx>
Envoyé : jeudi 7 janvier 2021 12:45
À : Wissem MIMOUNA <wissem.mimouna@xxxxxxxxxxxxxxxx>
Cc : ceph-users@xxxxxxx
Objet : Re:  Re: Ceph RadosGW & OpenStack swift problem

Hi,

  Adding below what I tested. Do you see from this what I am doing wrong?

  Thank you very much,
    -Mika

--clip clip--
OPENSTACK SIDE:
[root@controller ~]# openstack user create --domain default --password-prompt rgwswift
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field               | Value                            |
+---------------------+----------------------------------+
| domain_id           | default                          |
| enabled             | True                             |
| id                  | 85a86ec5c0264302b0471fd147042e0b |
| name                | rgwswift                         |
| options             | {}                               |
| password_expires_at | None                             |
+---------------------+----------------------------------+
[root@controller ~]# openstack role add --project service --user rgwswift admin

CEPH SIDE:
[root@ceph1 ~]# ceph config set mgr rgw_keystone_accepted_roles "admin, _member_, Member, member, creator"
[root@ceph1 ~]# ceph config set mgr rgw_keystone_admin_user rgwswift
[root@ceph1 ~]# ceph config set mgr rgw_keystone_admin_project service
[root@ceph1 ~]# ceph orch restart rgw.default.ou
restart rgw.default.ou.ceph1.gxblht from host 'ceph1'


CLIENT SIDE:
$ . swift-openrc
Where swift-openrc is like this:
  export OS_PROJECT_DOMAIN_NAME=Default
  export OS_USER_DOMAIN_NAME=Default
  export OS_PROJECT_NAME=service
  export OS_USERNAME=rgwswift
  export OS_PASSWORD=rgwswiftpw
  export OS_AUTH_URL=http://controller:5000/v3<https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000_v3&d=DwMFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=oc3C1TP2mMYCukAjjobWV7SPwto-zVeUvBG-JgRS3SI&s=xYsKH127snVkstwVzGM-ha6td0BdcY5-XQxutKOxNto&e=>
  export OS_IDENTITY_API_VERSION=3
  export OS_IMAGE_API_VERSION=2
$ swift stat --debug

Problem like earlier.
   First the swift client authenticates to the keystone and that works.
   Second it tries to contact radosgw, and that gives 401.

 Checked the rgw_process.cc : process_request and seems that there is no more debug information in the source. I assume the row int ret = client_io->init(g_ceph_context); gives < 0 which causes the process_request to return out with abort_early.

On Thu, Jan 7, 2021 at 1:16 PM Wissem MIMOUNA <wissem.mimouna@xxxxxxxxxxxxxxxx<mailto:wissem.mimouna@xxxxxxxxxxxxxxxx>> wrote:
Hi,

The radosgw should have a dedicated user (different from you swift user) for authentifiation with keystone ( openstack) in the project "service" and you should also add the role "_member_" in the rgw_keystone_accepted_roles.

Regards

-----Message d'origine-----
De : Mika Saari <mika.saari@xxxxxxxxx<mailto:mika.saari@xxxxxxxxx>>
Envoyé : jeudi 7 janvier 2021 11:35
À : ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx>
Objet :  Re: Ceph RadosGW & OpenStack swift problem

Hi,

  I have added debug_rgw 20 to configuration. When checking docker logs -f <radosgw container id> I get this error for my radowgw request (swift post
test3  --debug)

  Would there be a way to get more debug information from radosgw to solve this 401 problem ?

  Thanks a lot,
    -Mika

  --- clip clip ----
debug 2021-01-07T10:32:42.269+0000 7f1ae111b700  1 ====== starting new request req=0x7f1b5b32a6b0 ===== debug 2021-01-07T10:32:42.269+0000 7f1ae111b700  1 op->ERRORHANDLER:
err_no=-1 new_err_no=-1
debug 2021-01-07T10:32:42.269+0000 7f1ae111b700  1 ====== req done
req=0x7f1b5b32a6b0 op status=0 http_status=401 latency=0s ====== debug 2021-01-07T10:32:42.269+0000 7f1ae111b700  1 beast: 0x7f1b5b32a6b0:
10.0.2.10 - - [2021-01-07T10:32:42.269372+0000] "POST
/swift/v1/AUTH_50f0ce372a4a4ed6a41126852358f097/test3 HTTP/1.1" 401 12 - "python-swiftclient-3.9.0" -
  --- clip clip ----


On Tue, Jan 5, 2021 at 8:00 PM Mika Saari <mika.saari@xxxxxxxxx<mailto:mika.saari@xxxxxxxxx>> wrote:

> Hi,
>
>   I am using indeed OpenStack Ussuri release. I changed the "gw swift
> account in url = true" directly with ceph config set ... command. Also
> checked that rgw_keystone_accepted_roles is correctly set and not the
> admin one. Also tested disabling rgw_keystone_verify_ssl.
>
>   Should radosgw communicate with keystone somehow? I can not see my
> ceph-cluster requesting anything from keystone through any interface
> (tcpdump checked this one). I have tested restarting the radosgw with
> command "ceph orch restart rgw.default.ou" and seems that it brings
> the container down and up. Not sure though it is enough to bring the
> settings in use.q
>
>   Current status is:
>     1) swift command seems to be able to authenticate with keystone at
> the very beginning, this is done in the client side.
>     2) swift command makes a request to radosgw and gets 401
>        INFO:swiftclient:REQ: curl -i <radosgw url
> here>/swift/v1/AUTH_<some id here>/test3 -X POST -H "X-Auth-Token:
> here><token " -H "Content-Length: 0"
>       INFO:swiftclient:RESP STATUS: 401 Unauthorized
>
>   Thanks a lot again,
>      -Mika
>
> On Tue, Jan 5, 2021 at 11:19 AM Wissem MIMOUNA <
> wissem.mimouna@xxxxxxxxxxxxxxxx<mailto:wissem.mimouna@xxxxxxxxxxxxxxxx>> wrote:
>
>> Hi,
>>
>> Which version of OpenStack do you have ? I guess , since Usurri ( or
>> may be even before ) swift authentification through keystone require
>> the account in url . You have to add this option in
>> "/etc/ceph/ceph.conf" , section rgw "rgw swift account in url = true" or do it via setting directly
>> . Also , I noticed you did  this ==>     3) ceph config set mgr
>> rgw_keystone_accepted_admin_roles xxxx ||  I think , you should use
>> the option "rgw keystone accepted roles xxxx" instead.
>>
>> Regards
>>
>> -----Message d'origine-----
>> De : Mika Saari <mika.saari@xxxxxxxxx<mailto:mika.saari@xxxxxxxxx>> Envoyé : mardi 5 janvier 2021
>> 10:03 À : ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> Objet :  Ceph RadosGW &
>> OpenStack swift problem
>>
>> Hi,
>>
>>   Using Ceph 15.2.8 installed with cephadm. Trying to get RadosGW to work.
>> I have managed to get the RadosGW working. I can manage it through a
>> dashboard and use aws s3 client to create new buckets etc. When
>> trying to use swift I get errors.
>>
>>   Not sure how to continue to track the problem here. Any tips are
>> welcome.
>>
>>     Thank you very much,
>>       -Mika
>>
>> ------- What I have done and what are the results. Some data changed
>> manually  -------
>>   What I have done:
>>     At OpenStack Side:
>>       1) openstack user create --domain default --password-prompt swift
>>       2) openstack role add --project service --user swift admin
>>       3) openstack endpoint create --region RegionOne object-store
>> public https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AUTH-5F-25-255C-28project-5Fid-255C-29s&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=-1FtdhjTcNA8jPSUoyoUfsPl5uqTqu4I_ThTOJNLjtg&e=
>>       4) openstack endpoint create --region RegionOne object-store
>> internal
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AUTH-5F-25-255C-28project-5Fid-255C-29s&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=-1FtdhjTcNA8jPSUoyoUfsPl5uqTqu4I_ThTOJNLjtg&e=
>>       5) openstack endpoint create --region RegionOne object-store
>> admin
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1&d=
>> DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Ktt
>> b6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=bm67b3lMVeLC
>> 3sNvuyufFCe3AksJgfIgeI8SDorhHMU&e=
>>
>>   At Ceph side:
>>     1) ceph config set mgr rgw_keystone_api_version 3
>>     2) ceph config set mgr rgw_keystone_url
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7eX4&s=lyXWyh-BXrikPWqWM3dcPW4ZofvjiAxnq-nXsjifnEw&e=
>>     3) ceph config set mgr rgw_keystone_accepted_admin_roles admin
>>     4) ceph config set mgr rgw_keystone_admin_user swift
>>     5) ceph config set mgr rgw_keystone_admin_password swift_test
>>     6) ceph config set mgr rgw_keystone_admin_domain default
>>     7) ceph config set mgr rgw_keystone_admin_project service
>>       for project I have tested different projects e.g. service and
>> admin
>>
>>   Now when testing the API using swift client I get next:
>>     1) swift post test3 --debug
>>
>> DEBUG:keystoneclient.auth.identity.v3.base:Making authentication
>> request to
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000
>> _v3_auth_tokens&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KK
>> a6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U46oD9d1KMRwdpbF9VLg7
>> eX4&s=-98qpMcc8sdRTdN7AwNPIyGsIK1GaFvi_SC5GtZGUpY&e=
>> DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
>> controller:5000
>> DEBUG:urllib3.connectionpool:http://controller:5000<https://urldefense.proofpoint.com/v2/url?u=http-3A__controller-3A5000&d=DwMFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=oc3C1TP2mMYCukAjjobWV7SPwto-zVeUvBG-JgRS3SI&s=D3W7JtLCq7AbYLGXj1Tm-RTLE4w95svqucaeAg87aeE&e=> "POST
>> /v3/auth/tokens HTTP/1.1" 201 7032
>>
>> . some openstack data here .
>>
>> DEBUG:urllib3.connectionpool:Starting new HTTP connection (1):
>> ceph1:80
>> DEBUG:urllib3.connectionpool:http://ceph1:80<https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1-3A80&d=DwMFaQ&c=1tDFxPZjcWEmlmmx4CZtyA&r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=oc3C1TP2mMYCukAjjobWV7SPwto-zVeUvBG-JgRS3SI&s=vfsbb-sSKs_VnT0vrT_MZRnADOCDvRh0208AgDEvLeo&e=> "POST
>> /swift/v1/AUTH_adsfasdfasdfasdfasdfasdf/test3 HTTP/1.1" 401 12
>> INFO:swiftclient:REQ: curl -i
>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AU
>> TH-5Fadsfasdfasdfasdfasdfasdf_test3&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA
>> &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U
>> 46oD9d1KMRwdpbF9VLg7eX4&s=g1inMAENxiOpxc4L8FlmbLypegdcQwgH8drm6aoESZ0
>> &e=
>> -X POST -H
>> "X-Auth-Token: <Token would be here>" -H "Content-Length: 0"
>> INFO:swiftclient:RESP STATUS: 401 Unauthorized
>>
>> and finally I get
>> Container POST failed:
>>
>> https://urldefense.proofpoint.com/v2/url?u=http-3A__ceph1_swift_v1_AU
>> TH-5Fadsfasdfasdfasdfasdfasdf_test3&d=DwICAg&c=1tDFxPZjcWEmlmmx4CZtyA
>> &r=h1fIFv3Ydv-kdH6KKa6lmB20LbjUiXP9Kttb6tTs__E&m=EmlYLMTNHaWmSJrApw1U
>> 46oD9d1KMRwdpbF9VLg7eX4&s=g1inMAENxiOpxc4L8FlmbLypegdcQwgH8drm6aoESZ0
>> &e=
>> 401 Unauthorized
>>   b'AccessDenied'
>> _______________________________________________
>> ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> To unsubscribe send an
>> email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx>
>>
>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> To unsubscribe send an email to ceph-users-leave@xxxxxxx<mailto:ceph-users-leave@xxxxxxx>
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux