Re: s3cmd not working with luminous radosgw

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello,

>> Does anyone have tested s3cmd or other tools to manage ACL on luminous radosGW ?
> 
> Don't know about ACL, but s3cmd for other things works for me.  Version 1.6.1

Finally, I found out what happened, I had 2 issues. One, on s3cmd config file, radosgw with luminous does not support signature v2 anymore, only
v4 is supported, I had to add this to my .s3cfg file :

> signature_v2 = False

The second was in the rgw section into ceph.conf file. The line "rgw dns name" was missing. I have deployed my cluster with ceph-ansible and it
seems that I need a new option in the all.ym file :

> radosgw_resolve_cname: true # enable for radosgw to resolve DNS CNAME based bucket names

I have added it manually and now it works (ansible-playbook didn't add it, I must figure out why).

Thanks for you help

Best regards,

Yoann Moulin
PS : with config lines, it's better :)

>>> I have a fresh luminous cluster in test and I made a copy of a bucket (4TB 1.5M files) with rclone, I'm able to list/copy files with rclone but
>>> s3cmd does not work at all, it is just able to give the bucket list but I can't list files neither update ACL.
>>>
>>> does anyone already test this ?
>>>
>>> root@iccluster012:~# rclone --version
>>> rclone v1.37
>>>
>>> root@iccluster012:~# s3cmd --version
>>> s3cmd version 2.0.0
>>>
>>>
>>> ### rclone ls files ###
>>>
>>> root@iccluster012:~# rclone ls testadmin:image-net/LICENSE
>>>      1589 LICENSE
>>> root@iccluster012:~#
>>>
>>> nginx (as revers proxy) log :
>>>
>>>> 10.90.37.13 - - [15/Sep/2017:10:30:02 +0200] "HEAD /image-net/LICENSE HTTP/1.1" 200 0 "-" "rclone/v1.37"
>>>> 10.90.37.13 - - [15/Sep/2017:10:30:02 +0200] "GET /image-net?delimiter=%2F&max-keys=1024&prefix= HTTP/1.1" 200 779 "-" "rclone/v1.37"
>>>
>>> rgw logs :
>>>
>>>> 2017-09-15 10:30:02.620266 7ff1f58f7700  1 ====== starting new request req=0x7ff1f58f11f0 =====
>>>> 2017-09-15 10:30:02.622245 7ff1f58f7700  1 ====== req done req=0x7ff1f58f11f0 op status=0 http_status=200 ======
>>>> 2017-09-15 10:30:02.622324 7ff1f58f7700  1 civetweb: 0x56061584b000: 127.0.0.1 - - [15/Sep/2017:10:30:02 +0200] "HEAD /image-net/LICENSE HTTP/1.0" 1 0 - rclone/v1.37
>>>> 2017-09-15 10:30:02.623361 7ff1f50f6700  1 ====== starting new request req=0x7ff1f50f01f0 =====
>>>> 2017-09-15 10:30:02.689632 7ff1f50f6700  1 ====== req done req=0x7ff1f50f01f0 op status=0 http_status=200 ======
>>>> 2017-09-15 10:30:02.689719 7ff1f50f6700  1 civetweb: 0x560615850000: 127.0.0.1 - - [15/Sep/2017:10:30:02 +0200] "GET /image-net?delimiter=%2F&max-keys=1024&prefix= HTTP/1.0" 1 0 - rclone/v1.37
>>>
>>>
>>>
>>> ### s3cmds ls files ###
>>>
>>> root@iccluster012:~# s3cmd -v -c ~/.s3cfg-test-rgwadmin ls s3://image-net/LICENSE
>>> root@iccluster012:~#
>>>
>>> nginx (as revers proxy) log :
>>>
>>>> 10.90.37.13 - - [15/Sep/2017:10:30:04 +0200] "GET http://test.iccluster.epfl.ch/image-net/?location HTTP/1.1" 200 127 "-" "-"
>>>> 10.90.37.13 - - [15/Sep/2017:10:30:04 +0200] "GET http://image-net.test.iccluster.epfl.ch/?delimiter=%2F&prefix=LICENSE HTTP/1.1" 200 318 "-" "-"
>>>
>>> rgw logs :
>>>
>>>> 2017-09-15 10:30:04.295355 7ff1f48f5700  1 ====== starting new request req=0x7ff1f48ef1f0 =====
>>>> 2017-09-15 10:30:04.295913 7ff1f48f5700  1 ====== req done req=0x7ff1f48ef1f0 op status=0 http_status=200 ======
>>>> 2017-09-15 10:30:04.295977 7ff1f48f5700  1 civetweb: 0x560615855000: 127.0.0.1 - - [15/Sep/2017:10:30:04 +0200] "GET /image-net/?location HTTP/1.0" 1 0 - -
>>>> 2017-09-15 10:30:04.299303 7ff1f40f4700  1 ====== starting new request req=0x7ff1f40ee1f0 =====
>>>> 2017-09-15 10:30:04.300993 7ff1f40f4700  1 ====== req done req=0x7ff1f40ee1f0 op status=0 http_status=200 ======
>>>> 2017-09-15 10:30:04.301070 7ff1f40f4700  1 civetweb: 0x56061585a000: 127.0.0.1 - - [15/Sep/2017:10:30:04 +0200] "GET /?delimiter=%2F&prefix=LICENSE HTTP/1.0" 1 0 - 
>>>
>>>
>>>
>>> ### s3cmd : list bucket ###
>>>
>>> root@iccluster012:~# s3cmd -v -c ~/.s3cfg-test-rgwadmin ls s3://
>>> 2017-08-28 12:27  s3://image-net
>>> root@iccluster012:~#
>>>
>>> nginx (as revers proxy) log :
>>>
>>>> ==> nginx/access.log <==
>>>> 10.90.37.13 - - [15/Sep/2017:10:36:10 +0200] "GET http://test.iccluster.epfl.ch/ HTTP/1.1" 200 318 "-" "-"
>>>
>>> rgw logs :
>>>
>>>> 2017-09-15 10:36:10.645354 7ff1f38f3700  1 ====== starting new request req=0x7ff1f38ed1f0 =====
>>>> 2017-09-15 10:36:10.647419 7ff1f38f3700  1 ====== req done req=0x7ff1f38ed1f0 op status=0 http_status=200 ======
>>>> 2017-09-15 10:36:10.647488 7ff1f38f3700  1 civetweb: 0x56061585f000: 127.0.0.1 - - [15/Sep/2017:10:36:10 +0200] "GET / HTTP/1.0" 1 0 - -
>>>
>>>
>>>
>>> ### rclone : list bucket ###
>>>
>>>
>>> root@iccluster012:~# rclone lsd testadmin:
>>>           -1 2017-08-28 12:27:33        -1 image-net
>>> root@iccluster012:~#
>>>
>>> nginx (as revers proxy) log :
>>>
>>>> ==> nginx/access.log <==
>>>> 10.90.37.13 - - [15/Sep/2017:10:37:53 +0200] "GET / HTTP/1.1" 200 318 "-" "rclone/v1.37"
>>>
>>> rgw logs :
>>>
>>>> ==> ceph/luminous-rgw-iccluster015.log <==
>>>> 2017-09-15 10:37:53.005424 7ff1f28f1700  1 ====== starting new request req=0x7ff1f28eb1f0 =====
>>>> 2017-09-15 10:37:53.007192 7ff1f28f1700  1 ====== req done req=0x7ff1f28eb1f0 op status=0 http_status=200 ======
>>>> 2017-09-15 10:37:53.007282 7ff1f28f1700  1 civetweb: 0x56061586e000: 127.0.0.1 - - [15/Sep/2017:10:37:53 +0200] "GET / HTTP/1.0" 1 0 - rclone/v1.37
>>
>>
>> -- 
>> Yoann Moulin
>> EPFL IC-IT
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


-- 
Yoann Moulin
EPFL IC-IT
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux