Vivek, The problem is swift client is only downloading a chunk of object not the whole object so the etag mismatch. Could you paste the value of 'rgw_max_chunk_size'. Please be sure you set this to a sane value(<4MB, atleast for Giant release this works below this value). On Tue, Dec 16, 2014 at 12:26 PM, Vivek Varghese Cherian <vivekcherian@xxxxxxxxx> wrote: > Hi, > > I am integrating ceph firefly radosgw with openstack juno keystone, the > operating system used on the ceph > nodes and on the openstack node is Ubuntu 14.04. > > I am able to create containers and upload files using the swift client to > ceph. > > But when I try to download files, I am getting the following error, > > ppmuser@ppm-dc-c3sv3-ju:~$ swift --verbose --debug -V 1 -A > http://10.x.x.126/auth -U swift:swift -K <Key_here> download demo-container1 > file_12345.txt > > INFO:urllib3.connectionpool:Starting new HTTP connection (1): 10.x.x.126 > DEBUG:urllib3.connectionpool:Setting read timeout to None > DEBUG:urllib3.connectionpool:"GET /auth HTTP/1.1" 204 0 > DEBUG:swiftclient:REQ: curl -i http://10.x.x.126/auth -X GET > DEBUG:swiftclient:RESP STATUS: 204 No Content > DEBUG:swiftclient:RESP HEADERS: [('x-auth-token', > 'AUTH_rgwtk0b00000073776966743a73776966740484a024256146dc5719915468deb431889a0ff3c6514ac8ae2388abdfbaac6d262c3e3e'), > ('x-storage-token', > 'AUTH_rgwtk0b00000073776966743a73776966740484a024256146dc5719915468deb431889a0ff3c6514ac8ae2388abdfbaac6d262c3e3e'), > ('date', 'Tue, 16 Dec 2014 05:49:11 GMT'), ('x-storage-url', > 'http://10.x.x.126/swift/v1'), ('server', 'Apache/2.4.7 (Ubuntu)'), > ('content-type', 'application/json')] > INFO:urllib3.connectionpool:Starting new HTTP connection (1): 10.x.x.126 > DEBUG:urllib3.connectionpool:Setting read timeout to None > DEBUG:urllib3.connectionpool:"GET /swift/v1/demo-container1/file_12345.txt > HTTP/1.1" 200 14 > DEBUG:swiftclient:REQ: curl -i > http://10.x.x.126/swift/v1/demo-container1/file_12345.txt -X GET -H > "X-Auth-Token: > AUTH_rgwtk0b00000073776966743a73776966740484a024256146dc5719915468deb431889a0ff3c6514ac8ae2388abdfbaac6d262c3e3e" > DEBUG:swiftclient:RESP STATUS: 200 OK > DEBUG:swiftclient:RESP HEADERS: [('content-length', '14'), ('accept-ranges', > 'bytes'), ('server', 'Apache/2.4.7 (Ubuntu)'), ('last-modified', 'Mon, 15 > Dec 2014 13:35:37 GMT'), ('etag', '94b371af646d4c53d8f3f8ff7be74dbb'), > ('date', 'Tue, 16 Dec 2014 05:49:11 GMT'), ('x-object-meta-mtime', > '1418650375.550963')] > > Error downloading object 'demo-container1/file_12345.txt': 'Error > downloading file_12345.txt: read_length != content_length, 0 != 14' > > ppmuser@ppm-dc-c3sv3-ju:~ > > > The apache2 error and access logs on the radosgw ceph node is as follows, > > root@ppm-c240-ceph3:~# tail -f /var/log/apache2/error.log > > [Tue Dec 16 01:10:20.735061 2014] [fastcgi:error] [pid 31426:tid > 140231210624768] [client 10.x.x.175:55902] FastCGI: comm with server > "/var/www/s3gw.fcgi" aborted: idle timeout (30 sec) > [Tue Dec 16 01:10:20.735134 2014] [fastcgi:error] [pid 31426:tid > 140231210624768] [client 10.x.x.175:55902] FastCGI: incomplete headers (0 > bytes) received from server "/var/www/s3gw.fcgi" > [Tue Dec 16 01:10:20.735161 2014] [http:error] [pid 31426:tid > 140231210624768] [client 10.x.x.175:55902] not unsetting Content-Length on > HEAD response (rgw changes)\n > [Tue Dec 16 01:10:20.857215 2014] [:warn] [pid 31426:tid 140231202232064] > FastCGI: 10.x.x.175 HEAD > http://ppm-c240-ceph3.cisco.com/swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93 > auth > [Tue Dec 16 01:10:50.887381 2014] [fastcgi:error] [pid 31426:tid > 140231202232064] [client 10.x.x.175:55910] FastCGI: comm with server > "/var/www/s3gw.fcgi" aborted: idle timeout (30 sec) > [Tue Dec 16 01:10:50.887454 2014] [fastcgi:error] [pid 31426:tid > 140231202232064] [client 10.x.x.175:55910] FastCGI: incomplete headers (0 > bytes) received from server "/var/www/s3gw.fcgi" > [Tue Dec 16 01:10:50.887481 2014] [http:error] [pid 31426:tid > 140231202232064] [client 10.x.x.175:55910] not unsetting Content-Length on > HEAD response (rgw changes)\n > [Tue Dec 16 01:10:50.898281 2014] [:warn] [pid 31426:tid 140231193839360] > FastCGI: 10.x.x.175 GET > http://ppm-c240-ceph3.cisco.com/swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93 > auth > [Tue Dec 16 01:11:20.928476 2014] [fastcgi:error] [pid 31426:tid > 140231193839360] [client 10.x.x.175:55911] FastCGI: comm with server > "/var/www/s3gw.fcgi" aborted: idle timeout (30 sec) > [Tue Dec 16 01:11:20.928556 2014] [fastcgi:error] [pid 31426:tid > 140231193839360] [client 10.x.x.175:55911] FastCGI: incomplete headers (0 > bytes) received from server "/var/www/s3gw.fcgi" > > root@ppm-c240-ceph3:~# tail -f /var/log/apache2/access.log > > 10.81.83.175 - - [16/Dec/2014:01:00:20 -0500] "HEAD > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93 HTTP/1.1" 500 189 "-" > "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:00:50 -0500] "GET > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93?format=json HTTP/1.1" 500 > 719 "-" "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:08:22 -0500] "GET /auth HTTP/1.1" 204 431 > "-" "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:08:22 -0500] "GET > /swift/v1/demo-container1?format=json HTTP/1.1" 200 411 "-" > "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:08:22 -0500] "GET > /swift/v1/demo-container1?format=json&marker=file_12345.txt HTTP/1.1" 200 > 174 "-" "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:08:50 -0500] "HEAD > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93 HTTP/1.1" 500 189 "-" > "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:09:20 -0500] "GET > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93?format=json HTTP/1.1" 500 > 719 "-" "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:09:50 -0500] "HEAD > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93 HTTP/1.1" 500 189 "-" > "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:10:20 -0500] "HEAD > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93 HTTP/1.1" 500 189 "-" > "python-swiftclient-2.3.0" > 10.81.83.175 - - [16/Dec/2014:01:10:50 -0500] "GET > /swift/v1/AUTH_25bb0caaff834efdafa1c1fcbb6aaf93?format=json HTTP/1.1" 500 > 719 "-" "python-swiftclient-2.3.0" > > > The radosgw logs is as follows, > > root@ppm-c240-ceph3:/var/log/radosgw# tail -f ceph-client.admin > > 2014-12-09 11:39:19.862854 7fdbfa1257c0 0 librados: client.admin > initialization error (2) No such file or directory > 2014-12-09 11:39:19.863429 7fdbfa1257c0 -1 Couldn't init storage provider > (RADOS) > 2014-12-09 11:41:25.894934 7ffa8fe087c0 0 ceph version 0.80.7 > (6c0127fcb58008793d3c8b62d925bc91963672a3), process radosgw, pid 5843 > 2014-12-09 11:41:25.922673 7ffa8fe087c0 -1 monclient(hunting): ERROR: > missing keyring, cannot use cephx for authentication > 2014-12-09 11:41:25.922807 7ffa8fe087c0 0 librados: client.admin > initialization error (2) No such file or directory > 2014-12-09 11:41:25.924901 7ffa8fe087c0 -1 Couldn't init storage provider > (RADOS) > 2014-12-09 11:42:12.110334 7f145f77d7c0 0 ceph version 0.80.7 > (6c0127fcb58008793d3c8b62d925bc91963672a3), process radosgw, pid 5862 > 2014-12-09 11:42:12.139115 7f145f77d7c0 -1 monclient(hunting): ERROR: > missing keyring, cannot use cephx for authentication > 2014-12-09 11:42:12.139215 7f145f77d7c0 0 librados: client.admin > initialization error (2) No such file or directory > 2014-12-09 11:42:12.141559 7f145f77d7c0 -1 Couldn't init storage provider > (RADOS) > > > Any pointers as to what could be the cause of the error in downloading files > from the ceph cluster using the openstack juno swift client would be highly > appreciated. > > > Regards, > --- > Vivek Varghese Cherian > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- -Pushpesh _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |