On Tue, Apr 1, 2014 at 1:19 AM, Benedikt Fraunhofer
<given.to.lists.ceph-users.ceph.com.toasta.001 at traced.net> wrote:
> Hello everyone,
>
> I can't download anything that's been uploaded as a multipart-upload.
> I'm on 0.78 (f6c746c314d7b87b8419b6e584c94bfe4511dbd4)
> on a non-ec-pool.
>
> The upload is acknowledged as beeing ok
>
> 2014-03-31 14:56:56.722727 7f4080ff9700 2 req 8:0.023285:s3:POST
> /file1:complete_multipart:http status=200
> 2014-03-31 14:56:56.722736 7f4080ff9700 1 ====== req done
> req=0x7f407800b180 http_status=200 ======
>
>
> , but on download, some error is translated to 404.
>
> 2014-03-31 14:56:57.885610 7f408cef8700 20 rados->get_obj_iterate_cb
> oid=default.71025.1__multipart_file1.2/F9cdVNlUTdfYq7M9cTuQjYwbsS41Y5j.16
> obj-ofs=61865984 read_ofs=0 len=4194304
> 2014-03-31 14:56:57.885635 7f408cef8700 20 rados->aio_operate r=0 bl.length=0
> 2014-03-31 14:56:57.885639 7f408cef8700 20
> RGWObjManifest::operator++(): rule->part_size=4194304 rules.size()=2
> 2014-03-31 14:56:57.885640 7f408cef8700 20
> RGWObjManifest::operator++(): stripe_ofs=66060288 part_ofs=61865984
> rule->part_size=4194304
> 2014-03-31 14:56:57.885642 7f408cef8700 0
> RGWObjManifest::operator++(): result: ofs=66060288 stripe_ofs=66060288
> part_ofs=66060288 rule->part_size=4194304
> 2014-03-31 14:56:57.885737 7f40f27fc700 20 get_obj_aio_completion_cb:
> io completion ofs=49283072 len=4194304
> 2014-03-31 14:56:57.885746 7f408cef8700 20 rados->get_obj_iterate_cb
> oid=default.71025.1__multipart_file1.2/F9cdVNlUTdfYq7M9cTuQjYwbsS41Y5j.17
> obj-ofs=66060288 read_ofs=0 len=1048576
> 2014-03-31 14:56:57.885770 7f408cef8700 20 rados->aio_operate r=0 bl.length=0
> 2014-03-31 14:56:57.885772 7f408cef8700 20
> RGWObjManifest::operator++(): rule->part_size=4194304 rules.size()=2
> 2014-03-31 14:56:57.885773 7f408cef8700 20
> RGWObjManifest::operator++(): stripe_ofs=70254592 part_ofs=66060288
> rule->part_size=4194304
> 2014-03-31 14:56:57.885774 7f408cef8700 0
> RGWObjManifest::operator++(): result: ofs=67108864 stripe_ofs=67108864
> part_ofs=70254592 rule->part_size=4194304
> 2014-03-31 14:56:57.885786 7f408cef8700 10 get_obj_iterate() r=-2,
> canceling all io
> 2014-03-31 14:56:57.885787 7f408cef8700 20 get_obj_data::cancel_all_io()
> 2014-03-31 14:56:57.885822 7f408cef8700 2 req 9:0.081262:s3:GET
> /file1:get_obj:http status=404
> 2014-03-31 14:56:57.885837 7f408cef8700 1 ====== req done
> req=0x7f4078011510 http_status=404 ======
> 2014-03-31 14:56:57.885839 7f408cef8700 20 process_request() returned -2
> 2014-03-31 14:56:57.885903 7f40f27fc700 20 get_obj_aio_completion_cb:
> io completion ofs=53477376 len=4194304
> 2014-03-31 14:56:57.885958 7f40f27fc700 20 get_obj_aio_completion_cb:
> io completion ofs=57671680 len=4194304
> 2014-03-31 14:56:57.886566 7f40f27fc700 20 get_obj_aio_completion_cb:
> io completion ofs=66060288 len=1048576
> 2014-03-31 14:56:57.886636 7f40f27fc700 20 get_obj_aio_completion_cb:
> io completion ofs=61865984 len=4194304
>
> everything is working fine, if i disable the multipart-upload;
>
> here's the script I used to test it:
> ---------------------------------------
> #! /bin/bash
>
> set -x
>
> FN=file1
> SIZE=64
> S3CMD=s3cmd
> S3CMD=/home/bf/s3cmd-1.5.0-beta1/s3cmd
> MULTIPART="--disable-multipart"
> MULTIPART=""
>
> BUCK="s3://$(uuidgen)/"
> dd if=/dev/zero of=$FN bs=4M count=$(($SIZE/4)) &
> $S3CMD mb $BUCK
> wait
> $S3CMD put $MULTIPART $FN $BUCK
> sleep 1
> $S3CMD get --force ${BUCK}${FN} file1.1
> ---------------------------------------
>
> Is this s3cmd's fault or is radosgw doing something wrong?
>
It could be the gateway's fault, might be related to the new manifest
that went in before 0.78. I'll need more logs though, can you
reproduce with 'debug ms = 1', and 'debug rgw = 20', and provide a log
for all the upload and for the download?
Thanks,
Yehuda
