Re: AWS SDK and MultiPart Problem

[Georgios Dimitrakakis <giorgis@xxxxxxxxxxxx>]

Dear Yehuda,

I have installed the patched version as you can see:

$ radosgw --version
ceph version 0.80.7-1-gbd43759 
(bd43759f6e76fa827e2534fa4e61547779ee10a5)

$ ceph --version
ceph version 0.80.7-1-gbd43759 
(bd43759f6e76fa827e2534fa4e61547779ee10a5)

$ sudo yum info ceph-radosgw
Installed Packages
Name        : ceph-radosgw
Arch        : x86_64
Version     : 0.80.7
Release     : 1.gbd43759.el6
Size        : 3.8 M
Repo        : installed
From repo   : ceph-source
Summary     : Rados REST gateway
URL         : http://ceph.com/
License     : GPL-2.0
Description : radosgw is an S3 HTTP REST gateway for the RADOS object 
store. It is
            : implemented as a FastCGI module using libfcgi, and can be 
used in
            : conjunction with any FastCGI capable web server.


Unfortunately the problem on the multipart upload with aws-sdk still 
remains the same!


Here is a part of the apache log:


"PUT 
/clients-space/test/iip7.dmg?partNumber=3&uploadId=2%2F9rEUmdFcuW66VJfeH3_jbqqUz0jKvrO 
HTTP/1.1" 403 78 "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"

"PUT 
/clients-space/test/iip7.dmg?partNumber=1&uploadId=2%2F9rEUmdFcuW66VJfeH3_jbqqUz0jKvrO 
HTTP/1.1" 403 78 "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"

"PUT 
/clients-space/test/iip7.dmg?partNumber=2&uploadId=2%2F9rEUmdFcuW66VJfeH3_jbqqUz0jKvrO 
HTTP/1.1" 403 78 "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"



Directly modification of the binary so that the "2%2F" be changed to 
"2-" results in success and here is the log:


"PUT 
/clients-space/test/iip7.dmg?partNumber=1&uploadId=2-R6bxv4TM2Brxn-w9aHOcbb8OSJ3-Vh2 
HTTP/1.1" 200 - "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"

"PUT 
/clients-space/test/iip7.dmg?partNumber=2&uploadId=2-R6bxv4TM2Brxn-w9aHOcbb8OSJ3-Vh2 
HTTP/1.1" 200 - "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"

"PUT 
/clients-space/test/iip7.dmg?partNumber=4&uploadId=2-R6bxv4TM2Brxn-w9aHOcbb8OSJ3-Vh2 
HTTP/1.1" 200 - "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"

"POST 
/clients-space/test/iip7.dmg?uploadId=2-R6bxv4TM2Brxn-w9aHOcbb8OSJ3-Vh2 
HTTP/1.1" 200 302 "-" "aws-sdk-nodejs/2.1.0 darwin/v0.10.33"




Can you think of something else??


Best regards,


George




> OK! I will give it some time and will try again later!
>
> Thanks a lot for your help!
>
> Warmest regards,
>
> George
>
>
> > The branch I pushed earlier was based off recent development branch. 
> > I
> > just pushed one based off firefly (wip-10271-firefly). It will
> > probably take a bit to build.
> >
> > Yehuda
> >
> > On Thu, Dec 11, 2014 at 12:03 PM, Georgios Dimitrakakis
> > <giorgis@xxxxxxxxxxxx> wrote:
> > > Hi again!
> > >
> > > I have installed and enabled the development branch repositories as
> > > described here:
> > >
> > >
> > > http://ceph.com/docs/master/install/get-packages/#add-ceph-development
> > >
> > > and when I try to update the ceph-radosgw package I get the 
> > > following:
> > >
> > > Installed Packages
> > > Name        : ceph-radosgw
> > > Arch        : x86_64
> > > Version     : 0.80.7
> > > Release     : 0.el6
> > > Size        : 3.8 M
> > > Repo        : installed
> > > From repo   : Ceph
> > > Summary     : Rados REST gateway
> > > URL         : http://ceph.com/
> > > License     : GPL-2.0
> > > Description : radosgw is an S3 HTTP REST gateway for the RADOS 
> > > object store.
> > > It is
> > >             : implemented as a FastCGI module using libfcgi, and 
> > > can be used
> > > in
> > >             : conjunction with any FastCGI capable web server.
> > >
> > > Available Packages
> > > Name        : ceph-radosgw
> > > Arch        : x86_64
> > > Epoch       : 1
> > > Version     : 0.80.5
> > > Release     : 9.el6
> > > Size        : 1.3 M
> > > Repo        : epel
> > > Summary     : Rados REST gateway
> > > URL         : http://ceph.com/
> > > License     : GPL-2.0
> > > Description : radosgw is an S3 HTTP REST gateway for the RADOS 
> > > object store.
> > > It is
> > >             : implemented as a FastCGI module using libfcgi, and 
> > > can be used
> > > in
> > >             : conjunction with any FastCGI capable web server.
> > >
> > >
> > >
> > > Is this normal???
> > >
> > > I am concerned because the installed version is 0.80.7 and the 
> > > available
> > > update package is 0.80.5
> > >
> > > Have I missed something?
> > >
> > > Regards,
> > >
> > > George
> > >
> > >
> > >
> > > > Pushed a fix to wip-10271. Haven't tested it though, let me know 
> > > > if
> > > > you try it.
> > > >
> > > > Thanks,
> > > > Yehuda
> > > >
> > > > On Thu, Dec 11, 2014 at 8:38 AM, Yehuda Sadeh <yehuda@xxxxxxxxxx> 
> > > > wrote:
> > > > > I don't think it has been fixed recently. I'm looking at it now, 
> > > > > and
> > > > > not sure why it hasn't triggered before in other areas.
> > > > >
> > > > > Yehuda
> > > > >
> > > > > On Thu, Dec 11, 2014 at 5:55 AM, Georgios Dimitrakakis
> > > > > <giorgis@xxxxxxxxxxxx> wrote:
> > > > > > This issue seems very similar to these:
> > > > > >
> > > > > > http://tracker.ceph.com/issues/8202
> > > > > > http://tracker.ceph.com/issues/8702
> > > > > >
> > > > > >
> > > > > > Would it make any difference if I try to build CEPH from 
> > > > > > sources?
> > > > > >
> > > > > > I mean is someone aware of it been fixed on any of the recent 
> > > > > > commits
> > > > > > and
> > > > > > probably hasn't passed yet to the repositories?
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > > George
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Mon, 08 Dec 2014 19:47:59 +0200, Georgios Dimitrakakis wrote:
> > > > > > > I 've just created issues #10271
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > George
> > > > > > >
> > > > > > > On Fri, 5 Dec 2014 09:30:45 -0800, Yehuda Sadeh wrote:
> > > > > > > > It looks like a bug. Can you open an issue on 
> > > > > > > > tracker.ceph.com,
> > > > > > > > describing what you see?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Yehuda
> > > > > > > >
> > > > > > > > On Fri, Dec 5, 2014 at 7:17 AM, Georgios Dimitrakakis
> > > > > > > > <giorgis@xxxxxxxxxxxx> wrote:
> > > > > > > > > It would be nice to see where and how "uploadId"
> > > > > > > > >
> > > > > > > > > is being calculated...
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > George
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > For example if I try to perform the same multipart upload at 
> > > > > > > > > > an
> > > > > > > > > > older
> > > > > > > > > > version ceph version 0.72.2
> > > > > > > > > > (a913ded2ff138aefb8cb84d347d72164099cfd60)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I can see the upload ID in the apache log as:
> > > > > > > > > >
> > > > > > > > > > "PUT
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > /test/XXXX.dat?partNumber=25&uploadId=I3yihBFZmHx9CCqtcDjr8d-RhgfX8NW
> > > > > > > > > > HTTP/1.1" 200 - "-" "aws-sdk-nodejs/2.0.29 linux/v0.10.33"
> > > > > > > > > >
> > > > > > > > > > but when I try the same at ceph version 0.80.7
> > > > > > > > > > (6c0127fcb58008793d3c8b62d925bc91963672a3)
> > > > > > > > > >
> > > > > > > > > > I get the following:
> > > > > > > > > >
> > > > > > > > > > "PUT
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > /test/XXXX.dat?partNumber=12&uploadId=2%2Ff9UgnHhdK0VCnMlpT-XA8ttia1HjK36
> > > > > > > > > > HTTP/1.1" 403 78 "-" "aws-sdk-nodejs/2.0.29 linux/v0.10.33"
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > and my guess is that the "%2F" at the latter is the one that 
> > > > > > > > > > is
> > > > > > > > > > causing the problem and hence the 403 error.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > What do you think???
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > >
> > > > > > > > > > George
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Hi all!
> > > > > > > > > > >
> > > > > > > > > > > I am using AWS SDK JS v.2.0.29 to perform a multipart 
> > > > > > > > > > > upload into
> > > > > > > > > > > Radosgw with ceph version 0.80.7
> > > > > > > > > > > (6c0127fcb58008793d3c8b62d925bc91963672a3) and I am getting 
> > > > > > > > > > > a 403
> > > > > > > > > > > error.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I believe that the id which is send to all requests and has 
> > > > > > > > > > > been
> > > > > > > > > > > urlencoded by the aws-sdk-js doesn't match with the one in 
> > > > > > > > > > > rados
> > > > > > > > > > > because it's not urlencoded.
> > > > > > > > > > >
> > > > > > > > > > > Is that the case? Can you confirm it?
> > > > > > > > > > >
> > > > > > > > > > > Is there something I can do?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > >
> > > > > > > > > > > George
> > > > > > > > > > >
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > ceph-users mailing list
> > > > > > > > > > > ceph-users@xxxxxxxxxxxxxx
> > > > > > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > ceph-users mailing list
> > > > > > > > > > ceph-users@xxxxxxxxxxxxxx
> > > > > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > ceph-users mailing list
> > > > > > > > > ceph-users@xxxxxxxxxxxxxx
> > > > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > ceph-users mailing list
> > > > > > > ceph-users@xxxxxxxxxxxxxx
> > > > > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com