Thanks. -- Regards Dominik 2013/12/3 Yehuda Sadeh <yehuda@xxxxxxxxxxx>: > For bobtail at this point yes. You can try the unofficial version with > that fix off the gitbuilder. Another option is to upgrade everything > to dumpling. > > Yehuda > > On Mon, Dec 2, 2013 at 10:24 PM, Dominik Mostowiec > <dominikmostowiec@xxxxxxxxx> wrote: >> Thanks >> Workaround, don't use multipart when obj size == 0 ? >> >> On Dec 3, 2013 6:43 AM, "Yehuda Sadeh" <yehuda@xxxxxxxxxxx> wrote: >>> >>> I created earlier an issue (6919) and updated it with the relevant >>> issue. This has been fixed in dumpling, although I don't remember >>> hitting the scenario that you did. Was probably hitting it as part of >>> the development work that was done then. >>> In any case I created a branch with the relevant fixes in it (wip-6919). >>> >>> Thanks, >>> Yehuda >>> >>> On Mon, Dec 2, 2013 at 8:39 PM, Dominik Mostowiec >>> <dominikmostowiec@xxxxxxxxx> wrote: >>> > for another object. >>> > http://pastebin.com/VkVAYgwn >>> > >>> > >>> > 2013/12/3 Yehuda Sadeh <yehuda@xxxxxxxxxxx>: >>> >> I see. Do you have backtrace for the crash? >>> >> >>> >> On Mon, Dec 2, 2013 at 6:19 PM, Dominik Mostowiec >>> >> <dominikmostowiec@xxxxxxxxx> wrote: >>> >>> 0.56.7 >>> >>> >>> >>> W dniu poniedziałek, 2 grudnia 2013 użytkownik Yehuda Sadeh napisał: >>> >>> >>> >>>> I'm having trouble reproducing the issue. What version are you using? >>> >>>> >>> >>>> Thanks, >>> >>>> Yehuda >>> >>>> >>> >>>> On Mon, Dec 2, 2013 at 2:16 PM, Yehuda Sadeh <yehuda@xxxxxxxxxxx> >>> >>>> wrote: >>> >>>> > Actually, I read that differently. It only says that if there's >>> >>>> > more >>> >>>> > than 1 part, all parts except for the last one need to be > 5M. >>> >>>> > Which >>> >>>> > means that for uploads that are smaller than 5M there should be >>> >>>> > zero >>> >>>> > or one parts. >>> >>>> > >>> >>>> > On Mon, Dec 2, 2013 at 12:54 PM, Dominik Mostowiec >>> >>>> > <dominikmostowiec@xxxxxxxxx> wrote: >>> >>>> >> You're right. >>> >>>> >> >>> >>>> >> S3 api doc: >>> >>>> >> >>> >>>> >> http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html >>> >>>> >> "Err:EntityTooSmall >>> >>>> >> Your proposed upload is smaller than the minimum allowed object >>> >>>> >> size. >>> >>>> >> Each part must be at least 5 MB in size, except the last part." >>> >>>> >> >>> >>>> >> Thanks. >>> >>>> >> >>> >>>> >> This error should be triggered from radosgw also. >>> >>>> >> >>> >>>> >> -- >>> >>>> >> Regards >>> >>>> >> Dominik >>> >>>> >> >>> >>>> >> 2013/12/2 Yehuda Sadeh <yehuda@xxxxxxxxxxx>: >>> >>>> >>> Looks like it. There should be a guard against it (mulitpart >>> >>>> >>> upload >>> >>>> >>> minimum is 5M). >>> >>>> >>> >>> >>>> >>> On Mon, Dec 2, 2013 at 12:32 PM, Dominik Mostowiec >>> >>>> >>> <dominikmostowiec@xxxxxxxxx> wrote: >>> >>>> >>>> Yes, this is probably upload empty file. >>> >>>> >>>> This is the problem? >>> >>>> >>>> >>> >>>> >>>> -- >>> >>>> >>>> Regards >>> >>>> >>>> Dominik >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> 2013/12/2 Yehuda Sadeh <yehuda@xxxxxxxxxxx>: >>> >>>> >>>>> By any chance are you uploading empty objects through the >>> >>>> >>>>> multipart >>> >>>> >>>>> upload api? >>> >>>> >>>>> >>> >>>> >>>>> On Mon, Dec 2, 2013 at 12:08 PM, Dominik Mostowiec >>> >>>> >>>>> <dominikmostowiec@xxxxxxxxx> wrote: >>> >>>> >>>>>> Hi, >>> >>>> >>>>>> Another file with the same problems: >>> >>>> >>>>>> >>> >>>> >>>>>> 2013-12-01 11:37:15.556687 7f7891fd3700 1 ====== starting new >>> >>>> >>>>>> request >>> >>>> >>>>>> req=0x25406d0 ===== >>> >>>> >>>>>> 2013-12-01 11:37:15.556739 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.000052::::initializing >>> >>>> >>>>>> 2013-12-01 11:37:15.556789 7f7891fd3700 10 >>> >>>> >>>>>> s->object=files/192.txt >>> >>>> >>>>>> s->bucket=testbucket >>> >>>> >>>>>> 2013-12-01 11:37:15.556799 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.000112:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt::getting op >>> >>>> >>>>>> 2013-12-01 11:37:15.556804 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.000118:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:authorizing >>> >>>> >>>>>> 2013-12-01 11:37:15.560013 7f7891fd3700 10 >>> >>>> >>>>>> get_canon_resource(): >>> >>>> >>>>>> >>> >>>> >>>>>> >>> >>>> >>>>>> dest=/testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC >>> >>>> >>>>>> 2013-12-01 11:37:15.560027 7f7891fd3700 10 auth_hdr: >>> >>>> >>>>>> POST >>> >>>> >>>>>> >>> >>>> >>>>>> application/xml >>> >>>> >>>>>> Sun, 01 Dec 2013 10:37:10 GMT >>> >>>> >>>>>> >>> >>>> >>>>>> /testbucket/files/192.txt?uploadId=i92xi2olzDtFAeLXlfU2PFP9CDU87BC >>> >>>> >>>>>> 2013-12-01 11:37:15.560085 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.003399:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:reading >>> >>>> >>>>>> permissions >>> >>>> >>>>>> 2013-12-01 11:37:15.562356 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.005670:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op >>> >>>> >>>>>> permissions >>> >>>> >>>>>> 2013-12-01 11:37:15.562373 7f7891fd3700 5 Searching >>> >>>> >>>>>> permissions >>> >>>> >>>>>> for >>> >>>> >>>>>> uid=0 mask=2 >>> >>>> >>>>>> 2013-12-01 11:37:15.562377 7f7891fd3700 5 Found permission: >>> >>>> >>>>>> 15 >>> >>>> >>>>>> 2013-12-01 11:37:15.562378 7f7891fd3700 10 uid=0 requested >>> >>>> >>>>>> perm >>> >>>> >>>>>> (type)=2, policy perm=2, user_perm_mask=2, acl perm=2 >>> >>>> >>>>>> 2013-12-01 11:37:15.562381 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.005695:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:verifying op >>> >>>> >>>>>> params >>> >>>> >>>>>> 2013-12-01 11:37:15.562384 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.005698:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:executing >>> >>>> >>>>>> 2013-12-01 11:37:15.565461 7f7891fd3700 10 calculated etag: >>> >>>> >>>>>> d41d8cd98f00b204e9800998ecf8427e-0 >>> >>>> >>>>>> 2013-12-01 11:37:15.566718 7f7891fd3700 10 can't clone object >>> >>>> >>>>>> testbucket:files/192.txt to shadow object, tag/shadow_obj >>> >>>> >>>>>> haven't >>> >>>> >>>>>> been >>> >>>> >>>>>> set >>> >>>> >>>>>> 2013-12-01 11:37:15.566777 7f7891fd3700 0 setting object >>> >>>> >>>>>> tag=_leyAzxCw7YxpKv8P3v3QGwcsw__9VmP >>> >>>> >>>>>> 2013-12-01 11:37:15.678973 7f7891fd3700 2 req >>> >>>> >>>>>> 1314:0.122286:s3:POST >>> >>>> >>>>>> /testbucket/files/192.txt:complete_multipart:http status=200 >>> >>>> >>>>>> 2013-12-01 11:37:15.679192 7f7891fd3700 1 ====== req done >>> >>>> >>>>>> req=0x25406d0 http_status=200 ====== >>> >>>> >>>>>> >>> >>>> >>>>>> Yes, I can read oryginal object. >>> >>>> >>>>>> >>> >>>> >>>>>> -- >>> >>>> >>>>>> Regards >>> >>>> >>>>>> Dominik >>> >>>> >>>>>> >>> >>>> >>>>>> 2013/12/2 Yehuda Sadeh <yehuda@xxxxxxxxxxx>: >>> >>>> >>>>>>> That's unknown bug. I have a guess as to how the original >>> >>>> >>>>>>> object >>> >>>> >>>>>>> was >>> >>>> >>>>>>> created. Can you read the original object, but only copy >>> >>>> >>>>>>> fails? >>> >>>> >>>>>>> >>> >>>> >>>>>>> On Dec 2, 2013 4:53 AM, "Dominik Mostowiec" >>> >>>> >>>>>>> <dominikmostowiec@xxxxxxxxx> >>> >>>> >>>>>>> wrote: >>> >>>> >>>>>>>> >>> >>>> >>>>>>>> Hi, >>> >>>> >>>>>>>> I found that issue is related with "ETag: ....-0" (ends -0) >>> >>>> >>>>>>>> This is known bug ? >>> >>>> >>>>>>>> >>> >>>> >>>>>>>> -- >>> >>>> >>>>>>>> Regards >>> >>>> >>>>>>>> Dominik >>> >>>> >>>>>>>> >>> >>>> >>>>>>>> 2013/12/2 Dominik Mostowiec <dominikmostowiec@xxxxxxxxx>: >>> >>>> >>>>>>>> > Hi, >>> >>>> >>>>>>>> > I have strange problem. >>> >>>> >>>>>>>> > Obj copy (0 size) killing radosgw. >>> >>>> >>>>>>>> > >>> >>>> >>>>>>>> > Head for this file: >>> >>>> >>>>>>>> > Content-Type: application/octet-stream >>> >>>> >>>>>>>> > Server: Apache/2.2.22 (Ubuntu) >>> >>>> >>>>>>>> > ETag: "d41d8cd98f00b204e9800998ecf8427e-0" >>> >>>> >>>>>>>> > Last-Modified: 2013-12-01T10:37:15Z >>> >>>> >>>>>>>> > >>> >>>> >>>>>>>> > rgw log. >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196651 7f5308ff1700 1 ====== starting >>> >>>> >>>>>>>> > new >>> >>>> >>>>>>>> > request >>> >>>> >>>>>>>> > req=0x2be6fa0 ===== >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196709 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.000058::::initializing >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196752 7f5308ff1700 10 meta>> >>> >>>> >>>>>>>> > HTTP_X_AMZ_ACL=public-read >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196760 7f5308ff1700 10 meta>> >>> >>>> >>>>>>>> > HTTP_X_AMZ_COPY_SOURCE=/testbucket/testfile.xml >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196766 7f5308ff1700 10 meta>> >>> >>>> >>>>>>>> > HTTP_X_AMZ_METADATA_DIRECTIVE=COPY >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196771 7f5308ff1700 10 x>> >>> >>>> >>>>>>>> > x-amz-acl:public-read >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196772 7f5308ff1700 10 x>> >>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196773 7f5308ff1700 10 x>> >>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196786 7f5308ff1700 10 >>> >>>> >>>>>>>> > s->object=/testbucket/new_testfile.ini >>> >>>> >>>>>>>> > s->bucket=testbucket >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196792 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.000141:s3:PUT >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini::getting op >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.196797 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.000146:s3:PUT >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:authorizing >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200648 7f5308ff1700 10 >>> >>>> >>>>>>>> > get_canon_resource(): >>> >>>> >>>>>>>> > dest=/testbucket/new_testfile.ini >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200661 7f5308ff1700 10 auth_hdr: >>> >>>> >>>>>>>> > PUT >>> >>>> >>>>>>>> > 1B2M2Y8AsgTpgAmY7PhCfg== >>> >>>> >>>>>>>> > application/octet-stream >>> >>>> >>>>>>>> > Mon, 02 Dec 2013 07:18:55 GMT >>> >>>> >>>>>>>> > x-amz-acl:public-read >>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml >>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.200717 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.004066:s3:PUT >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:reading permissions >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.203330 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.006679:s3:PUT >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op >>> >>>> >>>>>>>> > permissions >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207627 7f5308ff1700 10 manifest: >>> >>>> >>>>>>>> > total_size >>> >>>> >>>>>>>> > = 0 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207649 7f5308ff1700 5 Searching >>> >>>> >>>>>>>> > permissions for >>> >>>> >>>>>>>> > uid=0 mask=1 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207652 7f5308ff1700 5 Found >>> >>>> >>>>>>>> > permission: 15 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207654 7f5308ff1700 10 uid=0 >>> >>>> >>>>>>>> > requested >>> >>>> >>>>>>>> > perm >>> >>>> >>>>>>>> > (type)=1, policy perm=1, user_perm_mask=15, acl perm=1 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207669 7f5308ff1700 5 Searching >>> >>>> >>>>>>>> > permissions for >>> >>>> >>>>>>>> > uid=0 mask=2 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207670 7f5308ff1700 5 Found >>> >>>> >>>>>>>> > permission: 15 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207671 7f5308ff1700 10 uid=0 >>> >>>> >>>>>>>> > requested >>> >>>> >>>>>>>> > perm >>> >>>> >>>>>>>> > (type)=2, policy perm=2, user_perm_mask=15, acl perm=2 >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207681 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.011030:s3:PUT >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:verifying op params >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207686 7f5308ff1700 2 req >>> >>>> >>>>>>>> > 237:0.011035:s3:PUT >>> >>>> >>>>>>>> > /testbucket/new_testfile.ini:copy_obj:executing >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207699 7f5308ff1700 10 x>> >>> >>>> >>>>>>>> > x-amz-acl:public-read >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207704 7f5308ff1700 10 x>> >>> >>>> >>>>>>>> > x-amz-copy-source:/testbucket/testfile.xml >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207709 7f5308ff1700 10 x>> >>> >>>> >>>>>>>> > x-amz-metadata-directive:COPY >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.207759 7f5308ff1700 5 Copy object >>> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):testfile.ini => >>> >>>> >>>>>>>> > testbucket(@.rgw.buckets[406250.1]):new_testfile.ini >>> >>>> >>>>>>>> > 2013-12-02 08:18:59.208903 7f5308ff1700 -1 *** Caught >>> >>>> >>>>>>>> > signal >>> >>>> >>>>>>>> > (Segmentation fault) ** >>> >>>> >>>>>>>> > in thread 7f5308ff1700 >>> >>>> >>>>>>>> > >>> >>>> >>>>>>>> > >>> >>>> >>>>>>>> > -- >>> >>>> >>>>>>>> > Regards >>> >>>> >>>>>>>> > Dominik >>> >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> >>>> >>>>>>>> -- >>> >>>> >>>>>>>> Pozdrawiam >>> >>>> >>>>>>>> Dominik >>> >>>> >>>>>>>> _______________________________________________ >>> >>>> >>>>>>>> ceph-users mailing list >>> >>>> >>>>>>>> ceph-users@xxxxxxxxxxxxxx >>> >>>> >>>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>>> >>>>>> >>> >>>> >>>>>> >>> >>>> >>>>>> >>> >>>> >>>>>> -- >>> >>>> >>>>>> Pozdrawiam >>> >>>> >>>>>> Dominik >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> -- >>> >>>> >>>> Pozdrawiam >>> >>>> >>>> Dominik >>> >>>> >> >>> >>>> >> >>> >>>> >> >>> >>>> >> -- >>> >>>> >> Pozdrawiam >>> >>>> >> Dominik >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Pozdrawiam >>> >>> Dominik >>> > >>> > >>> > >>> > -- >>> > Pozdrawiam >>> > Dominik -- Pozdrawiam Dominik _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |