Re: RGW -- 404 on keys in bucket.list() thousands of multipart ids listed as well.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



That's interesting, and might point at the underlying issue that
caused it. Could be a racing upload that somehow ended up with the
wrong object head. The 'multipart' object should be 4M in size, and
the 'shadow' one should have the remainder of the data. You can run
'rados stat -p .rgw.buckets <oid>' to validate that. If that's the
case, you can copy these to the expected object names:

$ src_uploadid=wksHvto9gRgHUJbhm_TZPXJTZUPXLT2
$ dest_uploadid=pcu5Hz6foFXjlSxBat22D8YMcHlQOBD

$ rados -p .rgw.buckets cp
default.384153.1__multipart_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_20150101130000_20150101135959.tar.2~${src_uploadid}.1
default.384153.1__multipart_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_20150101130000_20150101135959.tar.2~${dest_uploadid}.1

$ rados -p .rgw.buckets cp
default.384153.1__shadow_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_20150101130000_20150101135959.tar.2~${src_upload_id}.1_1
default.384153.1__shadow_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_20150101130000_20150101135959.tar.2~${dest_upload_id}.1_1

Yehuda


On Fri, Jan 15, 2016 at 1:02 PM, seapasulli@xxxxxxxxxxxx
<seapasulli@xxxxxxxxxxxx> wrote:
> lacadmin@kh28-10:~$ rados -p .rgw.buckets ls | grep 'pcu5Hz6'
> lacadmin@kh28-10:~$
>
> Nothing was found. That said when I run the command with another prefix
> snippet::
> lacadmin@kh28-10:~$ rados -p .rgw.buckets ls | grep 'wksHvto'
> default.384153.1__shadow_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_20150101130000_20150101135959.tar.2~wksHvto9gRgHUJbhm_TZPXJTZUPXLT2.1_1
> default.384153.1__multipart_2015/01/01/KABR/NWS_NEXRAD_NXL2DP_KABR_20150101130000_20150101135959.tar.2~wksHvto9gRgHUJbhm_TZPXJTZUPXLT2.1
>
>
>
>
> On 1/15/16 12:05 PM, Yehuda Sadeh-Weinraub wrote:
>>
>> On Fri, Jan 15, 2016 at 9:36 AM, seapasulli@xxxxxxxxxxxx
>> <seapasulli@xxxxxxxxxxxx> wrote:
>>>
>>> Hello Yehuda,
>>>
>>> Here it is::
>>>
>>> radosgw-admin object stat --bucket="noaa-nexrad-l2"
>>>
>>> --object="2015/01/01/PAKC/NWS_NEXRAD_NXL2DP_PAKC_20150101110000_20150101115959.tar"
>>> {
>>>      "name":
>>>
>>> "2015\/01\/01\/PAKC\/NWS_NEXRAD_NXL2DP_PAKC_20150101110000_20150101115959.tar",
>>>      "size": 7147520,
>>>      "policy": {
>>>          "acl": {
>>>              "acl_user_map": [
>>>                  {
>>>                      "user": "b05f707271774dbd89674a0736c9406e",
>>>                      "acl": 15
>>>                  }
>>>              ],
>>>              "acl_group_map": [
>>>                  {
>>>                      "group": 1,
>>>                      "acl": 1
>>>                  }
>>>              ],
>>>              "grant_map": [
>>>                  {
>>>                      "id": "",
>>>                      "grant": {
>>>                          "type": {
>>>                              "type": 2
>>>                          },
>>>                          "id": "",
>>>                          "email": "",
>>>                          "permission": {
>>>                              "flags": 1
>>>                          },
>>>                          "name": "",
>>>                          "group": 1
>>>                      }
>>>                  },
>>>                  {
>>>                      "id": "b05f707271774dbd89674a0736c9406e",
>>>                      "grant": {
>>>                          "type": {
>>>                              "type": 0
>>>                          },
>>>                          "id": "b05f707271774dbd89674a0736c9406e",
>>>                          "email": "",
>>>                          "permission": {
>>>                              "flags": 15
>>>                          },
>>>                          "name": "noaa-commons",
>>>                          "group": 0
>>>                      }
>>>                  }
>>>              ]
>>>          },
>>>          "owner": {
>>>              "id": "b05f707271774dbd89674a0736c9406e",
>>>              "display_name": "noaa-commons"
>>>          }
>>>      },
>>>      "etag": "b91b6f1650350965c5434c547b3c38ff-1\u0000",
>>>      "tag": "_cWrvEa914Gy1AeyzIhRlUdp1wJnek3E\u0000",
>>>      "manifest": {
>>>          "objs": [],
>>>          "obj_size": 7147520,
>>>          "explicit_objs": "false",
>>>          "head_obj": {
>>>              "bucket": {
>>>                  "name": "noaa-nexrad-l2",
>>>                  "pool": ".rgw.buckets",
>>>                  "data_extra_pool": ".rgw.buckets.extra",
>>>                  "index_pool": ".rgw.buckets.index",
>>>                  "marker": "default.384153.1",
>>>                  "bucket_id": "default.384153.1"
>>>              },
>>>              "key": "",
>>>              "ns": "",
>>>              "object":
>>>
>>> "2015\/01\/01\/PAKC\/NWS_NEXRAD_NXL2DP_PAKC_20150101110000_20150101115959.tar",
>>>              "instance": ""
>>>          },
>>>          "head_size": 0,
>>>          "max_head_size": 0,
>>>          "prefix":
>>>
>>> "2015\/01\/01\/PAKC\/NWS_NEXRAD_NXL2DP_PAKC_20150101110000_20150101115959.tar.2~pcu5Hz6foFXjlSxBat22D8YMcHlQOBD",
>>
>> Try running:
>> $ rados -p .rgw.buckets ls | grep pcu5Hz6
>>
>> Yehuda
>>
>>
>>>          "tail_bucket": {
>>>              "name": "noaa-nexrad-l2",
>>>              "pool": ".rgw.buckets",
>>>              "data_extra_pool": ".rgw.buckets.extra",
>>>              "index_pool": ".rgw.buckets.index",
>>>              "marker": "default.384153.1",
>>>              "bucket_id": "default.384153.1"
>>>          },
>>>          "rules": [
>>>              {
>>>                  "key": 0,
>>>                  "val": {
>>>                      "start_part_num": 1,
>>>                      "start_ofs": 0,
>>>                      "part_size": 0,
>>>                      "stripe_max_size": 4194304,
>>>                      "override_prefix": ""
>>>                  }
>>>              }
>>>          ]
>>>      },
>>>      "attrs": {}
>>>
>>> }
>>>
>>> On 1/15/16 11:17 AM, Yehuda Sadeh-Weinraub wrote:
>>>>
>>>> radosgw-admin object stat --bucket=<bucket> --object=<object>'
>>>
>>>
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux