Hi, The directory there should be some simulated hierarchical structure with '/' in the object names. Do you mind checking the rest objects in ceph pool .rgw.buckets? $ rados ls -p .rgw.buckets | grep default.157931.5_hive If there're still objects come out, you might try to delete them from the 'olla' bucket with S3 API. (Note I'm not sure how's your Hive data generated, so please do backup first if it's important.) thanks, -yuan -----Original Message----- From: Ceph-cn [mailto:ceph-cn-bounces@xxxxxxxxxxxxxx] On Behalf Of ??? Sent: Tuesday, November 3, 2015 12:22 PM To: ceph-cn@xxxxxxxxxxxxxx; ceph-users@xxxxxxxx Subject: Re: [Ceph-cn] librados: Objecter returned from getxattrs r=-2 With debug_objecter = 20/0 I get this, I guess the thing is: the object has been removed, but "directory" info still exists. 2015-11-03 12:07:22.264704 7f03c42f3700 10 client.214496.objecter ms_dispatch 0x2c18840 osd_op_reply(81 default.157931.5_hive/staging_hive_2015-11-01_14-57-40_861_3797779765210222008-1/_tmp.-ext-10000/ [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 So, how can I safely remove the "directory" info? On Tue, 3 Nov 2015 10:10:26 +0800 张绍文 <zhangshaowen@xxxxxxxx> wrote: > On Mon, 2 Nov 2015 16:47:11 +0800 > 张绍文 <zhangshaowen@xxxxxxxx> wrote: > > > On Mon, 2 Nov 2015 16:36:57 +0800 > > 张绍文 <zhangshaowen@xxxxxxxx> wrote: > > > > > Hi, all: > > > > > > I'm using hive via s3a, but it's not usable after I removed some > > > temp files with: > > > > > > /opt/hadoop/bin/hdfs dfs -rm -r -f s3a://olla/hive/ > > > > > > With debug_radosgw = 10/0, I got these messages repeatly: > > > > > > 2015-11-02 14:30:44.547271 7f08ef7fe700 10 librados: Objecter > > > returned from getxattrs r=-2 2015-11-02 14:30:44.549117 > > > 7f08ef7fe700 10 librados: getxattrs > > > oid=default.157931.5_hive/staging_hive_2015-11-01_14-57-40_861_379 > > > 7779765210222008-1/_tmp.-ext-10000/ > > > nspace= > > > > > > I dumped whole object list, and there's no object named starts > > > with hive/..., and hive is not usable now, please help. > > > > > > > Sorry, I forgot this: > > > > # ceph -v > > ceph version 0.94.2 (5fb85614ca8f354284c713a2f9c610860720bbf3) > > > > Other known "directories" under THE same bucket is readable. > > > > > > Also happened to others on ceph-users maillist, seems not resolved: > > http://article.gmane.org/gmane.comp.file-systems.ceph.user/7653/match= > objecter+returned+getxattrs > > -- 张绍文 _______________________________________________ Ceph-cn mailing list Ceph-cn@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-cn-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |