Re: Sudden RADOS Gateway issues caused by missing xattrs

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 02/16/2014 06:49 PM, Gregory Farnum wrote:
Did you maybe upgrade that box to v0.67.6? This sounds like one of the
bugs Sage mentioned in it.

No, I checked it again. Version is: ceph version 0.67.5 (a60ac9194718083a4b6a225fc17cad6096c69bd1)

All machines in the cluster are on that version.

Wido

-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com


On Sun, Feb 16, 2014 at 4:23 AM, Wido den Hollander <wido@xxxxxxxx> wrote:
Hi,

Yesterday I got a notification that a RGW setup was having issues with
objects suddenly giving errors (403 and 404) when trying to access them.

I started digging and after cranking up the logs with 'debug rados' and
'debug rgw' set to 20 I found what caused RGW to throw a error:

librados: Objecter returned from getxattrs r=-2

Using "ceph osd map .rgw.buckets <object>" I found which OSDs were primary
for that object's PG and I saw that they all came from one machine which got
a clean shutdown and start just 24 hours before that.

After taking that machine out of production the other OSDs took over and RGW
started serving the objects again, but I'm confused.

The underlying filesystem is XFS and all 6 filesystems were clean and
healthy. Like I said, the machine only got a clean shutdown 24 hours before
that due to a physical migration, but that's all.

Did anybody see this before? Suddenly the xattrs for those objects were
gone.

This was with Ceph 0.67.5

--
Wido den Hollander
42on B.V.

Phone: +31 (0)20 700 9902
Skype: contact42on
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


--
Wido den Hollander
42on B.V.

Phone: +31 (0)20 700 9902
Skype: contact42on
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux