I have another thread goign on about truncation of objects and I believe
this is a separate but equally bad issue in civetweb/radosgw. My cluster
is completely healthy
I have one (possibly more) objects stored in ceph rados gateway that
will return a different size every time I Try to download it::
http://pastebin.com/hK1iqXZH --- ceph -s
http://pastebin.com/brmxQRu3 --- radosgw-admin object stat of the object
http://pastebin.com/5TnvgMrX --- python download code
The weird part is every time I download the file it is of a different
size. I am grabbing the individual objects of the 14g file and will
update this email once I have them all statted out. Currently I am
getting, on average, 1.5G to 2Gb files when the total object should be
14G in size.
lacadmin@kh10-9:~$ python corruptpull.py
the download failed. The filesize = 2125988202. The actual size is
14577056082. Attempts = 1
the download failed. The filesize = 2071462250. The actual size is
14577056082. Attempts = 2
the download failed. The filesize = 2016936298. The actual size is
14577056082. Attempts = 3
the download failed. The filesize = 1643643242. The actual size is
14577056082. Attempts = 4
the download failed. The filesize = 1597505898. The actual size is
14577056082. Attempts = 5
the download failed. The filesize = 2075656554. The actual size is
14577056082. Attempts = 6
the download failed. The filesize = 650117482. The actual size is
14577056082. Attempts = 7
the download failed. The filesize = 1987576170. The actual size is
14577056082. Attempts = 8
the download failed. The filesize = 2109210986. The actual size is
14577056082. Attempts = 9
the download failed. The filesize = 2142765418. The actual size is
14577056082. Attempts = 10
the download failed. The filesize = 2134376810. The actual size is
14577056082. Attempts = 11
the download failed. The filesize = 2146959722. The actual size is
14577056082. Attempts = 12
the download failed. The filesize = 2142765418. The actual size is
14577056082. Attempts = 13
the download failed. The filesize = 1467482474. The actual size is
14577056082. Attempts = 14
the download failed. The filesize = 2046296426. The actual size is
14577056082. Attempts = 15
the download failed. The filesize = 2021130602. The actual size is
14577056082. Attempts = 16
the download failed. The filesize = 1773666666. The actual size is
14577056082. Attempts = 17
the download failed. The filesize = 2146959722. The actual size is
14577056082. Attempts = 18
the download failed. The filesize = 2016936298. The actual size is
14577056082. Attempts = 19
the download failed. The filesize = 1983381866. The actual size is
14577056082. Attempts = 20
the download failed. The filesize = 2134376810. The actual size is
14577056082. Attempts = 21
Notice it is always different. Once the rados -p .rgw.buckets ls | grep
finishes I will return the listing of objects as well but this is quite
odd and I think this is a separate issue.
Has anyone seen this before? Why wouldn't radosgw return an error and
why am I getting different file sizes?
I would post the log from radosgw but I don't see any "err|wrn|fatal"
mentions in the log and the client completes without issue every time.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
