Hi Sage,
On 06/18/2012 06:02 AM, Sage Weil wrote:
If you are using RBD, and want to avoid potential image corruption, add
filestore fiemap = false
to the [osd] section of your ceph.conf and restart your OSDs.
as far as this heals some trouble, but I fairly don't understand...
We've tracked down the source of some corruption to racy/buggy FIEMAP
ioctl behavior. The RBD client (when caching is diabled--the default)
uses a 'sparse read' operation that the OSD implements by doing an fsync
on the object file, mapping which extents are allocated, and sending only
that data over the wire. We have observed incorrect/changing FIEMAP on
both btrfs:
fsync
fiemap returns mapping
<time passes, no modifications to file>
fiemap returns different mapping
... that even an initial start of a VM leads to corruption of the read data?
I get s/t like:
--- 8-< ---
Loading, please wait
/sbin/init: relocation error: ...
not defined in file libc.so.6...
[ 0.81...] Kernel panic - not snycing: Attempted to kill init!
--- 8-< ---
host-kernel is now 3.4.1 + qemu-1.0.1, but shows failures with other
kernel/qemu-versions, too.
Keeping fingers crossed for Josh, though ;-)
Give me a shout, If I can do some debugging,
regards,
Oliver.
Josh is still tracking down which kernels and file system are affected;
fortunately it is relatively easy to reproduce with the test_librbd_fsx
tool. In the meantime, the (mis)feature can be safely disabled. It will
default to off in 0.48. It is unclear whether it's really much of a
performance win anyway.
Thanks!
sage
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Oliver Francke
filoo GmbH
Moltkestraße 25a
33330 Gütersloh
HRB4355 AG Gütersloh
Geschäftsführer: S.Grewing | J.Rehpöhler | C.Kunz
Folgen Sie uns auf Twitter: http://twitter.com/filoogmbh
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
