On Wed, Aug 3, 2011 at 7:16 AM, Christian Brunner <chb@xxxxxx> wrote: ... > I tried to reproduce this without ceph, but wasn't able to... > > In the meantime it seams, that I can also see the side effects on the > librbd side: I get an "librbd: data error!" when I do an "rbd copy". > > When I look at the librbd code this is related to a sparse_read not > returning the right size of the object. > > I don't know if it helps, but I think that the problem is also related > to sparse file usage. > There were a few sparse-read issues that we fixed not too long ago, but should have been fixed for at least the previous ceph version. I'm not sure what version you're using. There was a ext4 fiemap issue that I was hitting on specific environments but couldn't determine whether it was fixed in later kernel versions (I was using 2.6.32). Now is a good time to try and get to the bottom of it. Here's a script I was using to reproduce it: #!/bin/sh dd if=/dev/urandom of=bla bs=1 seek=$((0x6f000)) count=$((0x1000)); sync dd if=/dev/urandom of=bla bs=1 seek=$((0x70000)) count=$((0x1000)); sync dd if=/dev/urandom of=bla bs=1 seek=$((0x71000)) count=$((0x1000)); sync dd if=/dev/urandom of=bla bs=1 seek=$((0x72000)) count=$((0x1000)); sync dd if=/dev/urandom of=bla bs=1 seek=$((0x73000)) count=$((0x1000)); sync dd if=/dev/urandom of=bla bs=1 seek=$((0x74000)) count=$((0x2000)); sync dd if=/dev/urandom of=bla bs=1 seek=$((0x2ae000)) count=$((0x2000)); sync You can compile and run the following utility to dump all the extents: http://pastebin.com/h2Cnpk2Q Thanks, Yehuda Oh, btw, You can effectively disable the use of fiemap by setting the 'filestore fiemap threshold' config option with large enough value (e.g., anything bigger than 4 MB should be enough for rbd). -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
