On Fri, Sep 6, 2024 at 3:54 AM <jsterr@xxxxxxxxxxxxxx> wrote:
>
> Hello Ceph Users,
>
> * Problem: we get the following errors when using krbd, we are using rbd
> for vms.
> * Workaround: by switching to librbd the errors disappear.
>
> * Software:
> ** Kernel: 6.8.8-2 (parameters: intel_iommu=on iommu=pt
> pcie_aspm.policy=performance)
> ** Ceph: 18.2.2
>
> Description/Details: Errors from using krbd with ceph. Side-effects:
>
> [Wed Aug 21 03:04:17 2024] libceph: read_partial_message
> 0000000015af2284 data crc 1221767919 != exp. 282251377
> [Wed Aug 21 03:04:17 2024] libceph: read_partial_message
> 0000000066b200ab data crc 3817026135 != exp. 3925662391
> [Wed Aug 21 03:04:17 2024] libceph: osd15 (1)10.1.4.13:6836 bad
> crc/signature
> [Wed Aug 21 03:04:17 2024] libceph: osd13 (1)10.1.4.13:6809 bad
> crc/signature
Hi Jonas,
Are these VMs running Windows? If so, are you using rxbounce mapping
option ("rbd device map -o rxbounce ...")? It's more or less required
in case there is a Windows kernel on the I/O path:
rxbounce - Use a bounce buffer when receiving data (since 5.17).
The default behaviour is to read directly into the destination
buffer. A bounce buffer is needed if the destination buffer isn’t
guaranteed to be stable (i.e. remain unchanged while it is being
read to). In particular this is the case for Windows where
a system-wide “dummy” (throwaway) page may be mapped into the
destination buffer in order to generate a single large I/O.
Otherwise, “libceph: … bad crc/signature” or “libceph: … integrity
error, bad crc” errors and associated performance degradation are
expected.
Thanks,
Ilya
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
