On Mon, Mar 13, 2017 at 2:47 AM, James Hogan <james.hogan@xxxxxxxxxx> wrote: > On Sun, Mar 12, 2017 at 06:43:47PM -0700, Matt Turner wrote: >> On a Broadcom BCM91250a MIPS system I can reliably trigger NFS >> corruption on the first file read. >> >> To demonstrate, I downloaded five identical copies of the gcc-5.4.0 >> source tarball. On the NFS server, they hash to the same value: >> >> server distfiles # md5sum gcc-5.4.0.tar.bz2* >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.1 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.3 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.4 >> >> On the MIPS system (the NFS client): >> >> bcm91250a-le distfiles # md5sum gcc-5.4.0.tar.bz2.2 >> 35346975989954df8a8db2b034da610d gcc-5.4.0.tar.bz2.2 >> bcm91250a-le distfiles # md5sum gcc-5.4.0.tar.bz2* >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.1 >> 35346975989954df8a8db2b034da610d gcc-5.4.0.tar.bz2.2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.3 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.4 >> >> The first file read will contain some corruption, and it is persistent until... >> >> bcm91250a-le distfiles # echo 1 > /proc/sys/vm/drop_caches >> bcm91250a-le distfiles # md5sum gcc-5.4.0.tar.bz2* >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.1 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.2 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.3 >> 4c626ac2a83ef30dfb9260e6f59c2b30 gcc-5.4.0.tar.bz2.4 >> >> the caches are dropped, at which point it reads back properly. >> >> Note that the corruption is different across reboots, both in the size >> of the corruption and the location. I saw 1900~ and 1400~ byte >> sequences corrupted on separate occasions, which don't correspond to >> the system's 16kB page size. >> >> I've tested kernels from v3.19 to 4.11-rc1+ (master branch from >> today). All exhibit this behavior with differing frequencies. Earlier >> kernels seem to reproduce the issue less often, while more recent >> kernels reliably exhibit the problem every boot. >> >> How can I further debug this? > > It smells a bit like a DMA / caching issue. > > Can you provide a full kernel log. That might provide some information > about caching that might be relevant (e.g. does dcache have aliases?). Thanks for the reply. Please find attached dmesg from a clean boot (which reproduced the problem).
Attachment:
dmesg
Description: Binary data
