I am chasing a problem which might be a cache aliasing problem when a disk file is opened with O_DIRECT flag. I attached the source code of two programs. One generates a binary file and the other opens the file with O_DIRECT and reads it. It checks the content of the file while reading it. I tested this on a MIPS board with NEC vr5432 CPU, which has a virtually indexed, two-way set associative d-cache, and can easily re-produce the data corruption problem. I attached a patch which apparently solves the problem. I am not an expert in fs and mm, but my guess is: 1) user process allocates a big buffer 2) the user buffer is mapped into kernel virtual space for doing direct IO through map_user_kiobuf() 3) since the virtual address for buffer area is different in user space from that in kernel virtual, kernel should do a flush cache for those pages after doing the IO. That is why my attached patch makes it work. Does this make sense? However, I still have some puzzles. For it to work completely, another cache flushing needs to be done for the address range of the buffer in user space. I thought this should be done some where inside map_user_kiobuf() but could not find it anywhere. Did I miss it? Or it just happens to work even without it? Another puzzling part is that I also tested the program on another couple of MIPS boards which *should* suffer from this problem, but failed to re-produce it. Any thoughts? Jun
