Hi, I apologize if this topic is not proper for this mail list. I asked the question on other channel, but haven't got answers yet (see https://stackoverflow.com/questions/47245162/is-concurrent-file-read-write-with-o-direct-flag-atomic). Basically, I have a non-sparse binary file. A writer process opens the file using O_DIRECT flag, and it keeps calling pwrite() to update the first 128KB data of the file. Meanwhile, multiple readers also keeps calling pread() to read the first 128KB data. The readers open the file using O_DIRECT flag. Although I could not find any document saying that O_DIRECT guarantee atomicity of concurrent read/write, I thought that the readers should read back consistent data. Since the data to read/write from/to the file is block aligned, I assume that kernel would just submit a single scatter-gather command for one pwrite() or one pread(). Per my knowledge of SCSI/SATA driver, HDD should (correct me if I'm wrong) process each scatter-gather command atomically. Therefore, I thought read/write operations in this scenario are atomic. I wrote a program to verify my thought. Surprisingly, the readers did occasionally read back mixed data. For example, in the first pwrite(), the writer writes all 0x11, and in the 2nd pwrite(), it writes all 0x22, and in the 3rd write, it writes all 0x33... Occasionally, a reader can read back data like "0x11, 0x11, .... 0x11, 0x22, 0x22.... 0x22". The data appears to be from two consecutive pwrite() calls. I checked the offset where the broken starts. The offset seems to be sector-aligned (512-byte-aligned). Why could such consistent issue happen? Did I miss anything in my analysis and theory? Thanks, Leo
