Some debugging with gdb reveals this issue has been patched in https://github.com/axboe/fio/commit/9781c080385364d24ac18a95544484864a27611a which is included in fio-3.14 or later. The important detail missing from the commit summary is that the fix is not limited to Windows as I encountered it in a 32-bit Linux kernel. -- Alex Fetters -----Original Message----- From: Fetters, Alex Sent: Wednesday, January 29, 2020 4:55 PM To: 'fio@xxxxxxxxxxxxxxx' <fio@xxxxxxxxxxxxxxx> Subject: Random fill data I'm having issue getting a full fill when using the default buffer fill. Specifically, every other 32-bits of fill is 0x00000000. This is similar to the issue seen here: https://www.spinics.net/lists/fio/msg07529.html The command I'm using is: fio --filename=test.bin --direct=1 --rw=write --bs=512 --size=512 --name=test --verify=crc32c which produces: xxd test.bin 00000000: caac 0500 0002 0000 c249 e005 0000 0000 .........I...... 00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 00000020: 0100 0000 c3e9 1fe9 42df bf59 0000 0000 ........B..Y.... 00000030: c320 0c14 0000 0000 1824 fe0e 0000 0000 . .......$...... 00000040: 83c4 0414 0000 0000 9038 c504 0000 0000 .........8...... 00000050: 12a7 7601 0000 0000 e294 4a00 0000 0000 ..v.......J..... 00000060: 9c12 1b0f 0000 0000 53e2 d30a 0000 0000 ........S....... 00000070: 4a1c 951c 0000 0000 8963 8f1b 0000 0000 J........c...... 00000080: 71cc 7f01 0000 0000 8ed9 7815 0000 0000 q.........x..... 00000090: 315b df0e 0000 0000 66cb 8c1c 0000 0000 1[......f....... 000000a0: 6cd9 6e18 0000 0000 2d5b d40e 0000 0000 l.n.....-[...... 000000b0: 65eb 6f0d 0000 0000 6c5d c41e 0000 0000 e.o.....l]...... 000000c0: ad0b 1f1b 0000 0000 7541 690c 0000 0000 ........uAi..... 000000d0: 2e88 510a 0000 0000 0571 0616 0000 0000 ..Q......q...... 000000e0: 202e 830e 0000 0000 c465 ec16 0000 0000 ........e...... 000000f0: b80c 811d 0000 0000 9721 210d 0000 0000 .........!!..... 00000100: 3244 d20e 0000 0000 8608 3212 0000 0000 2D........2..... 00000110: 1081 5f13 0000 0000 22f0 b915 0000 0000 .._....."....... 00000120: 04fe 2015 0000 0000 c09f ff03 0000 0000 .. ............. 00000130: f8f3 c71a 0000 0000 7ffe a112 0000 0000 ................ 00000140: cf5f 3d04 0000 0000 f9cb 760b 0000 0000 ._=.......v..... 00000150: 7fb9 7e11 0000 0000 2ff7 b809 0000 0000 ..~...../....... 00000160: e53e da04 0000 0000 dca7 c11b 0000 0000 .>.............. 00000170: fbb4 801e 0000 0000 9fb6 4d1d 0000 0000 ..........M..... 00000180: d3d6 6e0f 0000 0000 da7a 180e 0000 0000 ..n......z...... 00000190: 5bcf dd1f 0000 0000 eb59 cd1a 0000 0000 [........Y...... 000001a0: 3d4b c90e 0000 0000 6789 dc1b 0000 0000 =K......g....... 000001b0: 2cb1 5f06 0000 0000 2576 da07 0000 0000 ,._.....%v...... 000001c0: c4ae f907 0000 0000 d8b5 8211 0000 0000 ................ 000001d0: bb56 5d0e 0000 0000 d74a 5103 0000 0000 .V]......JQ..... 000001e0: 5a49 3019 0000 0000 2bc9 f00c 0000 0000 ZI0.....+....... 000001f0: 25b9 e519 0000 0000 2417 db16 0000 0000 %.......$....... I included the verify header to show that the byte can be written to. To me, this looks like a 32 vs 64 bit issue. Specifically, somewhere a 32-bit value is used for the fill and then the buffer is incremented by 64-bits. fio is 32-bit. file /usr/bin/fio /usr/bin/fio: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib, for GNU/Linux 3.2.0, BuildID[sha1]=61891ac32f94190d63ea4f182e585b4a093bd8b6, stripped As is my Kernel (ARMv7 architectures are 32-bit: https://en.wikipedia.org/wiki/ARM_architecture#Cores) uname -s -r -m Linux 4.14.5-92 armv7l Digging into the fio, fill_random_buf() https://github.com/axboe/fio/blob/master/lib/rand.c#L125 is suspicious. Specifically, when the state->use32 is true, __rand() will return a number contained within 32-bits of the uint64_t. __fill_random_buf() then should increment by sizeof(int64_t). Unfortunately, that should only impact the first few values. GOLDEN_RATIO_PRIME is sufficiently large that when multiplied with <seed> in __fill_random_buf() it should quickly grow past 32-bits. And now I'm lost. Any ideas where the holes are coming from? I have verified that on a VM running 64-bit Linux and FIO, there are no holes. -- Alex Fetters
