On Thu, Jun 30, 2016 at 1:43 PM, Sitsofe Wheeler <sitsofe@xxxxxxxxx> wrote: > Could you bottom post replies? It makes things a bit easier here... > Sorry about that. >> On Thu, Jun 30, 2016 at 10:58 AM, Srinath Krishna Ananthakrishnan >> <ska@xxxxxxxxx> wrote: >>> I don't want fio to vary the data and need it to be more deterministic. > > Don't forget you can use can also use randseed to generate the same > random data between invocations. > >>> After playing with it for quite some time, I have the following >>> (inconclusive) theory. >>> >>> 1. If no verify options are set, fio generates >>> buffer_compress_percentage worth of compressible data per block size. >>> Compressible data is always zeroed out. >>> 2. With verify options, fio generates buffer_compress_percentage worth >>> of compressible data (zeroes) for some blocks but there are these >>> bunch of blocks from time to time that contain purely random data. > > You can inspect the source code directly (e.g. > https://github.com/axboe/fio/blob/8c07860de982fabaaf41d44c22aa86aba2539b58/io_u.c#L2031 > ) to find out what's happening. > >>> From the manual, verify options add additional meta data to the header >>> of every block for verification but I'm not sure why some blocks turn >>> out to be completely random with this setting on. I tried multiple >>> verify hashes with the same result. >>> >>> With either of the cases, I don't seem to get the buffer_pattern >>> setting working. > > I don't quite see the behaviour you describe. I used the following > with the latest git fio on x86_64: > [global] > randrepeat=1 > ioengine=sync > iodepth=64 > iodepth_batch=16 > direct=1 > numjobs=1 > verify_fatal=1 > verify_dump=1 > filename=./my_file > > [small_write] > rw=write > blocksize=4k > size=100M > verify=crc32c-intel > buffer_compress_percentage=50 > buffer_pattern=0xdeadbeef > > Using hexdump showed an initial header followed by random data up to > 0x800 and from 0x801 to 0xfff was the pattern deadbeef (in binary). So > 50% of the block was the header + random and the other 50% was the > buffer_pattern I specified. Bear in mind that if something were to > compress these blocks it would likely get better than 50% compression > because the repeating deadbeef pattern is itself highly > compressible... > > On 30 June 2016 at 19:35, Srinath Krishna Ananthakrishnan <ska@xxxxxxxxx> wrote: >> Another idiosyncrasy that I observed was, >> >> With rw=write, compression options are not heeded. They seem to work only rw=rw. >> Thanks, >> Srinath > > See the above job file where I used rw=write. Are you using a git > version of fio? I just tried on the tip of the branch fio and it works. The fio version I was using was 2.1.10 which is quite old. I believe these issues have been addressed in the more recent versions. Thanks Sitsofe. > > -- > Sitsofe | http://sucs.org/~sits/ -- To unsubscribe from this list: send the line "unsubscribe fio" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
