I was just verifying if i could generate 100% duplicable data with
FIO. I have configured small workload with bs of 256k and writing 2MB
of file. I tried to get the checksum of each of 256k blocks of data
from the file and the checksums do not match. If i am not wrong, when
i specify data as 100% deduppable, my checksums should match isn't it?
# cat ddp_file.fio
[dedupe]
filename=test.tmp
bs=256k
rw=write
size=2m
dedupe_percentage=100
write_iolog=test.tmp.log
# fio ddp_file.fio
dedupe: (g=0): rw=write, bs=256K-256K/256K-256K/256K-256K,
ioengine=sync, iodepth=1
fio-2.2.7-24-g7c30
Starting 1 process
dedupe: Laying out IO file(s) (1 file(s) / 2MB)
dedupe: (groupid=0, jobs=1): err= 0: pid=31497: Mon Apr 27 09:13:35 2015
write: io=2048.0KB, bw=2000.0MB/s, iops=8000, runt= 1msec
clat (usec): min=123, max=183, avg=150.50, stdev=22.35
lat (usec): min=125, max=184, avg=152.38, stdev=22.08
clat percentiles (usec):
| 1.00th=[ 123], 5.00th=[ 123], 10.00th=[ 123], 20.00th=[ 124],
| 30.00th=[ 139], 40.00th=[ 145], 50.00th=[ 145], 60.00th=[ 155],
| 70.00th=[ 159], 80.00th=[ 177], 90.00th=[ 183], 95.00th=[ 183],
| 99.00th=[ 183], 99.50th=[ 183], 99.90th=[ 183], 99.95th=[ 183],
| 99.99th=[ 183]
lat (usec) : 250=100.00%
cpu : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=28
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=8/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=2048KB, aggrb=2000.0MB/s, minb=2000.0MB/s,
maxb=2000.0MB/s, mint=1msec, maxt=1msec
Disk stats (read/write):
sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
# ls -lh test.tmp
-rw-r--r-- 1 root root 2.0M Apr 27 09:13 test.tmp
# cat test.tmp.log
fio version 2 iolog
test.tmp add
test.tmp open
test.tmp write 0 262144
test.tmp write 262144 262144
test.tmp write 524288 262144
test.tmp write 786432 262144
test.tmp write 1048576 262144
test.tmp write 1310720 262144
test.tmp write 1572864 262144
test.tmp write 1835008 262144
test.tmp close
# for each in {0..7}; do dd if=test.tmp bs=262144 count=1 skip=$each
2>/dev/null | hexdump -C | md5sum; done
71a1660503bcff7c4e20a763d569d069 -
9c9bb7ec1020b4d4249028aecc896e6b -
68b9685812d47c822532854201c9b352 -
e5c8ef471a27ba92b86893ee5ded654b -
14e0e798a8af3f4e6abdaf022ddf91c3 -
85528ae970bd25dde8c39ecaaffa4cf3 -
60b8ccf0e0793094b9356544fb541f3a -
ef736cc9cbf7588cb7b84467cb37c44e -
# fio -v
fio-2.2.7-24-g7c30
