Re: FIO - dedup checksums or specified blocksize does not match.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On 04/27/2015 07:18 AM, Srinivasa Chamarthy wrote:
I was just verifying if i could generate 100% duplicable data with
FIO. I have configured small workload with bs of 256k and writing 2MB
of file. I tried to get the checksum of each of 256k blocks of data
from the file and the checksums do not match. If i am not wrong, when
i specify data as 100% deduppable, my checksums should match isn't it?

# cat ddp_file.fio
[dedupe]
filename=test.tmp
bs=256k
rw=write
size=2m
dedupe_percentage=100
write_iolog=test.tmp.log

# fio ddp_file.fio
dedupe: (g=0): rw=write, bs=256K-256K/256K-256K/256K-256K,
ioengine=sync, iodepth=1
fio-2.2.7-24-g7c30
Starting 1 process
dedupe: Laying out IO file(s) (1 file(s) / 2MB)

dedupe: (groupid=0, jobs=1): err= 0: pid=31497: Mon Apr 27 09:13:35 2015
   write: io=2048.0KB, bw=2000.0MB/s, iops=8000, runt=     1msec
     clat (usec): min=123, max=183, avg=150.50, stdev=22.35
      lat (usec): min=125, max=184, avg=152.38, stdev=22.08
     clat percentiles (usec):
      |  1.00th=[  123],  5.00th=[  123], 10.00th=[  123], 20.00th=[  124],
      | 30.00th=[  139], 40.00th=[  145], 50.00th=[  145], 60.00th=[  155],
      | 70.00th=[  159], 80.00th=[  177], 90.00th=[  183], 95.00th=[  183],
      | 99.00th=[  183], 99.50th=[  183], 99.90th=[  183], 99.95th=[  183],
      | 99.99th=[  183]
     lat (usec) : 250=100.00%
   cpu          : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=28
   IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
      issued    : total=r=0/w=8/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
      latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   WRITE: io=2048KB, aggrb=2000.0MB/s, minb=2000.0MB/s,
maxb=2000.0MB/s, mint=1msec, maxt=1msec

Disk stats (read/write):
   sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

# ls -lh test.tmp
-rw-r--r-- 1 root root 2.0M Apr 27 09:13 test.tmp

# cat test.tmp.log
fio version 2 iolog
test.tmp add
test.tmp open
test.tmp write 0 262144
test.tmp write 262144 262144
test.tmp write 524288 262144
test.tmp write 786432 262144
test.tmp write 1048576 262144
test.tmp write 1310720 262144
test.tmp write 1572864 262144
test.tmp write 1835008 262144
test.tmp close

# for each in {0..7}; do dd if=test.tmp bs=262144 count=1 skip=$each
2>/dev/null | hexdump -C | md5sum; done
71a1660503bcff7c4e20a763d569d069  -
9c9bb7ec1020b4d4249028aecc896e6b  -
68b9685812d47c822532854201c9b352  -
e5c8ef471a27ba92b86893ee5ded654b  -
14e0e798a8af3f4e6abdaf022ddf91c3  -
85528ae970bd25dde8c39ecaaffa4cf3  -
60b8ccf0e0793094b9356544fb541f3a  -
ef736cc9cbf7588cb7b84467cb37c44e  -

# fio -v
fio-2.2.7-24-g7c30

Can you try with current -git? The corner cases of being 100% dedupable was broken.

--
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe fio" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [Linux Kernel]     [Linux SCSI]     [Linux IDE]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux SCSI]

  Powered by Linux