Hello Mr Tomonori!
FUJITA Tomonori schrieb:
- the target box got the data from the initiator box and wrote it to
disk properly.
- the target box reads the data and and sends it to the initiator
properly on the first run after restarting tgtd.
- then the target box sends the wrong data after that.
Right?
Perfectly summarised.
How about writing twice? The target can still store the data (which
written on the second run) on disk?
Multiple writes are no problem.
Restartet tgtd:
target:~#pkill -9 tgtd
target:~#tgtd
target:~#tgtadm --lld iscsi --op new --mode target --tid 1 -T
de.inqbus.athene:test
target:~#tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1
-b /dev/vg0/test
target:~#tgtadm --lld iscsi --op bind --mode target --tid 1 -I 10.1.3.0/24
Clearing target partition: #1
target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 0,633535 s, 1,7 GB/s
Sending Data #1
initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1
1000.0000 MB in 1.0469 secs, 955.2348 MB/sec
Checking Data #1
target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 0.9031 secs, 1107.2542 MB/sec
Clearing patition #2
target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 0,736639 s, 1,4 GB/s
Sending Data #2
initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1
1000.0000 MB in 1.0115 secs, 988.5917 MB/sec
checking data #2
target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 0.8151 secs, 1226.7726 MB/sec
clearing data #3
target:~# dd if=/dev/zero of=/dev/vg0/test bs=1M count=1000
1000+0 Datensätze ein
1000+0 Datensätze aus
1048576000 Bytes (1,0 GB) kopiert, 0,622108 s, 1,7 GB/s
Sending data #3
initiator:~# lmdd if=internal of=/dev/sdc opat=1 bs=1M count=1000 mismatch=1
1000.0000 MB in 1.0047 secs, 995.3487 MB/sec
Checking data #3
target:~# lmdd of=internal if=/dev/vg0/test ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 0.8914 secs, 1121.8673 MB/sec
But reading fails (sometimes):
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 2.8950 secs, 345.4241 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 2.9058 secs, 344.1340 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 2.9133 secs, 343.2515 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 2.8820 secs, 346.9850 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000
mismatch=10
1000.0000 MB in 2.7672 secs, 361.3819 MB/sec
initiator:~# lmdd of=internal if=/dev/sdc ipat=1 bs=1M count=1000
mismatch=10
off=44000000 want=2ac0000 got=2ac1000
off=44000000 want=2ac0004 got=2ac1004
off=44000000 want=2ac0008 got=2ac1008
off=44000000 want=2ac000c got=2ac100c
off=44000000 want=2ac0010 got=2ac1010
off=44000000 want=2ac0014 got=2ac1014
off=44000000 want=2ac0018 got=2ac1018
off=44000000 want=2ac001c got=2ac101c
off=44000000 want=2ac0020 got=2ac1020
off=44000000 want=2ac0024 got=2ac1024
44.0000 MB in 0.1243 secs, 353.8798 MB/sec
Statistically 1 out of 5 reads fail.
It seems that the frequency of the reading failures depends somehow on
timing.
If I give the system some time after a read error then the probability
that the next read succeeds rises.
Since the readingfails in a non deterministic manner - this faillure
maybe lies deeper than stgt/open-iscsi layer.
I will give scst/SRP a try to check if the RDMA-Layer is ok.
Do you have another idea what I may test?
Best regards
Volker
--
====================================================
inqbus it-consulting +49 ( 341 ) 5643800
Dr. Volker Jaenisch http://www.inqbus.de
Herloßsohnstr. 12 0 4 1 5 5 Leipzig
N O T - F Ä L L E +49 ( 170 ) 3113748
====================================================
--
To unsubscribe from this list: send the line "unsubscribe stgt" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
