On Wed, Jan 8, 2020 at 11:41 PM Sitsofe Wheeler <sitsofe@xxxxxxxxx> wrote: > > On Wed, 8 Jan 2020 at 19:19, smitha sunder <sundersmitha@xxxxxxxxx> wrote: > > > > How ever if I format the drive with LBAF type 4 that is supported by > > the drive, then I get the following error : > > > > # nvme format /dev/nvme1 --namespace-id=1 --lbaf=4 --pi=1 --ms=0 > > Success formatting namespace:1 > > # nvme list > > Node SN Model > > Namespace Usage Format FW Rev > > ---------------- -------------------- > > ---------------------------------------- --------- > > -------------------------- ---------------- -------- > > /dev/nvme0n1 160401RICHC01 PM1733V4TLC > > 1 1.92 TB / 1.92 TB 4 KiB + 64 B TEMC8881 > > /dev/nvme1n1 160401RICHC01 PM1733V4TLC > > 1 1.92 TB / 1.92 TB 4 KiB + 64 B TEMC8881 > > > > > > # fio --filename=/dev/nvme0n1 --size=1M --rw=write --ioengine=libaio > > --direct=1 --verify=crc32 --verify_dump=1 --name=test --bs=4096 > > test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) > > 4096B-4096B, ioengine=libaio, iodepth=1 > > fio-3.7 > > Starting 1 process > > fio: io_u error on file /dev/nvme0n1: Invalid or incomplete multibyte > > or wide character: write offset=4096, buflen=4096 > > fio: pid=32709, err=84/file:io_u.c:1747, func=io_u error, > > error=Invalid or incomplete multibyte or wide character > > [...] > > > Am I missing something? > > Thanks for the help ! > > All fio is saying is that it didn't get back the data that it wrote > down on the second block but I don't know what using "LBAF type 4" > might do to your disk (you are saying it changes the logical block > size but does it change anything else?). Do you don't have anything > else that might be writing to that the device too and might have > interfered with the data? What happens if you add --offset=1M? > > -- > Sitsofe | http://sucs.org/~sits/ Thanks Sitsofe for your reply and your time. The drive supports various LBA formats : LBA Format 0 : Metadata Size: 0 bytes - Data Size: 512 bytes - Relative Performance: 0x1 Better LBA Format 1 : Metadata Size: 8 bytes - Data Size: 512 bytes - Relative Performance: 0x2 Good LBA Format 2 : Metadata Size: 0 bytes - Data Size: 4096 bytes - Relative Performance: 0 Best LBA Format 3 : Metadata Size: 8 bytes - Data Size: 4096 bytes - Relative Performance: 0x2 Good (in use) LBA Format 4 : Metadata Size: 64 bytes - Data Size: 4096 bytes - Relative Performance: 0x2 Good So LBAF 4 is 4K formmated with 64B metadata Nothing else is running on the device. The error is seen even if I issue offset=1M or 50% etc. # fio --filename=/dev/nvme0n1 --rw=write --ioengine=libaio --direct=1 --verify=crc32 --verify_dump=1 --name=test --bs=4096 --offset=1M test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 fio-3.7 Starting 1 process fio: io_u error on file /dev/nvme0n1: Invalid or incomplete multibyte or wide character: write offset=1048576, buflen=4096 test: No I/O performed by libaio, perhaps try --debug=io option for details? fio: pid=3392, err=84/file:io_u.c:1747, func=io_u error, error=Invalid or incomplete multibyte or wide character test: (groupid=0, jobs=1): err=84 (file:io_u.c:1747, func=io_u error, error=Invalid or incomplete multibyte or wide character): pid=3392: Thu Jan 9 09:28:24 2020 cpu : usr=0.00%, sys=0.00%, ctx=1, majf=0, minf=21 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=50.0%, 4=50.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=0,1,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): Disk stats (read/write): nvme0n1: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00% If I format the drive to 1KiB+8B metadata then fio runs fine : # nvme list Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 160401RICHC01 PM1733V4TLC 1 1.92 TB / 1.92 TB 4 KiB + 8 B TEMC8881 /dev/nvme1n1 160401RICHC01 PM1733V4TLC 1 1.92 TB / 1.92 TB 4 KiB + 8 B TEMC8881 # fio --filename=/dev/nvme0n1 --rw=write --ioengine=libaio --direct=1 --verify=crc32 --verify_dump=1 --name=test --bs=4096 --offset=1M --size=1G test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=1 fio-3.7 Starting 1 process Jobs: 1 (f=1): [V(1)][100.0%][r=43.5MiB/s,w=0KiB/s][r=11.1k,w=0 IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=3486: Thu Jan 9 09:30:19 2020 read: IOPS=11.2k, BW=43.6MiB/s (45.7MB/s)(1024MiB/23485msec) slat (nsec): min=1496, max=5603.7k, avg=1745.06, stdev=11027.69 clat (usec): min=35, max=517, avg=78.56, stdev=13.52 lat (usec): min=37, max=5672, avg=80.33, stdev=17.43 clat percentiles (usec): | 1.00th=[ 63], 5.00th=[ 73], 10.00th=[ 73], 20.00th=[ 73], | 30.00th=[ 74], 40.00th=[ 75], 50.00th=[ 75], 60.00th=[ 75], | 70.00th=[ 89], 80.00th=[ 89], 90.00th=[ 90], 95.00th=[ 90], | 99.00th=[ 90], 99.50th=[ 91], 99.90th=[ 347], 99.95th=[ 375], | 99.99th=[ 400] write: IOPS=27.2k, BW=106MiB/s (112MB/s)(1024MiB/9625msec) slat (nsec): min=11232, max=68549, avg=12131.36, stdev=469.35 clat (usec): min=18, max=372, avg=24.25, stdev= 4.91 lat (usec): min=29, max=384, avg=36.41, stdev= 4.95 clat percentiles (nsec): | 1.00th=[20352], 5.00th=[20352], 10.00th=[20608], 20.00th=[21120], | 30.00th=[22144], 40.00th=[22144], 50.00th=[22400], 60.00th=[22656], | 70.00th=[23168], 80.00th=[24448], 90.00th=[33536], 95.00th=[34048], | 99.00th=[36096], 99.50th=[38656], 99.90th=[48384], 99.95th=[54016], | 99.99th=[74240] bw ( KiB/s): min=28447, max=114880, per=96.25%, avg=104854.35, stdev=18105.13, samples=20 iops : min= 7111, max=28720, avg=26213.65, stdev=4526.47, samples=20 lat (usec) : 20=0.01%, 50=50.05%, 100=49.86%, 250=0.01%, 500=0.07% lat (usec) : 750=0.01% cpu : usr=17.96%, sys=3.42%, ctx=524342, majf=0, minf=6158 IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwts: total=262144,262144,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: bw=43.6MiB/s (45.7MB/s), 43.6MiB/s-43.6MiB/s (45.7MB/s-45.7MB/s), io=1024MiB (1074MB), run=23485-23485msec WRITE: bw=106MiB/s (112MB/s), 106MiB/s-106MiB/s (112MB/s-112MB/s), io=1024MiB (1074MB), run=9625-9625msec Disk stats (read/write): nvme0n1: ios=262222/262144, merge=0/0, ticks=19871/6270, in_queue=0, util=99.61% Thanks smitha
