On Thu, May 11, 2023 at 11:56 AM Hannes Reinecke <hare@xxxxxxx> wrote: > > On 5/11/23 20:41, Ming Lin wrote: > > Hi list, > > > > I have an application that needs to use buffered_io to access SMR disk > > for good performance. > > > > From "ZBD Support Restrictions" at https://zonedstorage.io/docs/linux/overview > > " > > Direct IO Writes The kernel page cache does not guarantee that cached > > dirty pages will be flushed to a block device in sequential sector > > order. This can lead to unaligned write errors if an application uses > > buffered writes to write to the sequential write required zones of a > > device. To avoid this pitfall, applications that directly use a zoned > > block device without a file system should always use direct I/O > > operations to write to the sequential write required zones of a > > host-managed disk (that is, they should issue write() system calls > > with a block device "file open" that uses the O_DIRECT flag). > > " > > > > Raw zbd disk only supports direct_io. > > > > Does dm-zoned support buffered io (without O_DIRECT)? > > > Yes. But I _think_ the above paragraph is ever so slightly outdated, as > we've spent quite a lot of time fixing sequential writes (cf blk-zoned > etc). So while dm-zoned is using bufferet writes there won't be any > sequential write issues. > > At least, I have not uncovered any of those during testing. Hi Hannes, I use 5.10.90 kernel and smr disk capacity is 24T. I followed the below guide to create dm_zone device on top of smr disk. https://zonedstorage.io/docs/linux/dm Then mkfs.ext4 /dev/dm-0, but it seems hung. Any ideas? [37552.217472] device-mapper: uevent: version 1.0.3 [37552.217549] device-mapper: ioctl: 4.43.0-ioctl (2020-10-01) initialised: dm-devel@xxxxxxxxxx [37575.608500] device-mapper: zoned metadata: (dmz-5000cca2bfc0db21): DM-Zoned metadata version 2 [37575.608502] device-mapper: zoned metadata: (sdx): Host-managed zoned block device [37575.608503] device-mapper: zoned metadata: (sdx): 50782535680 512-byte logical sectors (offset 0) [37575.608503] device-mapper: zoned metadata: (sdx): 96860 zones of 524288 512-byte logical sectors (offset 0) [37575.608504] device-mapper: zoned metadata: (dmz-5000cca2bfc0db21): 96860 zones of 524288 512-byte logical sectors [37575.609204] device-mapper: zoned: (dmz-5000cca2bfc0db21): Target device: 50771001344 512-byte logical sectors (6346375168 blocks) [38101.543353] INFO: task mkfs.ext4:1411791 blocked for more than 122 seconds. [38101.543380] Tainted: G OE 5.10.90.bm.1-amd64+ #2 [38101.543395] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [38101.543411] task:mkfs.ext4 state:D stack: 0 pid:1411791 ppid:1388660 flags:0x00004000 [38101.543415] Call Trace: [38101.543422] __schedule+0x3fd/0x760 [38101.543425] schedule+0x46/0xb0 [38101.543426] io_schedule+0x12/0x40 [38101.543429] wait_on_page_bit+0x133/0x270 [38101.543431] ? __page_cache_alloc+0xa0/0xa0 [38101.543432] wait_on_page_writeback+0x25/0x70 [38101.543434] __filemap_fdatawait_range+0x86/0xf0 [38101.543435] file_write_and_wait_range+0x74/0xb0 [38101.543438] blkdev_fsync+0x16/0x40 [38101.543441] do_fsync+0x38/0x60 [38101.543442] __x64_sys_fsync+0x10/0x20 [38101.543445] do_syscall_64+0x2d/0x70 [38101.543446] entry_SYSCALL_64_after_hwframe+0x44/0xa9 === Below are the steps I did: root@smr_dev:~# blkzone reset /dev/sdx root@smr_dev:~# dmzadm --format /dev/sdx /dev/sdx: 50782535680 512-byte sectors (24215 GiB) Host-managed device 96860 zones, offset 0 96860 zones of 524288 512-byte sectors (256 MiB) 65536 4KB data blocks per zone Resetting sequential zones Writing primary metadata set Writing mapping table Writing bitmap blocks Writing super block to sdx block 0 Writing secondary metadata set Writing mapping table Writing bitmap blocks Writing super block to sdx block 196608 Syncing disk Done. root@smr_dev:~# dmzadm --start /dev/sdx /dev/sdx: 50782535680 512-byte sectors (24215 GiB) Host-managed device 96860 zones, offset 0 96860 zones of 524288 512-byte sectors (256 MiB) 65536 4KB data blocks per zone sdx: starting dmz-5000cca2bfc0db21, metadata ver. 2, uuid 7495e21a-23d9-49f4-832a-76b32136078b root@smr_dev:~# mkfs.ext4 /dev/dm-0 mke2fs 1.44.5 (15-Dec-2018) Discarding device blocks: done Creating filesystem with 6346375168 4k blocks and 396648448 inodes Filesystem UUID: c47de06d-6cf6-4a85-9502-7830ca2f4526 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848, 512000000, 550731776, 644972544, 1934917632, 2560000000, 3855122432, 5804752896 Allocating group tables: done Writing inode tables: done Creating journal (262144 blocks): done Writing superblocks and filesystem accounting information: === At another terminal, root@smr_dev:~# ps aux | grep mkfs.ext4 root 1411791 2.8 0.0 30992 19864 pts/1 D+ 01:30 0:01 mkfs.ext4 /dev/dm-0 root 1413640 0.0 0.0 13972 2496 pts/0 S+ 01:31 0:00 grep mkfs.ext4 root@smr_dev:~# cat /proc/1411791/stack [<0>] wait_on_page_bit+0x133/0x270 [<0>] wait_on_page_writeback+0x25/0x70 [<0>] __filemap_fdatawait_range+0x86/0xf0 [<0>] file_write_and_wait_range+0x74/0xb0 [<0>] blkdev_fsync+0x16/0x40 [<0>] do_fsync+0x38/0x60 [<0>] __x64_sys_fsync+0x10/0x20 [<0>] do_syscall_64+0x2d/0x70 [<0>] entry_SYSCALL_64_after_hwframe+0x44/0xa9 -- dm-devel mailing list dm-devel@xxxxxxxxxx https://listman.redhat.com/mailman/listinfo/dm-devel
