Hi folks, The first of my test VMs that I upgraded to 4.3-rc1 from 4.2 has been behaving rather strangely w.r.t. boot hangs and ext3 filesystems. One the first cold boot of a new kernel, the boot appears to hang. What i've discovered (which took a long time thanks to the shitpile that is systemd) is that it appears to be doing a e2fsck on the root device, and that is failing resulting in systemd outputing: [FAILED] Failed to start File System Check on Root Device. systemd then goes to shutting down the system and reboot but that fail because it's still starting other stuff up and simultaneously shutting shit down tha the stuff starting up depends on. It fucks up badly, with teh last concole entries looking like it has hung waiting for lvm/md devices to appear. I can't tell you what the e2fsck failure is, because systemd oh so helpfully overwrites the console output from fsck with all the other shit that it is doing concurrently. it also fails to log it anywhere because this has happened before A) the console logging has been started and b) the the root fs is still only mounted RO at this point. Hence I have absolutely zero output apart from seeing the "checking xxx%" update on the bottom line of the screen occasionally before it is immediately overwritten by systemd logging something else. So, warm reboot the VM (via system_reset in the qemu console), and the system comes up. No fsck is run, but the filesystem is mounted as ext2, not ext3. What I see is this in dmesg: [ 2.322798] EXT2-fs (sda1): warning: mounting ext3 filesystem as ext2 [ 2.325123] VFS: Mounted root (ext2 filesystem) readonly on device 8:1. It's definitely an ext3 filesystem, but the interesting point is that it has a clean journal so can be mounted as ext2: $ sudo blkid /dev/sda1 /dev/sda1: UUID="b21615e5-fe8a-4ffc-ab80-c24cdc8b740a" SEC_TYPE="ext2" TYPE="ext3" PARTUUID="000efa91-01" $ sudo dumpe2fs -h /dev/sda1 | grep -i feature dumpe2fs 1.42.13 (17-May-2015) Filesystem features: has_journal ext_attr resize_inode dir_index filetype sparse_super large_file Journal features: journal_incompat_revoke $ So, I change the kernel config from: CONFIG_EXT2_FS=y # CONFIG_EXT4_USE_FOR_EXT2 is not set to # CONFIG_EXT2_FS is not set CONFIG_EXT4_USE_FOR_EXT2=y And what I see is this: [ 2.228894] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem [ 2.238832] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) Which tells me that there's a problem with fstype probe ordering regressions w.r.t ext2 and ext3 as a result of removing the ext3 module. It also doesn't fail fsck checks now, so boots successfully every time. I suspect the "boot hang" problem is that e2fsck sees a dirty journal, fixes everything and then asks for a reboot, which fails. So, suspecting this, I switched back to the original CONFIG_EXT2_FS=y build: [ 2.254213] EXT2-fs (sda1): error: couldn't mount because of unsupported optional features (4) [ 2.257076] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem [ 2.259712] EXT4-fs (sda1): INFO: recovery required on readonly filesystem [ 2.261740] EXT4-fs (sda1): write access will be enabled during recovery [ 2.336798] EXT4-fs (sda1): orphan cleanup on readonly fs [ 2.338492] EXT4-fs (sda1): 2 orphan inodes deleted [ 2.339922] EXT4-fs (sda1): recovery complete [ 2.346063] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) [ 2.348349] VFS: Mounted root (ext3 filesystem) readonly on device 8:1. Which makes it pretty clean that if the journal was clean it would have mounted as an ext2 filesystem, not ext3. Basically, we need to ensure that the ext4 module probes filesystems before the ext2 module when CONFIG_EXT2_FS=y is set so that ext3 filesystems are correctly mounted.... Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
