Ping? On Mon, Sep 21, 2015 at 05:07:15PM +1000, Dave Chinner wrote: > Hi folks, > > The first of my test VMs that I upgraded to 4.3-rc1 from 4.2 has > been behaving rather strangely w.r.t. boot hangs and ext3 > filesystems. > > One the first cold boot of a new kernel, the boot appears to hang. > What i've discovered (which took a long time thanks to the shitpile > that is systemd) is that it appears to be doing a e2fsck on the root > device, and that is failing resulting in systemd outputing: > > [FAILED] Failed to start File System Check on Root Device. > > systemd then goes to shutting down the system and reboot but that > fail because it's still starting other stuff up and simultaneously > shutting shit down tha the stuff starting up depends on. It fucks up > badly, with teh last concole entries looking like it has hung > waiting for lvm/md devices to appear. > > I can't tell you what the e2fsck failure is, because systemd oh so > helpfully overwrites the console output from fsck with all the other > shit that it is doing concurrently. it also fails to log it anywhere > because this has happened before A) the console logging has been > started and b) the the root fs is still only mounted RO at this > point. > > Hence I have absolutely zero output apart from seeing the "checking > xxx%" update on the bottom line of the screen occasionally before it > is immediately overwritten by systemd logging something else. > > So, warm reboot the VM (via system_reset in the qemu console), and > the system comes up. No fsck is run, but the filesystem is mounted > as ext2, not ext3. What I see is this in dmesg: > > [ 2.322798] EXT2-fs (sda1): warning: mounting ext3 filesystem as ext2 > [ 2.325123] VFS: Mounted root (ext2 filesystem) readonly on device 8:1. > > It's definitely an ext3 filesystem, but the interesting point is > that it has a clean journal so can be mounted as ext2: > > $ sudo blkid /dev/sda1 > /dev/sda1: UUID="b21615e5-fe8a-4ffc-ab80-c24cdc8b740a" SEC_TYPE="ext2" TYPE="ext3" PARTUUID="000efa91-01" > $ sudo dumpe2fs -h /dev/sda1 | grep -i feature > dumpe2fs 1.42.13 (17-May-2015) > Filesystem features: has_journal ext_attr resize_inode dir_index filetype sparse_super large_file > Journal features: journal_incompat_revoke > $ > > So, I change the kernel config from: > > CONFIG_EXT2_FS=y > # CONFIG_EXT4_USE_FOR_EXT2 is not set > > to > > # CONFIG_EXT2_FS is not set > CONFIG_EXT4_USE_FOR_EXT2=y > > And what I see is this: > > [ 2.228894] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem > [ 2.238832] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) > > Which tells me that there's a problem with fstype probe ordering > regressions w.r.t ext2 and ext3 as a result of removing the ext3 > module. It also doesn't fail fsck checks now, so boots successfully > every time. I suspect the "boot hang" problem is that e2fsck sees a > dirty journal, fixes everything and then asks for a reboot, which > fails. > > So, suspecting this, I switched back to the original > CONFIG_EXT2_FS=y build: > > [ 2.254213] EXT2-fs (sda1): error: couldn't mount because of unsupported optional features (4) > [ 2.257076] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem > [ 2.259712] EXT4-fs (sda1): INFO: recovery required on readonly filesystem > [ 2.261740] EXT4-fs (sda1): write access will be enabled during recovery > [ 2.336798] EXT4-fs (sda1): orphan cleanup on readonly fs > [ 2.338492] EXT4-fs (sda1): 2 orphan inodes deleted > [ 2.339922] EXT4-fs (sda1): recovery complete > [ 2.346063] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) > [ 2.348349] VFS: Mounted root (ext3 filesystem) readonly on device 8:1. > > Which makes it pretty clean that if the journal was clean it would > have mounted as an ext2 filesystem, not ext3. Basically, we need to > ensure that the ext4 module probes filesystems before the ext2 > module when CONFIG_EXT2_FS=y is set so that ext3 filesystems are > correctly mounted.... > > Cheers, > > Dave. > -- > Dave Chinner > david@xxxxxxxxxxxxx > -- > To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- Dave Chinner david@xxxxxxxxxxxxx -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
