On Tue, Jul 08, 2014 at 11:49:07PM +0200, Arkadiusz Miśkiewicz wrote: > On Tuesday 08 of July 2014, Arkadiusz Miśkiewicz wrote: > > On Tuesday 08 of July 2014, Arkadiusz Miśkiewicz wrote: > > > My broken fs > > > http://ixion.pld-linux.org/~arekm/p2/x1/web2-home.metadump.gz > > > thanks to "xfsprogs: fixes for 3.2.1" by Dave is now almost in good > > > shape. > > > > > > What I found interesting is that repairing it with current git xfs_repair > > > and then mounting (-o usrquota,prjquota) on 3.16.0-rc4 gives me: > > > > > > [32208.142316] XFS (sdb): Mounting V4 Filesystem > > > [32208.205449] XFS (sdb): Ending clean mount > > > [32208.258991] XFS (sdb): Quotacheck needed: Please wait. > > > [32821.930437] XFS (sdb): Quotacheck: Done. > > > > > > Then I umount it and mount again (same options as before) but on 3.10.43 > > > kernel: > > > > > > [ 111.325889] XFS (sdb): Mounting Filesystem > > > [ 111.419331] XFS (sdb): Failed to initialize disk quotas. > > > [ 111.419339] XFS (sdb): Ending clean mount > > > > > > What did change in later kernels that could have meaning here? (so I > > > could try to backport that change). > > > > > > For testing I mounted again on 3.16 and: > > > [33870.472769] XFS (sdb): Mounting V4 Filesystem > > > [33870.543539] XFS (sdb): Ending clean mount > > > [33870.597791] XFS (sdb): Quotacheck needed: Please wait. > > > [34484.332879] XFS (sdb): Quotacheck: Done > > > > > > and again on 3.10.43: > > > [ 1649.215390] XFS (sdb): Mounting Filesystem > > > [ 1649.316894] XFS (sdb): Failed to initialize disk quotas. > > > [ 1649.316902] XFS (sdb): Ending clean mount > > > > Fails in xfs_iget at xfs_qm_init_quotainos(): > > > > if (XFS_IS_OQUOTA_ON(mp) && > > mp->m_sb.sb_gquotino != NULLFSINO) { > > ASSERT(mp->m_sb.sb_gquotino > 0); > > if ((error = xfs_iget(mp, NULL, > > mp->m_sb.sb_gquotino, 0, 0, &gip))) { > > if (uip) > > IRELE(uip); > > return XFS_ERROR(error); > > } > > } > > > > and in xfs_iget: > > > > /* reject inode numbers outside existing AGs */ > > if (!ino || XFS_INO_TO_AGNO(mp, ino) >= mp->m_sb.sb_agcount) > > return EINVAL; > > > > > > m_sb.sb_gquotino/ino looks to be 0 > > > > # xfs_db /dev/sdb -c "sb 0" -c "print" |grep quot > > uquotino = 4077961 > > gquotino = 0 > > pquotino = 0 > > > > and older kernels (testing on 3.10.46 now) seem to be unable to deal with > > this. > > > > Newer kernels survive probably due to this: > > > > xfs_sb_quota_from_disk(struct xfs_sb *sbp) > > { > > /* > > * older mkfs doesn't initialize quota inodes to NULLFSINO. This > > * leads to in-core values having two different values for a quota > > * inode to be invalid: 0 and NULLFSINO. Change it to a single > > value * NULLFSINO. > > * > > * Note that this change affect only the in-core values. These > > * values are not written back to disk unless any quota information > > * is written to the disk. Even in that case, sb_pquotino field is > > * not written to disk unless the superblock supports pquotino. > > */ > > if (sbp->sb_uquotino == 0) > > sbp->sb_uquotino = NULLFSINO; > > if (sbp->sb_gquotino == 0) > > sbp->sb_gquotino = NULLFSINO; > > if (sbp->sb_pquotino == 0) > > sbp->sb_pquotino = NULLFSINO; > > > > so sb_gquotino = NULLFSINO gets set and it never reaches error condition at > > xfs_qm_init_quotainos() > > Small summary from my observation: > > - mkfs.xfs sets all three [ugp]quotino to 0 (contrary to what comment above > says: "older mkfs doesn't initialize quota inodes to NULLFSINO.". New/fresh > mkfs.xfs also doesn't do that. Were any intermediate versions actually setting > NULLFSINO ? Not yet. Mainly because the handling of quota in superblocks appears to be broadly broken in the xfsprogs code base. > - freshly created fs; 3.10 kernel; mount image -o usrquota causes: uquotino > gets NEW inode; gquotino is set to null; pquotino not used (stays at 0) That is triggered by the XFS_SB_VERSION_QUOTABIT not being set, and xfs_qm_qino_alloc() setting the incore values quotaino values to NULLFSINO. > - 3.16 behaviour is different; freshly created fs; mount image -o usrquota > causes: uquotino gets NEW inode; gquotino/pquotino stays at 0 (while on 3.10 > gquotino was being set to null) Same thing, except we never write the sb_gquotino because group quota is not enabled. That's a bug in the 3.16 kernel code that needs fixing. > - 3.10 kernel is not able to handle case when uquotino == value, gquotino == > 0. For 3.10 this case is impossible / should never happen. 3.10 expects > (uquotino == value, gquotino == null) or (uquotino == value, gquotino == > othervalue) or (uqotinfo == null, gruotino == value) only. Only when the XFS_SB_VERSION_QUOTABIT feature bit is set. If it is not set, then the code will first set them to NULLFSINO as per the above "new fs without quota" case. > - this means that fresh fs; mount under 3.16 with -o usrquota. Try to mount > the same fs on 3.10 but with -o usrquota,grpquota -> will fail "Failed to > initialize disk quotas." Right, that's the 3.16 issue, and so the fix will need to go to -stable kernels. > - 3.16 handles both cases (uquotino == val, gquotino == 0, too) because it > does translation when reading from disk (the one cited at beginning of this > mail) Right, which is necessary for the code to handle the optional project quota correctly. > Probably solution is to make xfs_repair make if [ugp]quotino == 0 set these > to null. IMO old and new kernels should handle null properly (regardless of > quota feature bit state). mkfs.xfs should also get updated to set these to > null initially. New kernels already handle the mixed state just fine, old kernels don't. New kernels need to be fixed not to leave mixed state on unmount. Mkfs can remain doing what it currently does - it's not leaving mixed state behind. Repair, well, it's just broken when it comes to quotas. It doesn't actually check quotas, or even check the validity of the dquots in the quota files. It doesn't leave the quota inodes in consistent state, nor does it handle the quota version bits appropriately. What I think I'll do is finish the quota lobotomisation patch I have for repair - it just nulls all the quota inodes, clears the quota flags and marks the quota inodes as unused. It the removes all the knowledge of quota and quota inodes from repair. This will ensure that quotas are always in a known state after repair, and they will always be rebuilt on the next mount. Note that repair currently always clears the quota checked flags even if there was nothing wrong with the filesystem so that quotas are regenerated on the next mount. Hence jsut trashing everything quota related is not going to change behaviour at all. > Last question/issue is: should mount fail in case of "Failed to initialize > disk quotas." problem? Right now user requests quotas to be enabled but gets > fs without quota (and the only failure indication is one line in dmesg). Not according to history. Any quota mount failure results in quotas being disabled until the problem is corrected... > Citation: > 23:24 < dchinner__> arekm:it's just a nasty can of worms Yup, I have to fix multiple kernel versions as well as xfs_repair. Not to mention it's clear that the xfsprogs change-over to support the seprate project quota inode was fundamentally broken, and so I have to audit everything else in xfsprogs that reads and writes superblocks and work out how to fix the problems that exist. Cheers, Dave. -- Dave Chinner david@xxxxxxxxxxxxx _______________________________________________ xfs mailing list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
