[in the future please refrain from posting to LKML for such a narrow topic like dm-cache... not seeing the point in adding to the LKML noise -- dm-devel should suffice] On Fri, May 10 2013 at 1:51pm -0400, Darrick J. Wong <darrick.wong@xxxxxxxxxx> wrote: > On Fri, May 10, 2013 at 11:22:24AM +0100, Joe Thornber wrote: > > On Thu, May 09, 2013 at 01:47:51PM -0700, Darrick J. Wong wrote: > > > Create a new 'flush' message that causes the dmcache to write all of its > > > metadata out to disk. This enables us to ensure that the disk reflects > > > whatever's in memory without having to tear down the cache device. This helps > > > me in the case where I have a cached ro fs that I can't umount and therefore > > > can't tear down the cache device, but want to save the cache metadata anyway. > > > The command syntax is as follows: > > > > > > # dmsetup message mycache 0 flush now > > > > Nack. > > > > [Ignoring the ugly 'now' parameter.] > > > > I think you're in danger of hiding the real issue. Which is if the > > target's destructor and post suspend is not being called then, as far > > as dm-cache is concerned this is a crash. Any open transactions will > > be lost as it automatically rolls back. > > > > We need to understand more why this is happening. It's actually > > harmless atm for dm-cache, because we're forced to commit before using > > a new migration. But for dm-thin you can lose writes. Why are you > > never tearing down your dm devices? > > afaict, there isn't anything in the initscripts that tears down dm devices > prior to invoking reboot(), and the kernel drivers don't have reboot notifiers > to flush things out either. I've been told that lvm does this, but I don't see > anything in the Ubuntu or RHEL6 that would suggest a teardown script... See: https://git.fedorahosted.org/cgit/lvm2.git/commit/?id=c698ee14bbb1310cf2383c8977d14a8e29139f8c But I'm not sure which distros have hooked blkdeactivate in (cc'ing prajnoha for his insight). > What am I missing? My observation of Ubuntu is that at best it shuts down > services, umounts most of the filesystems, syncs, and reboots. RHEL seems to > shut down multipath and dmcrypt, but that was all I found. For /most/ users of > dm it seems like the system simply reboots, and nobody's the worse for the > wear. DM devices should be properly torn down; as Joe said this is particularly important for dm-thinp (otherwise it looks like a crash and the open transaction is rolled back). > In the meantime I've added a script to my dmcache test tools to tear things > down at the end, which works unless the umount fails. :/ You should switch to using blkdeactivate. > I guess I could simply suspend the devices, but the postsuspend flush > only seems to get called if I actually redefine the device to some > driver that isn't cache. > > (I guess I could suspend the device and replace cache with zero... yuck.) You _really_ shouldn't need to play these games. postsuspend will get called regardless of whether you're changing the table in any way. See: do_suspend -> dm_suspend -> dm_table_postsuspend_targets -> suspend_targets (the only way I'm seeing that the postsuspend could not get called is if the freeze_bdev/thaw_bdev were to fail, via {lock,unlock}_fs()) -- dm-devel mailing list dm-devel@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/dm-devel
