We've finally resolved this issue; it had nothing to do with LVM. I tried upgrading to the LVM2 2.00.09 and device-mapper 1.00.09 but still had occasional orphaned inodes when we mounted the snapshot. I then tried the unstable patches from people.sistina.com/~thornber/dm; this didn't fix the problem either. I finally tried using lsof to see what was open on the origin logical volume. This showed us that mysqld still had a log file open: mysqld 1198 root 5w REG 254,1 2978 32771 /opt/data/mysql/mysqldb/slow.log.1 (deleted) Because slow.log had been logrotated and compressed, the inode was in fact an orphan. By adding 'mysqladmin flush-logs' to the postrotate section of our mysql logrotate file, the problem went away. We've now reverted back to LVM2 2.00.09 and device-mapper 1.00.09. Thanks to Joe and Steve for your suggestions. On Mon, 15 Mar 2004 18:23:56 -0700 "Donald Murray P.Eng." <donald.murray@pason.com> wrote: > Problem: > LVM2 snapshots of an EXT3 partition do not appear to be taken > from a quiesced lv. In /var/log/messages, we see messages about > an orphaned inode. > > Details: > We're using LVM2 on a RedHat 7.2 box with a 2.4.22 kernel from kernel.org. > The kernel has been patched with the following patches: > linux-2.4.22-VFS-lock.patch > linux-2.4.22-devmapper-ioctl.patch > > Once per hour, we use an LVM snapshot to backup a MySQL database. The > procedure is: > 1. mysql server: flush tables with read lock; hold this lock until step 5 > 2. sync > 3. lvcreate --extents N --snapshot --name dbsnapshotlv /dev/datavg/datalv > 4. mount -o ro /dev/datavg/dbsnapshotlv /mnt/snapshot > 5. mysql server: unlock tables > 6. create backup of /mnt/snapshot > 7. umount /mnt/snapshot && lvremove /dev/datavg/dbsnapshotlv > > This appears to be working, but there is a message in /var/log/messages > which is a little disturbing. Whenever we create a snapshot and mount > the resulting lv, we see the following: > > Mar 15 05:00:05 PC kernel: EXT3-fs: INFO: recovery required on readonly filesystem. > Mar 15 05:00:05 PC kernel: EXT3-fs: write access will be enabled during recovery. > Mar 15 05:00:05 PC kernel: kjournald starting. Commit interval 5 seconds > Mar 15 05:00:05 PC kernel: EXT3-fs: device-mapper(254,4): orphan cleanup on readonly fs > Mar 15 05:00:05 PC kernel: EXT3-fs: device-mapper(254,4): 1 orphan inode deleted > Mar 15 05:00:05 PC kernel: EXT3-fs: recovery complete. > Mar 15 05:00:05 PC kernel: EXT3-fs: mounted filesystem with ordered data mode. > ... > Mar 15 06:00:05 PC kernel: EXT3-fs: INFO: recovery required on readonly filesystem. > Mar 15 06:00:05 PC kernel: EXT3-fs: write access will be enabled during recovery. > Mar 15 06:00:07 PC kernel: kjournald starting. Commit interval 5 seconds > Mar 15 06:00:07 PC kernel: EXT3-fs: device-mapper(254,4): orphan cleanup on readonly fs > Mar 15 06:00:07 PC kernel: EXT3-fs: device-mapper(254,4): 1 orphan inode deleted > Mar 15 06:00:07 PC kernel: EXT3-fs: recovery complete. > Mar 15 06:00:07 PC kernel: EXT3-fs: mounted filesystem with ordered data mode. > ... > Mar 16 01:00:06 PC kernel: EXT3-fs: INFO: recovery required on readonly filesystem. > Mar 16 01:00:06 PC kernel: EXT3-fs: write access will be enabled during recovery. > Mar 16 01:00:08 PC kernel: kjournald starting. Commit interval 5 seconds > Mar 16 01:00:08 PC kernel: EXT3-fs: device-mapper(254,4): orphan cleanup on readonly fs > Mar 16 01:00:08 PC kernel: EXT3-fs: device-mapper(254,4): 1 orphan inode deleted > Mar 16 01:00:08 PC kernel: EXT3-fs: recovery complete. > Mar 16 01:00:08 PC kernel: EXT3-fs: mounted filesystem with ordered data mode. > > > When we mount the snapshot read-write, we instead see the following: > > Mar 16 01:09:17 PC kernel: kjournald starting. Commit interval 5 seconds > Mar 16 01:09:17 PC kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on device-mapper(254,4), internal journal > Mar 16 01:09:17 PC kernel: EXT3-fs: device-mapper(254,4): 1 orphan inode deleted > Mar 16 01:09:17 PC kernel: EXT3-fs: recovery complete. > Mar 16 01:09:17 PC kernel: EXT3-fs: mounted filesystem with ordered data mode. > > > All we expected to see in /var/log/messages was something more like: > > Mar 17 01:09:03 PC kernel: kjournald starting. Commit interval 5 seconds > Mar 17 01:09:03 PC kernel: EXT3 FS 2.4-0.9.19, 19 August 2002 on device-mapper(254,4), internal journal > Mar 17 01:09:03 PC kernel: EXT3-fs: mounted filesystem with ordered data mode. > > > Why would the snapshot volume have an orphaned inode? Doesn't the VFS patch > allow us to snapshot EXT3? Do we need to quiesce the origin lv some other > way? > > > > ------------------------------------------ > Donald Murray, P.Eng. > Senior Software Engineer > Pason Systems Corp. > Direct: (403) 301-3473 > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ------------------------------------------ Donald Murray, P.Eng. Senior Software Engineer Pason Systems Corp. Direct: (403) 301-3473 _______________________________________________ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
