On Tue, Jan 04, 2011 at 09:46:39AM -0800, Patrick J. LoPresti wrote:
> Hey, what's the worst that could happen?
That's just asking for trouble. ;)
> I recently learned that some of my colleagues have configured two
> Linux systems to simultaneously mount a single XFS partition residing
> on shared storage. Specifically, "system R" has the partition mounted
> read-only while "system W" has it mounted read/write.
>
> I told them that this sounds like a very bad idea because XFS is not a
> clustered file system. But they are skeptical because "it seems to be
> working fine". I need to know what the actual risks are and whether
> they can be mitigated.
Ok, so it will appear to work fine most of the time...
> This partition holds large amounts of essentially archival data; that
> is, it is read frequently but written rarely. When they do want to
> write to it, they do so via system W and then reboot system R.
You could probably just run "echo 3 > /proc/sys/vm/drop_caches" or
just umount/mount the device again to get the same effect as
rebooting.
> I am no expert on XFS, but there are essentially two risks that I can see:
>
> Risk 1: When making changes via system W, the view of the file system
> from system R can become corrupted or inconsistent. My colleagues are
> aware of this and believe they can live with it, as long as the
> underlying file system is not being damaged ("we can just reboot").
Yup, so long as system R does not cache anything, or the caches are
dropped after system W writes, you should be fine. However, there is a
window between system W starting to write and system R
being rebooted that system R could read inconsistent metadata and/or
data. There's not much you can do about that apart from take system
R offline while system W is writing.
> Risk 2: Any time the file system is mounted, even read-only, it will
> replay the journal if it is non-empty. (At least, I believe this is
> true. Could one of you please confirm or deny?) So if machine R
> should reboot while the journal is non-empty, it will replay it,
> causing fairly unpredictable on-disk corruption.
Yup.
> Here are my questions.
>
> 1) When can a read-only XFS mount write to the disk, exactly?
Log recovery only. Use mount -o ro,norecovery to avoid that.
> 2) If I do a "sync" on machine W (and perform no further writes), will
> that truncate the journal?
FYI, the journal cannot be truncated - it is a fixed size circular
log.
To get the log clean, I'd freeze the filesystem on system W while
system R mounts. e.g:
system W system R
unmount <fs>
write data
freeze fs
mount -o ro,norecovery <fs>
unfreeze fs
> 3) What am I missing?
1. NFS/CIFS. No need for shared access to the block device. NFs
works pretty well for read only access, especially if you put a
dedicated 10GbE link between the two machines...
2. Snapshots. If you must share the block device, snapshot the
active filesystem and mount that readonly on system R - the snapshot
will be unchanging. When system W knows a snapshot is unmounted and
finished with, it can delete it. That is:
system W system R
write data
....
write data
snapshot <fs.ss2>
umount <fs.ss1>
mount -o ro,norecovery <fs.ss2>
delete snapshot <fs.ss1>
....
write data
.....
write data
snapshot <fs.ss3>
umount <fs.ss2>
mount -o ro,norecovery <fs.ss3>
delete snapshot <fs.ss2>
....
Cheers,
Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
