On Fri, Apr 23, 2010 at 05:20:05PM -0400, Ryan Lynch wrote: > Would it be possible to run a shared-disk, clustered filesystem over a > dm-crypt block device, which in turn runs on a shared iSCSI device? > I'd be interested in knowing if anyone has tried, or has theoretical > knowledge of why it would (not) work. Thoretically, it should work. A block device is a block device. The question becomes performance as performance characteristics cannot be hidden beind an uniform block device interface. > Multiple Linux machines can simultaneously R/W mount a clustered > filesystem (like GFS/GFS2 or OCFS). Performance can suffer when > multiple hosts write in parallel, but otherwise it works pretty well > (in my limited experience with GFS2). All of the participating hosts > need some kind of shared access directly to the same block device: > I've used iSCSI and DRBD, but I think FC is common, too. Each host > also runs a DLM (distributed lock manager) daemon, which sends and > receives info about which hosts are writing to which inodes, so they > can all keep their caches consistent with the disk. (better > explanation on WP, here: > http://en.wikipedia.org/wiki/Global_File_System#Differences_from_a_local_filesystem) > > From my limited understanding of how iSCSI, GFS2 work, and dm-crypt > work, I have no idea how they'll cooperate with each other, though. > I'd like to test it, when I get a chance, but it would be nice to know > a little more, in advance. As I said, GFS2 does not care whether the block device it runs on is directly hardware or in any way transformed by LVM/dm-crypt, whatever. The only practical experience I have is RAID1/5/6 -> dm-crypt -> ext3 -> NFS export. I have not noticed any additional problems there in several years of doing it. There are of course the individual problems of the layers. E.g. slow access because files are heavily fragmented (don't ask) or NFS (old version) being unreliable when not used with TCP. You can get pretty bad "synergies" even locally though. Once I had XFS on a Linux software RADI5 and they interacted so badly when both did a resync/FS check, that it would have taken months to finish. Basically both seemd to work, then backed off, then tried again, backed off.... The RAID resync speed was arounf 10kB/sec. I think unless somebody has the exact same configuration and workload as you do, you really need to try ot out. One thing though: If this works currently, and you just want to put in the dm-crypt layer, chances are it is still going to work with dm-cryopt. Arno > Ryan B. Lynch > ryan.b.lynch@xxxxxxxxx > _______________________________________________ > dm-crypt mailing list > dm-crypt@xxxxxxxx > http://www.saout.de/mailman/listinfo/dm-crypt > -- Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno@xxxxxxxxxxx GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F ---- Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans If it's in the news, don't worry about it. The very definition of "news" is "something that hardly ever happens." -- Bruce Schneier _______________________________________________ dm-crypt mailing list dm-crypt@xxxxxxxx http://www.saout.de/mailman/listinfo/dm-crypt
