From: Valerie Aurora <vaurora@xxxxxxxxxx> Document design and implementation of union mounts (a.k.a. writable overlays). With corrections from Andreas Gruenbacher <agruen@xxxxxxx>. Original-author: Valerie Aurora <vaurora@xxxxxxxxxx> Signed-off-by: David Howells <dhowells@xxxxxxxxxx> --- Documentation/filesystems/union-mounts.txt | 712 ++++++++++++++++++++++++++++ 1 files changed, 712 insertions(+), 0 deletions(-) create mode 100644 Documentation/filesystems/union-mounts.txt diff --git a/Documentation/filesystems/union-mounts.txt b/Documentation/filesystems/union-mounts.txt new file mode 100644 index 0000000..596bfe6 --- /dev/null +++ b/Documentation/filesystems/union-mounts.txt @@ -0,0 +1,712 @@ +Union mounts (a.k.a. writable overlays) +======================================= + +This document describes the architecture and current status of union mounts, +also known as writable overlays. + +In this document: + - Overview of union mounts + - Terminology + - VFS implementation + - Locking strategy + - VFS/file system interface + - Userland interface + - NFS interaction + - Status + - Contributing to union mounts + +Overview +======== + +A union mount layers one read-write file system over one or more read-only file +systems, with all writes going to the writable file system. The namespace of +both file systems appears as a combined whole to userland, with files and +directories on the writable file system covering up any files or directories +with matching pathnames on the read-only file system. The read-write file +system is the "topmost" or "upper" file system and the read-only file systems +are the "lower" file systems. A few use cases: + +- Root file system on CD with writes saved to hard drive (LiveCD) +- Multiple virtual machines with the same starting root file system +- Cluster with NFS mounted root on clients + +Most if not all of these problems could be solved with a COW block device or a +clustered file system (include NFS mounts). However, for some use cases, +sharing is more efficient and better performing if done at the file system +namespace level. COW block devices only increase their divergence as time goes +on, and a fully coherent writable file system is unnecessary synchronization +overhead if no other client needs to see the writes. + +What union mounts are not +------------------------- + +Union mounts are not a general-purpose unioning file system. They do not +provide a generic "union of namespaces" operation for an arbitrary number of +file systems. Many interesting features can be implemented with a generic +unioning facility: dynamic insertion and removal of branches, write policies +based on space available, online upgrade, etc. Some unioning file systems that +do this are UnionFS and AUFS. + +Terminology +=========== + +The main physical metaphor for union mounts is that a writable file system is +mounted "on top" of a read-only file system. Lookups start at the "topmost" +read-write file system and travel "down" to the "bottom" read-only file system +only if no blocking entry exists on the top layer. + +Topmost layer: The read-write file system. Lookups begin here. + +Bottom layer: The read-only file system. Lookups end here. + +Path: Combination of the vfsmount and dentry structure. + +Follow down: Given a path from the top layer, find the corresponding path on +the bottom layer. + +Follow up: Given a path from the bottom layer, find the corresponding path on +the top layer. + +Whiteout: A directory entry in the top layer that prevents lookups from +travelling down to the bottom layer. Created on unlink()/rmdir() if a +corresponding directory entry exists in the bottom layer. + +Opaque flag: A flag on a directory in the top layer that prevents lookups of +entries in this directory from travelling down to the bottom layer (unless +there is an explicit fallthru entry allowing that for a particular entry). Set +on creation of any new directory in in the topmost layer (that is, a directory +that does not have any matching visible directory below it). + +Fallthru: A directory entry which allows lookups to "fall through" to the +bottom layer for that exact directory entry. This serves as a placeholder for +directory entries from the bottom layer during readdir(). Fallthrus override +opaque flags. + +File copyup: Create a file on the top layer that has the same metadata and +contents as the file with the same pathname on the bottom layer. + +Directory copyup: Copy up the visible directory entries from the bottom layer +as fallthrus in the matching top layer directory. Mark the directory opaque to +avoid unnecessary negative lookups on the bottom layer. + +Examples +======== + +What happens when I... + +- creat() /newfile -> creates on topmost layer +- unlink() /oldfile -> creates a whiteout on topmost layer +- Edit /existingfile -> copies up to top layer at open(O_WR) time +- truncate /existingfile -> copies up to topmost layer + N bytes if specified +- touch()/chmod()/chown()/etc. -> copies up to topmost layer +- mkdir() /newdir -> creates opaque dir on topmost layer +- rmdir() /olddir -> creates a whiteout on topmost layer +- mkdir() /olddir after above -> creates opaque dir on topmost layer +- readdir() /shareddir -> copies up entries from bottom layer as + fallthrus, processes duplicates and whiteouts +- link() /oldfile /newlink -> copies up /oldfile, creates /newlink on + topmost layer +- symlink() /oldfile /symlink -> nothing special +- rename() /oldfile /newfile -> copies up /oldfile to /newfile on top layer, + whiteouts /oldfile +- rename() /olddir /newdir -> EXDEV +- rename() /topmost_only_dir /topmost_only_dir2 -> success +- stat() /oldfile - inode & dev from lower layer +- stat() /newfile - inode & dev from topmost layer +- readdir() /shareddir - d_ino & d_type from lower layer on fallthrus + +Getting to a root file system with union mounts: + +- Mount the base read-only file system as the root file system +- Mount the read-only file system again on /newroot +- Mount the read-write layer on /newroot: + # mount -o union /dev/sda /newroot +- pivot_root to /newroot +- Start init + +See scripts/pivot.sh in the UML devkit linked to from: + +http://valerieaurora.org/union/ + +VFS implementation +================== + +Union mounts are implemented as an integral part of the VFS, rather than as a +VFS client file system (i.e., a stacked file system like unionfs or ecryptfs). +Implementing unioning inside the VFS eliminates the need for duplicate copies +of VFS data structures, unnecessary indirection, and code duplication, but +requires very maintainable, low overhead code. Union mounts require no change +to file systems serving as the read-only layer, and requires some minor support +from file systems serving as the read-write layer. File systems that want to +be the writable layer must implement the new ->whiteout() and ->fallthru() +inode operations, which create special dummy directory entries. + +The union mounts code must accomplish the following major tasks: + +1) Pass lookups through to the lower level file system. +2) Copy files and directories up to the topmost layer when written. +3) Create whiteouts and fallthrus as necessary. + +VFS objects and union mounts +---------------------------- + +First, some VFS basics: + +The VFS allows multiple mounts of the same file system. For example, /dev/sda +can be mounted at /usr and also at /mnt. The same file system can be mounted +read-only at one point and read-write at another. Each of these mounts has its +own vfsmount data structure in the kernel. However, each underlying file +system has exactly one in-kernel superblock structure no matter how many times +it is mounted. All the separate vfsmounts for the same file system reference +the same superblock data structure. + +Directory entries are cached by the VFS in dentry structures. The VFS keeps +one dentry structure for each file or directory in a file system, no matter how +many times it is mounted. Each dentry represents only one element of a path +name. When the VFS looks up a pathname (e.g., "/sbin/init"), the result is a +combination of vfsmount and dentry. This <mnt,dentry> pair is usually stored +in a kernel structure named "path", which is simply two pointers, one to the +vfsmount and one to the dentry. A "struct path" is this structure; a pathname +is a string like "/etc/fstab". + +In union mounts, a file system can only be the topmost layer for one union +mount. A file system can be part of multiple union mounts if it is a read-only +layer. So dentries in the read-only layers can be part of multiple unions, +while a dentry in the read-write layer can only be part of one unin. + +union_dir structure +--------------------- + +The first job of union mounts is to map directories from the topmost layer to +directories with the same pathname in the lower layer. That is, given the +<mnt,dentry> pair for a directory pathname in the topmost layer, we need to +find all the <mnt,dentry> pairs for the directory with the same pathname in the +lower layer. We do this with the union_dir structure, which is an array +containing struct paths (mnt, dentry pointer pairs) for each directory unioned +with the topmost union. The array is pointed to from the new d_union_stack +member of struct dentry. + +/* + * The union_stack structure. It is an array of struct paths of + * directories below the topmost directory in a unioned directory, The + * topmost dentry has a pointer to this structure. The topmost dentry + * can only be part of one union, so we can reference it from the + * dentry, but lower dentries can be part of multiple union stacks. + * + * The number of dirs actually allocated is kept in the superblock, + * s_union_count. + */ +struct union_stack { + struct path u_dirs[0]; +}; + +This structure is flexible enough to support an arbitrary number of layers of +unioned file systems. Since there can be more than two layers, this section +will talk about mapping "upper" directories to "lower" directories, instead of +"topmost" directories to "bottom" directories. + +Traversing the union stack +-------------------------- + +The set of union_dir structures referring to a particular pathname are called +collectively the union stack for that directory. To traverse the union stack, +iterate through the number of layers in the union (stored in sb->s_union_count) +with union_find_dir(). Example: freeing the union stack: + +void d_free_unions(struct dentry *topmost) +{ + struct path *path; + unsigned int i, layers = topmost->d_sb->s_union_count; + + if (!IS_DIR_UNIONED(topmost)) + return; + + for (i = 0; i < layers; i++) { + path = union_find_dir(topmost, i); + if (path->mnt) + path_put(path); + } + kfree(topmost->d_union_stack); + topmost->d_union_stack = NULL; +} + +Code paths +---------- + +Union mounts modify the following key code paths in the VFS: + +- mount()/umount() +- Pathname lookup +- Any path that modifies an existing file + +Mount +----- + +Union mounts are created in two steps: + +1. Mount the read-only layer file systems read-only in the usual manner, all on +the same mountpoint. Submounts are permitted as long as they are also +read-only and not shared (part of a mount propagation group). + +2. Mount the top layer with the "-o union" option at the same mountpoint. All +read-only file systems mounted at this mountpoint will be included in the union +mount. + +The bottom layers must be read-only and the top layer must be read-write and +support whiteouts and fallthrus. A file system that supports whiteouts and +fallthrus indicates this by setting the MS_WHITEOUT and MS_FALLTHRU flags in +the superblock. Currently, the top layer is forced to "noatime" to avoid a +copyup on every access of a file. Supporting atime with the current +infrastructure would require a copyup on every open(). The "relatime" option +would be equally efficient if the atime is the same or more recent than the +mtime/ctime for every object on the read-only file system, and if the 24-hour +timeout on relatime was disabled. However, this is probably not worthwhile for +the majority of union mount use cases. + +File systems can only be union mounted at their root directories, for +simplicity and performance. + +pivot_root() to a union mounted file system is supported. The recommended way +to get to a union mounted root file system is to boot with the read-only mount +as the root file system, construct the union mount on an entirely new mount, +and pivot_root() to the new union mount root. Attempting to union mount the +root file system later in boot will result in covering other file systems, +e.g., /proc, which isn't permitted in the current code and is a bad idea +anyway. + +Hard read-only file systems +--------------------------- + +Union mounts require the lower layer of the file system to be read-only. +However, in Linux, any individual file system may be mounted at multiple places +in the namespace, and a file system can be changed from read-only to read-write +while still mounted. Thus, simply checking that the bottom layer is read-only +at the time the writable overlay is mounted over it is pointless, since at any +time the bottom layer may become read-write. + +We have to guarantee that a file system will be read-only for as long as it is +the bottom layer of a union mount. To do this, we track the number of hard +read-only users of a file system in its VFS superblock structure. When we +union mount a writable overlay over a file system, we increment its read-only +user count. The file system can only be mounted read-write if its read-only +users count is zero. + +Todo: + +- Support hard read-only NFS mounts. See discussion here: + + http://markmail.org/message/3mkgnvo4pswxd7lp + +Pathname lookup +--------------- + +Pathname lookup in a unioned directory traverses down the union stack for the +parent directory, looking up each pathname element in each layer of the file +system (according to the rules of whiteouts, fallthrus, and opaque flags). At +mount time, the union stack for the root directory of the file system is +created, and the union stack creation for every other unioned directory in the +file system is boot-strapped using the already-existing union stack of the +directory's parent. In order to simplify the code greatly, every visible +directory on the lower file system is required to have a matching directory on +the upper file system. If this matching directory does not already exist, it +is created during pathname lookup. Therefore, each unioned directory is the +child of another unioned directory (or is the root directory of the file +system). + +The actual union lookup function is called in the following code paths: + +do_lookup()->do_union_lookup()->lookup_union()->__lookup_union() +lookup_hash()->lookup_union()->__lookup_union() + +__lookup_union() is where the rules of whiteouts, fallthrus, and opaque flags +are actually implemented. __lookup_union() returns either the first visible +dentry, or a negative dentry from the topmost file system if no matching dentry +exists. If it finds a directory, it looks up any potential matching lower +layer directories. If it finds a lower layer directory, it first creates the +topmost dir if necessary via union_create_topmost_dir(), and then calls +union_add_dir() to append the lower directory to the end of the union stack. + +Note that not all directories in a union mount are unioned, only those with +matching directories on the lower layer. The macro IS_DIR_UNIONED() is a +cheap, constant time way to check if a directory is unioned, while +IS_MNT_UNION() checks if the entire mount is unioned (and therefore whether the +directory in question is potentially unioned). + +Currently, lookup of a negative dentry or a directory with no matching +directories below it requires a lookup in every directory in the union stack +every time it is looked up. We could avoid subsequent lookups by adding the +equivalent of a negative dcache entry. + +File copyup +----------- + +Any system call that alters the data or metadata of a file on the bottom layer, +or creates or changes a hard link to it will trigger a copyup of the target +file from the lower layer to the topmost layer + + - open(O_WRITE | O_RDWR | O_APPEND) + - truncate()/open(O_TRUNC) + - link() + - rename() + - chmod() + - chown()/lchown() + - utimes() + - setxattr()/lsetxattr() + +Copyup of a file due to open(O_WRITE) has already occurred when: + + - write() + - ftruncate() + - writable mmap() + +The following system calls will fail on an fd opened O_RDONLY: + + - fchmod() + - fchown() + - fsetxattr() + - futimensat() + +Contrary to common sense, the above system calls are defined to succeed on +O_RDONLY fds. The idea seems to be that the O_RDONLY/O_RDWR/O_WRITE flags only +apply to the actual file data, not to any form of metadata (times, owner, mode, +or even extended attributes). Applications making these system calls on +O_RDONLY fds are correct according to the standard and work on non-union +mounts. They will need to be rewritten (O_RDONLY -> O_RDWR) to work on union +mounts. We suspect this usage is uncommon. + +This deviation from standard is due to technical limitations of the union mount +implementation. Specifically, we would need to replace an open file descriptor +from the lower layer with an open file descriptor for a file with matching +pathname and contents on the upper layer, which is difficult to do. We avoid +this in other system calls by doing the copyup before the file is opened. +Unionfs doesn't encounter this problem because it creates a dummy file struct +which redirects or fans out operations to the struct files for the underlying +file systems. + +From an application's point of view, the result of an in-kernel file copyup is +the logical equivalent of another application updating the file via the +rename() pattern: creat() a new file, copy the data over, make changes the +copy, and rename() over the old version. Any existing open file descriptors +for that file (including those in the same application) refer to a now +invisible object that used to have the same pathname. Only opens that occur +after the copyup will see updates to the file. + +Permission checks +----------------- + +We want to be sure we have the correct permissions to actually succeed in a +system call before copying a file up to avoid unnecessary IO. At present, the +permission check for a single system call may be spread out over many hundreds +of lines of code (e.g., open()). In order to check permissions, we +occasionally need to determine if there is a writable overlay on top of this +inode. This requires a full path, but often we only have the inode at this +point. In particular, inode_permission() returns EROFS if the inode is on a +read-only file system, which is the wrong answer if there is a writable overlay +mounted on top of it. + +The current solution is to split out the file-system-wide permission checks +from the per-inode permission checks. inode_permission() becomes: + +sb_permission() +__inode_permission() + +inode_permission() calls sb_permission() and __inode_permission() on the same +path. We create path_permission() which calls sb_permission() on the parent +directory from the top layer, and __inode_permission() on the target on the +lower layer. This gets us the correct write permissions consdering that the +file will be copied up. + +Todo: + + - Currently, we don't deal with differing directory permissions at + different levels of the stack. This is a bug. + +Impact on non-union kernels and mounts +-------------------------------------- + +Union-related data structures, extra fields, and function calls are #ifdef'd +out at the function/macro level with CONFIG_UNION_MOUNT in nearly all cases +(see fs/union.h). When CONFIG_UNION_MOUNT is enabled, struct dentry has one +more pointer, reducing the size of dentry names stored in the dentry itself by +4 to 8 bytes. + +Todo: + + - Do performance tests + +Locking strategy +================ + +The current union mount locking strategy is based on the following +rules: + +* The lower layer file system is always read-only +* The topmost file system is always read-write + => A file system can never a topmost and lower layer at the same time + +Additionally, the topmost layer may only be mounted exactly once. Don't think +of the topmost layer as a separate independent file system; when it is part of +a union mount, it is only a file system in conjunction with the read-only +bottom layer. The read-only bottom layer is an independent file system in and +of itself and can be mounted elsewhere, including as the bottom layer for +another union mount. + +Thus, we may define a stable locking order in terms of top layer and bottom +layer locks, since a top layer is never a bottom layer and a bottom layer is +never a top layer. Another simplifying assumption is that all directories in a +pathname exist on the top layer, as they are created step-by-step during +lookup. This prevents us from ever having to walk backwards up the path +creating directory entries, which can get complicated. By implication, parent +directories paths during any operation (rename(), unlink(),etc.) are from the +top layer. Dentries for directories from the bottom layer are only ever seen +or used by the lookup code. + +The two major problems we avoid with the above rules are: + +Lock ordering: Imagine two union stacks with the same two file systems: A +mounted over B, and B mounted over A. Sometimes locks on objects in both A and +B will have to be held simultanously. What order should they be acquired in? +Simply acquiring them from top to bottom will create a lock-ordering problem - +one thread acquires lock on object from A and then tries for a lock on object +from B, while another thread grabs the lock on object from B and then waits for +the lock on object from A. Some other lock ordering must be defined. + +Movement/change/disappearance of objects on multiple layers: A variety of nasty +corner cases arise when more than one layer is changing at the same time. +Changes in the directory topology and their effect on inheritance are of +special concern. Al Viro's canonical email on the subject: + +http://lkml.indiana.edu/hypermail/linux/kernel/0802.0/0839.html + +We don't try to solve any of these cases, just avoid them in the first place. + +Todo: Prevent top layer from being mounted more than once. + +Cross-layer interactions +------------------------ + +The VFS code simultaneously holds references to and/or modifies objects from +both the top and bottom layers in the following cases: + +Path lookup: + +Grabs i_mutex on bottom layer while holding i_mutex on top layer directory +inode. + +File copyup: + +Holds i_mutex on the parent directory from the top layer while copying up file +from lower layer. + +link(): + +File copyup of target while holding i_mutex on parent directory on top layer. +Followed by a normal link() operation. + +rename(): + +Holds s_vfs_rename_mutex on the top layer, i_mutex of the source's parent dir +(top layer), and i_mutex of the target's parent dir (also top layer) while +looking up and copying the bottom layer target and also creating the whiteout. + +Notes on rename(): + +First, renaming of directories returns EXDEV. It's not at all reasonable to +recursively copy directory trees and userspace has to handle this case anyway. +An exception is rename() of directories that exist only on the topmost layer; +this succeeds. + +Rename involves three steps on a union mount: (1) copyup of the file from the +bottom layer, (2) rename of the new top-layer copy to the target in the usual +manner, (3) creation of a whiteout covering the source of the rename. + +Directory copyup: + +Directory entries are copied up on the first readdir(). We hold the top layer +directory i_mutex throughout and sequentially acquire and drop the i_mutex for +each lower layer directory. + +VFS-fs interface +================ + +Read-only layer: No support necessary other than enforcement of really really +read-only semantics (done by VFS for local file systems). + +Writable layer: Must implement two new inode operations: + +int (*whiteout) (struct inode *, struct dentry *, struct dentry *); +int (*fallthru) (struct inode *, struct dentry *); + +And set the MS_WHITEOUT and MS_FALLTHRU flags to indicate support of +these operations. + +Todo: + +- Implement whiteouts and fallthrus in ext3 +- Implement whiteouts and fallthrus in btrfs + +Supported file systems +---------------------- + +Any file system can be a read-only layer. File systems must explicitly support +whiteouts and fallthrus in order to be a read-write layer. This patch set +implements whiteouts for ext2, tmpfs, and jffs2. We have tested ext2, tmpfs, +and iso9660 as the read-only layer. + +Todo: + - Test corner cases of case-insensitive/oversensitive file systems + +NFS interaction +=============== + +NFS is currently not supported as either type of layer. NFS as read-only layer +requires support from the server to honor the read-only guarantee needed for +the bottom layer. To do this, the server needs to revoke access to clients +requesting read-only file systems if the exported file system is remounted +read-write or unmounted (during which arbitrary changes can occur). Some +recent discussion: + +http://markmail.org/message/3mkgnvo4pswxd7lp + +NFS as the read-write layer would require implementation of the ->whiteout() +and ->fallthru() methods. DT_WHT directory entries are theoretically already +supported. + +Also, technically the requirement for a readdir() cookie that is stable across +reboots comes only from file systems exported via NFSv2: + +http://oss.oracle.com/pipermail/btrfs-devel/2008-January/000463.html + +Todo: + +- Guarantee really really read-only on NFS exports +- Implement whiteout()/fallthru() for NFS + +Userland support +================ + +The mount command must support the "-o union" mount option and pass the +corresponding MS_UNION flag to the kerel. A util-linux git tree with union +mount support is here: + +git://git.kernel.org/pub/scm/utils/util-linux-ng/val/util-linux-ng.git + +File system utilities must support whiteouts and fallthrus. An e2fsprogs git +tree with union mount support is here: + +git://git.kernel.org/pub/scm/fs/ext2/val/e2fsprogs.git + +Currently, whiteout directory entries are not returned to userland. While the +directory type for whiteouts, DT_WHT, has been defined for many years, very +little userland code handles them. Userland will never see fallthru directory +entries. + +Known non-POSIX behaviors +------------------------- + +- Any writing system call (unlink()/chmod()/etc.) can return ENOSPC or EIO + + Most programs are not tested and don't work well under conditions of ENOSPC. + The solution is to add more disk space. + +- Link count may be wrong for files on bottom layer with > 1 link count + + A file may have more than one hard link to it. When a file with multiple + hard links is copied up, any other hard links pointing to the same inode will + remain unchanged. If the file is looked up via one of the hard links on the + read-only layer, it will have the original link count (which is off by one at + this point). An example: + + /bin/link1 -> inode 100 + /etc/link2 -> inode 100 + + inode 100 will have link count 2. + + # echo "blah" > /bin/link1 + + Now /bin/link1 will be copied up to the topmost layer. But /etc/link2 will + still point to the original inode 100, and its link count will still be 2. + +- Link count on directories will be wrong before readdir() (fixable) +- File copyup is the logical equivalent of an update via copy + + rename(). Any existing open file descriptors will continue to refer + to the read-only copy on the bottom layer and will not see any + changes that occur after the copy-up. +- rename() of directory may fail with EXDEV +- fchmod()/fchown()/futimensat()/fsetattr() fail on O_RDONLY fds + +Status +====== + +The current union mounts implementation is feature-complete on local file +systems and passes an extensive union mounts test suite, available in the union +mounts Usermode Linux-based development kit: + +http://valerieaurora.org/union/union_mount_devkit.tar.gz + +The whiteout code has had some non-trivial level of review and testing, but +much of the code has had no external review or testing outside the authors' +machines. + +The latest version is available at: + +git://git.kernel.org/pub/scm/linux/kernel/git/val/linux-2.6.git + +Check the union mounts web page for the name of the latest branch: + +http://valerieaurora.org/union/ + +Todo: + +- Run more tests (e.g., XFS test suite) +- Get review from VFS maintainers + +Non-features +------------ + +Features we do not currently plan to support in union mounts: + +Online upgrade: E.g., installing software on a file system NFS exported to +clients while the clients are still up and running. Allowing the read-only +bottom layer of a union mount to change invalidates our locking strategy. + +Recursive copying of directories: E.g., implementing rename() across layers for +directories. Doing an in-kernel copy of a single file is bad enough. +Recursively copying a directory is a big no-no. + +Read-only top layer: The readdir() strategy fundamentally requires the ability +to create persistent directory entries on the top layer file system (which may +be tmpfs). However, you can union two read-only file systems by union mounting +a third file system (such as tmpfs) over the two read-onlly file systems. +Numerous alternatives to this readdir() strategy (including in-kernel or +in-application caching) exist and are compatible with union mounts with its +writing-readdir() implementation disabled. Creating a readdir() cookie that is +stable across multiple readdir()s requires one of: + +- Write to stable storage (e.g., fallthru dentries) +- Non-evictable kernel memory cache (doesn't handle NFS server reboot) +- Per-application caching by glibc readdir() + +Often these features are supported by other unioning file systems or by other +versions of union mounts. + +Contributing to union mounts +============================ + +The union mounts web page is here: + +http://valerieaurora.org/union/ + +It links to: + + - All git repositories + - Documentation + - An entire self-contained UML-based dev kit with README, etc. + +The best mailing list for discussing union mounts is: + +linux-fsdevel@xxxxxxxxxxxxxxx + +http://vger.kernel.org/vger-lists.html#linux-fsdevel + +Thank you for reading! -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
