On Jun 22, 2006 09:43 -0700, Jeff Anderson-Lee wrote: > I'm part of a project at University of California Berkeley that is > trying to put together a predominantly archival file system for petabyte > class data stores using Linux with clusters of commodity server > hardware. We currently have multiple terabytes of hardware on top of > which we intend to build such a system. However, our hope is that the > end system would be useful for a wide range of users from someone with 3 > large disk or three disk servers to groups with 3 or more distributed > storage sites. > > Main Goals/Features: > 1) Tapeless: maintain multiple copies on disk (minimize > backup/restore lag) > 2) "Mirroring" across remote sites: for disaster recovery (we sit on > top of the Hayward Fault) > 3) Persistent snapshots: as archival copies instead of > backup/restore scanning > 4) Copy-On-Write: in support of snapshots/archives > 5) Append-mostly log structured file system: make synchronization of > remote mirrors easier (tail the log). > 6) Avoid (insofar as possible) single point of failure and > bottlenecks (for scalability) > > I've looked into the existing file systems I know about, and none of > them seem to fit the bill. > Before we begin however, it is important to find out: > 1) Is there anything sufficiently like this to either (a) use > instead, or (b) start from. > 2) Is there community support for insertion in the main kernel tree > (without which it is just another toy project)? > 3) Anyone care to join in (a) design, (b) implementation, or (c) > testing? Lustre isn't quite where you want to be yet, but features like mirroring (closer), snapshots, and disconnected operation+resync (further out) are all on our roadmap. Lustre is GPL. If you are interested to contribute to it we are happy to work with you. There is a "lustre.org" (non-CFS Lustre development) planning session next week in Boulder, CO (and telecon) that you could join in if you are interested. Please email pbojanic@xxxxxxxxxxxxx if you are interested. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
