Jeff Anderson-Lee wrote:
I'm part of a project at University of California Berkeley that is
trying to put together a predominantly archival file system for petabyte
class data stores using Linux with clusters of commodity server
hardware. We currently have multiple terabytes of hardware on top of
which we intend to build such a system. However, our hope is that the
end system would be useful for a wide range of users from someone with 3
large disk or three disk servers to groups with 3 or more distributed
storage sites.
Main Goals/Features:
1) Tapeless: maintain multiple copies on disk (minimize
backup/restore lag)
2) "Mirroring" across remote sites: for disaster recovery (we sit on
top of the Hayward Fault)
3) Persistent snapshots: as archival copies instead of backup/restore
scanning
4) Copy-On-Write: in support of snapshots/archives
5) Append-mostly log structured file system: make synchronization of
remote mirrors easier (tail the log).
6) Avoid (insofar as possible) single point of failure and
bottlenecks (for scalability)
I've looked into the existing file systems I know about, and none of
them seem to fit the bill.
Parts of the Open Solaris ZFS file system looks interesting, except (a)
it is not on Linux and (b) seems to mix together too many levels (volume
manager and file system). I can see how using some of the concepts and
implementing something like it on top of an append-mostly distributed
logical device might work however. By splitting the project into two
parts ((a) a robust, distributed logical block device and (b) a flexible
file system with snapshots) it might make it easier to design and build.
Before we begin however, it is important to find out:
1) Is there anything sufficiently like this to either (a) use
instead, or (b) start from.
2) Is there community support for insertion in the main kernel tree
(without which it is just another toy project)?
3) Anyone care to join in (a) design, (b) implementation, or (c)
testing?
I would recommend checking out Venti:
http://cm.bell-labs.com/sys/doc/venti.html
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
