Chris Mason wrote:
On Thu, Jun 14, 2007 at 02:20:26PM -0400, Chuck Lever wrote:
NetApp happens to use the standard NDMP protocol for sending the
flattened file system. NetApp uses it for synchronous replication,
volume migration, and back up to nearline storage and tape. AFS used
"vol dump" and "vol restore" for migration, replication, and back-up.
ZFS has the "zfs send" and "zfs receive" commands that do basically the
same (Eric Kustarz recently published a blog entry that described how
these work). And of course, all file system objects are able to be sent
this way: streams, xattrs, ACLs, and so on are all supported.
Note also that NFSv4 supports the idea of migrated or replicated file
objects. All that is needed to support it is a mechanism on the servers
to actually move the data.
Stringing the replication together with the underlying FS would be neat.
Is there a way to deal with a master/slave setup, where the slave may be
out of date?
Among the implementations I'm aware of, there is a varying degree of
integration into the physical file system. In general, it depends on
how far out of date the slave is, and how closely the slave is supposed
to be synchronized to the master.
A hot backup file system, for example, should be data-consistent within
a few seconds of the master. A snapshot is used to initialize a slave,
followed by a live stream of updates to the master being sent to slaves.
Such a mechanism already exists on NetApp filers because they gather
changes in NVRAM before committing them to the local file system.
Simply put, these changes can also be bundled and sent to a local hot
backup filer that is attached via Infiniband, or over the network to a
remote hot backup filer.
For AFS, replication is done by maintaining a rw and ro copy of a volume
on the designated master server. Changes are made to the rw copy over
time. When admins want to push out a new version to replicas on another
server, the ro copy on the master is replaced with a new snapshot, then
this is pushed to the slaves. The replicas are always ro and are used
mostly for load balancing; clients contact the closest or fastest server
containing a replica of the volume they want to access. They always
have a complete copy of the volume (ie no COW on the slaves).
I think you have designed into btrfs a lot of opportunity to implement
this kind of data virtualization and management... I'm excited to see
what can be done.
begin:vcard
fn:Chuck Lever
n:Lever;Chuck
org:Oracle Corporation;Corporate Architecture: Linux Projects Group
adr:;;1015 Granger Avenue;Ann Arbor;MI;48104;USA
title:Principal Member of Staff
tel;work:+1 248 614 5091
x-mozilla-html:FALSE
url:http://oss.oracle.com/~cel/
version:2.1
end:vcard
