On Nov 4, 2009, at 21:39 , Jeff Garzik wrote:
Yes, I think it departed too much from a desirable filesystem-like
API. Using lock/unlock/trylock for ordered contention resolution,
leader election, garbage collection, and similar tasks is a
convenient model. CLD will probably be more Chubby-like than
Zookeeper.
I haven't actually studied the Zookeeper API closely enough to be
aware of these differences. I'll take a closer look.
My interest in Zookeeper is just that implementing replicated state
machine protocols (eg. Paxos), is surprisingly tricky. Re-using a
somewhat well tested implementation that has been deployed in
production seems like it would save considerable time. Although I will
be the first to admit that doing it yourself is a heck of a lot more
fun.
b) Has there been any consideration for re-using an existing
serialization format (XDR, Thrift, Protocol Buffers, whatever)?
In what context should these be considered? We are certainly open
to using open, documented protocols -- one big reason why Amazon S3
(HTTP REST) and NFSv4 were chosen as production examples for the
tabled and nfs4d projects, respectively.
For network message serialization and deserialization. For example,
CLD and chunkd both have have defined their own message structures.
These could be expressed in some pseudo-standard serialization format.
This (theoretically) makes it easier to write clients in other
languages, as the serialization/deserialization would be automatic
rather than needing to be re-implemented. This makes life somewhat
easier when there are protocol changes. Some of these may provide
other advantages, such as assisting with seamless upgrades.
Additionally, if Project Hail is successful, and people start building
distributed systems on top of its components, it might be nice if it
included some sort of "standard" protocol/message format, so that it
is easier for service provided by X to interact with service provided
by Y.
chunkd's protocol is custom because I'm not aware of a highly
efficient binary network storage protocol that fits its niche.
Although the "message header" part could potentially be encoded using
some sort of serialization tool.
Thanks for your insight!
Evan
--
Evan Jones
http://evanjones.ca/
--
To unsubscribe from this list: send the line "unsubscribe hail-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
