Well, Zookeeper is written in Java. Presumably it requires you to put a JVM on each node. Unfortunately the JVM has kind of a large memory footprint. That's great if your software is written in Java. If your company didn't go that route, ZK doesn't seem like such a great option. On Tue, Jun 8, 2010 at 8:07 AM, Jeff Darcy <jdarcy@xxxxxxxxxx> wrote: > On 06/07/2010 10:32 PM, Jeff Garzik wrote: >> I think you're overselling that angle a bit. Google discourages use of >> Chubby as a strict publish-subscribe mechanism, so watches and >> ephemerals aren't the bread-and-butter of Chubby necessarily. The Chubby paper specifically calls out using Chubby for publish / subscribe as "abusive behavior." But ZooKeeper "is used at Yahoo! as the coordination and failure recovery service for Yahoo! Message Broker, which is a highly scalable publish-subscribe system" according to hadoop.apache.org. Although they might be equivalent in some theoretical computer science sense, I get the impression that the two systems are very different beasts... >> A lot of >> the supposed commonality comes simply from the attribute of being a >> centralized respository of data for autonomous cloud systems -- a >> shared, highly reliable filesystem -- not any particular attribute >> related to watches or ephemerals. > > Nonetheless, those are features both share, and many might argue that > they're preferable to locks. Locking is a fundamentally lousy way to > build scalable and reliable distributed systems, as has been well known > for more than a decade. I think *fine-grained* locking is a fundamentally lousy way to build distributed systems. I haven't heard anyone argue that coarse-grained locking is bad. Have you read any interesting papers or books about this topic? > If somebody who already had ZK running could point > chunkd/tabled at that instead of having to download/install/configure > CLD (and fight with their IT folks about the DNS dependency), then they > might actually be more likely to use chunkd/tabled instead of something > even more broken. Some few of them might even start to contribute. > Surely that would be a good thing. If self-imposed technical obstacles > still mean it's too much effort then fine, but I don't think we need > more NIH syndrome. It would be cool to have support another backend, I guess. However... It seems to me that once you make the decision to use ZooKeeper and Java, Walrus or Hadoop is probably a more practical choice for the upper layers. I've never deployed either though, so I could be wrong. Colin McCabe -- To unsubscribe from this list: send the line "unsubscribe hail-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
