On Wed, Jan 19, 2011 at 4:14 AM, Tomasz Chmielewski <mangoo@xxxxxxxx> wrote: > On 19.01.2011 12:41, Wido den Hollander wrote: >> >> Hi Thomas, >> >> I think the answer is Yes and No on this question, the devs might have >> another approach for your situation. >> >> If you would do this, you would have a MON, MDS and OSD on every server, >> in theory that would work. Mounting would be done by connecting to on of >> the MON's (doesn't matter which one). >> >> But Ceph requires, well, advises a odd number of monitors (Source: >> http://ceph.newdream.net/wiki/Designing_a_cluster ) >> >> So you would require a third node which is running your third monitor to >> keep track of both nodes. >> >> My advise, for two nodes, use something like DRBD in Primary<> Primary >> and use a cluster filesystem like OCFS2. > > Currently, I'm running glusterfs in such a scenario (two servers, each being > also clients), but I wanted to give ceph a try (glusterfs has some > performance issues with lots of small files), also because of its nice > features (snapshots, rbd etc.). Rather than running 3 monitors you could just put a monitor on one of the machines -- your cluster will go down if it fails, but in a 2-node system it's not like resilience from one-node failure would be very helpful anyway. However, there is a serious issue with running clients and servers on one machine, which may or may not be a problem depending on your use case: Deadlock becomes a significant possibility. This isn't a problem we've come up with a good solution for, unfortunately, but imagine you're writing a lot of files to Ceph. Ceph dutifully writes them and the kernel dutifully caches them. You also have a lot of write activity so the Ceph kernel client is doing local caching. Then the kernel comes along and says "I'm low on memory! Flush stuff to disk!" and the kernel client tries to flush it out...which involves creating another copy of the data in memory on the same machine. Uh-oh! Now if you use the FUSE client this won't be an issue, but your performance also won't be so good. :/ -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
