On 19.01.2011 16:57, Gregory Farnum wrote:
Currently, I'm running glusterfs in such a scenario (two servers, each being
also clients), but I wanted to give ceph a try (glusterfs has some
performance issues with lots of small files), also because of its nice
features (snapshots, rbd etc.).
Rather than running 3 monitors you could just put a monitor on one of
the machines -- your cluster will go down if it fails, but in a 2-node
system it's not like resilience from one-node failure would be very
helpful anyway.
OK, I could imagine starting the monitor on just one node i.e. with the
help of heartbeat - so if the node with the monitor goes down, heartbeat
starts the monitor process on the other machine.
However, there is a serious issue with running clients and servers on
one machine, which may or may not be a problem depending on your use
case: Deadlock becomes a significant possibility.
Sounds like the "freezes" issue mentioned by Dong Jin Lee?
This isn't a problem
we've come up with a good solution for, unfortunately, but imagine
you're writing a lot of files to Ceph. Ceph dutifully writes them and
the kernel dutifully caches them. You also have a lot of write
activity so the Ceph kernel client is doing local caching. Then the
kernel comes along and says "I'm low on memory! Flush stuff to disk!"
and the kernel client tries to flush it out...which involves creating
another copy of the data in memory on the same machine. Uh-oh!
Uh-oh, it doesn't sound encouraging, and will likely happen sooner or later.
Would some sort of zero-copy help here? But perhaps it's not that easy
to solve, otherwise, we wouldn't be discussing it here.
I think swapping over NFS (or, iSCSI) has a similar problem ("need to
write, but the network buffer is full, so we can't write over network ->
deadlock"), and there were some patches floating around some years ago
to solve it. Not sure what's the state of it and how similar it is to
Ceph though.
Last I checked, 2 < 3, so having a budget HA solution which just needed
2 servers instead of 3 would be a great thing to have! ;)
--
Tomasz Chmielewski
http://wpkg.org
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
