After reading all the research papers and docs over the last few months and waiting for Cuttlefish, I finally deployed a test cluster of 18 osds across 6 hosts. It's performing better than I expected so far, all on the default single interface.
I was also surprised by the minimal ceph.conf and non-functioning "service ceph" too, so perhaps worth mentioning in the doc? ceph-deploy also failed to start one of the osds (with activate), but running ceph-osd manually worked.
I now want to enter a secondary interface for the cluster-side network, so I assume I have to enter all the OSD's in ceph.conf manually anyway?
Very impressed so far!
Danny
--
On 9 May 2013 19:29, Sage Weil <sage@xxxxxxxxxxx> wrote:
On Thu, 9 May 2013, Greg Chavez wrote:This is correct. It is no longer necessary to enumerate daemons in
> So I feel like I'm missing something. I just deployed 3 storage nodes
> with ceph-deploy, each with a monitor daemon and 6-8 osd's. All of
> them seem to be active with health OK. However, it doesn't seem that
> I ended up with a useful ceph.conf.
>
> ( running 0.61-113-g61354b2-1raring )
>
> This is all I get:
>
> [global]
> fsid = af1581c1-8c45-4e24-b4f1-9a56e8a62aeb
> mon_initial_members = kvm-cs-sn-10i, kvm-cs-sn-14i, kvm-cs-sn-15i
> mon_host = 192.168.241.110,192.168.241.114,192.168.241.115
> auth_supported = cephx
> osd_journal_size = 1024
> filestore_xattr_use_omap = true
ceph.conf when using the default locations (/var/lib/ceph/osd/*).
'service' invokes the sysvinit scripts, and on Ubuntu we are using
> When you run "service ceph status", it returns nothing because it
> can't find any osd stanzas.
upstart. Try
initctl list | grep ceph
to see what is running.
sage
> Based on the directory names in
> /var/lib/ceph/mon and the parsing of the mounted storage volumes I
> wrote this out:
>
> http://pastebin.com/DLgtiC23
>
> And then pushed it out.
>
> The monitor id names seem odd to me, with the hostname instead of a,
> b, and c, but whatever.
>
> Now I get this output:
>
> root@kvm-cs-sn-10i:/etc/ceph# service ceph -a status
> === mon.kvm-cs-sn-10i ===
> mon.kvm-cs-sn-10i: not running.
> === mon.kvm-cs-sn-14i ===
> mon.kvm-cs-sn-14i: not running.
> === mon.kvm-cs-sn-15i ===
> mon.kvm-cs-sn-15i: not running.
> === osd.0 ===
> osd.0: not running.
> === osd.1 ===
> osd.1: not running.
> === osd.10 ===
> osd.10: not running.
> ...etc....
>
> Not true! Even worse, when I try to run "service ceph -a start", It
> freaks and complains about missing keys. So now I have this process
> hanging around:
>
> /usr/bin/python /usr/sbin/ceph-create-keys -i kvm-cs-sn-10i
>
> Here's the output from that attempt:
>
> root@kvm-cs-sn-10i:/tmp# service ceph -a start
> === mon.kvm-cs-sn-10i ===
> Starting Ceph mon.kvm-cs-sn-10i on kvm-cs-sn-10i...
> failed: 'ulimit -n 8192; /usr/bin/ceph-mon -i kvm-cs-sn-10i
> --pid-file /var/run/ceph/mon.kvm-cs-sn-10i.pid -c /etc/ceph/ceph.conf
> '
> Starting ceph-create-keys on kvm-cs-sn-10i...
>
> Luckily I hadn't set up my ssh keys yet, so that's as far as I got.
>
> Would dearly love some guidance. Thanks in advance!
>
> --Greg Chavez
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Mean Trading Systems LLP
http://www.meantradingsystems.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
