On 2018/02/01 10:58 am, Alfredo Deza wrote:
On Thu, Feb 1, 2018 at 10:44 AM, Andre Goree <andre@xxxxxxxxxx> wrote:
Does ceph-volume support lvm+Bluestore?
Yes it does. Since version 12.2.2 which seems to be what you have.
I ask this, bc I'm trying to use
ceph-ansible to provision an OSD node, and am getting a hang when the
'ceph-volume create' command is trying to be run. I'm also using
ceph-ansible/master (not stable-3.0 or any other stable version),
which has
the parameters for bluestore+lvm, but may not actually be implemented
in
12.2.2. ceph-ansible stable-3.0 branch does NOT contain a section for
bluestore in the ceph-volume/lvm section.
Correct, stable-3.0 doesn't have it.
Here you can see the full command(s) being run (via ansible), and an
strace
on that process. I've left that process running overnight and it
appears it
is still timing out (apologies if I should've used pastebin):
This is not really ceph-volume. It is trying to communicate with the
cluster but it is not going through. Something in your configuration
is not quite right, so the command is hanging.
This is the actual command:
/usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring
/var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new
a2ee64a4-b5ba-4ca9-8528-4205f3ad8c99
What that command is trying to do is to tell the monitor about the
newly created OSD. It is easy to replicate this "hanging" problem if
you modify your ceph.conf to point to an invalid IP for
the monitors.
Thank you for confirming that and pointing me in the right direction!
It would appear my network configuration is certainly correct (from my
understanding; "public" network is 172.16.238.0/24, cluster network is
172.16.239.0/24 -- a configuration that works for the other OSDs built
with ceph-ansible/ceph-disk) and I can reach port 6789 on my MON node:
~# ping -c4 172.16.238.11 && ping -c4 172.16.239.11
PING 172.16.238.11 (172.16.238.11) 56(84) bytes of data.
64 bytes from 172.16.238.11: icmp_seq=1 ttl=64 time=0.141 ms
64 bytes from 172.16.238.11: icmp_seq=2 ttl=64 time=0.102 ms
64 bytes from 172.16.238.11: icmp_seq=3 ttl=64 time=0.107 ms
64 bytes from 172.16.238.11: icmp_seq=4 ttl=64 time=0.096 ms
--- 172.16.238.11 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2999ms
rtt min/avg/max/mdev = 0.096/0.111/0.141/0.020 ms
PING 172.16.239.11 (172.16.239.11) 56(84) bytes of data.
64 bytes from 172.16.239.11: icmp_seq=1 ttl=64 time=0.252 ms
64 bytes from 172.16.239.11: icmp_seq=2 ttl=64 time=0.133 ms
64 bytes from 172.16.239.11: icmp_seq=3 ttl=64 time=0.098 ms
64 bytes from 172.16.239.11: icmp_seq=4 ttl=64 time=0.103 ms
--- 172.16.239.11 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2998ms
rtt min/avg/max/mdev = 0.098/0.146/0.252/0.063 ms
~# telnet 172.16.238.11 6789
Trying 172.16.238.11...
Connected to 172.16.238.11.
Escape character is '^]'.
ceph v027???^?^]quit
telnet> quit
Connection closed.
Is there a command you'd recommend I use to try to ensure connectivity
to the MON node from this new OSD node to perhaps help troubleshoot this
issue I'm having?
In the meantime, I'm going over my ceph-ansible configuration(s) again
to ensure I didn't miss something.
Thank you for your guidance Alfredo, it's been invaluable.
--
Andre Goree
-=-=-=-=-=-
Email - andre at drenet.net
Website - http://blog.drenet.net
PGP key - http://www.drenet.net/pubkey.html
-=-=-=-=-=-
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
