Dear all,
I am facing problem in deploying ceph-mon (Segmentation fault). I am deploying Ceph on single-board Raspberry pi 3, Hyperiot debian 8.0 jessie OS. I downloaded ceph packages from the following repository :
====
ii ceph 10.2.5-7.2+rpi1 armhf distributed storage and file system
ii ceph-base 10.2.5-7.2+rpi1 armhf common ceph daemon libraries and management tools
ii ceph-common 10.2.5-7.2+rpi1 armhf common utilities to mount and interact with a ceph storage cluster
ii ceph-deploy 2.0.0+dfsg1-1 all Ceph cluster deployment and configuration over ssh
ii ceph-mds 10.2.5-7.2+rpi1 armhf metadata server for the ceph distributed file system
ii ceph-mon 10.2.5-7.2+rpi1 armhf monitor server for the ceph storage system
ii ceph-osd 10.2.5-7.2+rpi1 armhf OSD server for the ceph storage system
ii libcephfs1 10.2.5-7.2+rpi1 armhf Ceph distributed file system client library
ii python-cephfs 10.2.5-7.2+rpi1 armhf Python libraries for the Ceph libcephfs library
====
The packages are installed successfully. The problem is when I deploy ceph-mon service in the node raising segmentation fault. I think because of this admin socket was not created successfully.
The log file of the mon and ceph-deploy is given bellow:
I don't know if somebody has faced the similar problem. Could you please help me how to fix this bug?
The log file of the mon and ceph-deploy is given bellow:
mon.log
Aug 14 16:11:22 rpi3-4 systemd[1]: Started Ceph cluster key creator task.
Aug 14 16:11:22 rpi3-4 systemd[1]: Started Ceph cluster monitor daemon.
Aug 14 16:11:23 rpi3-4 ceph-mon[31611]: *** Caught signal (Segmentation fault) **
Aug 14 16:11:23 rpi3-4 ceph-mon[31611]: in thread 7561cb80 thread_name:admin_socket
Aug 14 16:11:23 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: exception: [Errno 104] Connection reset by peer
Aug 14 16:11:23 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Main process exited, code=killed, status=11/SEGV
Aug 14 16:11:23 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Failed with result 'signal'.
Aug 14 16:11:23 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:23 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Service RestartSec=100ms expired, scheduling restart.
Aug 14 16:11:23 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Scheduled restart job, restart counter is at 1.
Aug 14 16:11:23 rpi3-4 systemd[1]: Stopped Ceph cluster monitor daemon.
Aug 14 16:11:23 rpi3-4 systemd[1]: Started Ceph cluster monitor daemon.
Aug 14 16:11:25 rpi3-4 ceph-mon[31624]: *** Caught signal (Segmentation fault) **
Aug 14 16:11:25 rpi3-4 ceph-mon[31624]: in thread 75620b80 thread_name:admin_socket
Aug 14 16:11:25 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: exception: [Errno 104] Connection reset by peer
Aug 14 16:11:25 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Main process exited, code=killed, status=11/SEGV
Aug 14 16:11:25 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Failed with result 'signal'.
Aug 14 16:11:25 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:25 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Service RestartSec=100ms expired, scheduling restart.
Aug 14 16:11:25 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Scheduled restart job, restart counter is at 2.
Aug 14 16:11:25 rpi3-4 systemd[1]: Stopped Ceph cluster monitor daemon.
Aug 14 16:11:25 rpi3-4 systemd[1]: Started Ceph cluster monitor daemon.
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: *** Caught signal (Segmentation fault) **
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: in thread 75673b80 thread_name:admin_socket
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: ceph version 10.2.5 (c461ee19ecbc0c5c330aca20f7392c9a00730367)
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: 1: (()+0x4b1348) [0x5502d348]
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: 2: (__default_sa_restorer()+0) [0x768d7bc0]
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: 3: (AdminSocket::do_accept()+0x28) [0x55149154]
Aug 14 16:11:25 rpi3-4 ceph-mon[31640]: 4: (AdminSocket::entry()+0x22c) [0x5514b458]
Aug 14 16:11:26 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Main process exited, code=killed, status=11/SEGV
Aug 14 16:11:26 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Failed with result 'signal'.
Aug 14 16:11:26 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Service RestartSec=100ms expired, scheduling restart.
Aug 14 16:11:26 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Scheduled restart job, restart counter is at 3.
Aug 14 16:11:26 rpi3-4 systemd[1]: Stopped Ceph cluster monitor daemon.
Aug 14 16:11:26 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Start request repeated too quickly.
Aug 14 16:11:26 rpi3-4 systemd[1]: ceph-mon@rpi3-4.service: Failed with result 'signal'.
Aug 14 16:11:26 rpi3-4 systemd[1]: Failed to start Ceph cluster monitor daemon.
Aug 14 16:11:26 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:26 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:28 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:28 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:29 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:29 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:31 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:31 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:32 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:32 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:34 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:34 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:35 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:35 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:37 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:37 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:38 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:38 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:39 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:40 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:41 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:41 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:42 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:42 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:44 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:44 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:45 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:45 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:47 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:47 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
Aug 14 16:11:48 rpi3-4 ceph-create-keys[31609]: admin_socket: exception getting command descriptions: [Errno 111] Connection refused
Aug 14 16:11:48 rpi3-4 ceph-create-keys[31609]: INFO:ceph-create-keys:ceph-mon admin socket not ready yet.
ceph-deploy log:
[2018-08-14 18:09:42,866][ceph_deploy.conf][DEBUG ] found configuration file at: /var/lib/ceph/.cephdeploy.conf
[2018-08-14 18:09:42,868][ceph_deploy.cli][INFO ] Invoked (2.0.0): /usr/bin/ceph-deploy new rpi3-4
[2018-08-14 18:09:42,869][ceph_deploy.cli][INFO ] ceph-deploy options:
[2018-08-14 18:09:42,870][ceph_deploy.cli][INFO ] username : None
[2018-08-14 18:09:42,871][ceph_deploy.cli][INFO ] verbose : False
[2018-08-14 18:09:42,871][ceph_deploy.cli][INFO ] overwrite_conf : False
[2018-08-14 18:09:42,872][ceph_deploy.cli][INFO ] quiet : False
[2018-08-14 18:09:42,873][ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x7642e210>
[2018-08-14 18:09:42,873][ceph_deploy.cli][INFO ] cluster : ceph
[2018-08-14 18:09:42,874][ceph_deploy.cli][INFO ] ssh_copykey : True
[2018-08-14 18:09:42,875][ceph_deploy.cli][INFO ] mon : ['rpi3-4']
[2018-08-14 18:09:42,876][ceph_deploy.cli][INFO ] func : <function new at 0x76365a70>
[2018-08-14 18:09:42,876][ceph_deploy.cli][INFO ] public_network : None
[2018-08-14 18:09:42,877][ceph_deploy.cli][INFO ] ceph_conf : None
[2018-08-14 18:09:42,878][ceph_deploy.cli][INFO ] cluster_network : None
[2018-08-14 18:09:42,878][ceph_deploy.cli][INFO ] default_release : False
[2018-08-14 18:09:42,879][ceph_deploy.cli][INFO ] fsid : None
[2018-08-14 18:09:42,880][ceph_deploy.new][DEBUG ] Creating new cluster named ceph
[2018-08-14 18:09:42,882][ceph_deploy.new][INFO ] making sure passwordless SSH succeeds
[2018-08-14 18:09:43,046][rpi3-4][DEBUG ] connection detected need for sudo
[2018-08-14 18:09:43,195][rpi3-4][DEBUG ] connected to host: rpi3-4
[2018-08-14 18:09:43,199][rpi3-4][DEBUG ] detect platform information from remote host
[2018-08-14 18:09:43,391][rpi3-4][DEBUG ] detect machine type
[2018-08-14 18:09:43,409][rpi3-4][DEBUG ] find the location of an executable
[2018-08-14 18:09:43,420][rpi3-4][INFO ] Running command: sudo /bin/ip link show
[2018-08-14 18:09:43,538][rpi3-4][INFO ] Running command: sudo /bin/ip addr show
[2018-08-14 18:09:43,590][rpi3-4][DEBUG ] IP addresses found: [u'172.24.1.1', u'172.17.0.1', u'131.254.143.4']
[2018-08-14 18:09:43,592][ceph_deploy.new][DEBUG ] Resolving host rpi3-4
[2018-08-14 18:09:43,595][ceph_deploy.new][DEBUG ] Monitor rpi3-4 at 131.254.143.4
[2018-08-14 18:09:43,597][ceph_deploy.new][DEBUG ] Monitor initial members are ['rpi3-4']
[2018-08-14 18:09:43,599][ceph_deploy.new][DEBUG ] Monitor addrs are ['131.254.143.4']
[2018-08-14 18:09:43,601][ceph_deploy.new][DEBUG ] Creating a random mon key...
[2018-08-14 18:09:43,603][ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
[2018-08-14 18:09:43,605][ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
[2018-08-14 18:11:16,409][ceph_deploy.conf][DEBUG ] found configuration file at: /var/lib/ceph/.cephdeploy.conf
[2018-08-14 18:11:16,412][ceph_deploy.cli][INFO ] Invoked (2.0.0): /usr/bin/ceph-deploy --overwrite-conf mon create rpi3-4
[2018-08-14 18:11:16,413][ceph_deploy.cli][INFO ] ceph-deploy options:
[2018-08-14 18:11:16,413][ceph_deploy.cli][INFO ] username : None
[2018-08-14 18:11:16,414][ceph_deploy.cli][INFO ] verbose : False
[2018-08-14 18:11:16,415][ceph_deploy.cli][INFO ] overwrite_conf : True
[2018-08-14 18:11:16,416][ceph_deploy.cli][INFO ] subcommand : create
[2018-08-14 18:11:16,416][ceph_deploy.cli][INFO ] quiet : False
[2018-08-14 18:11:16,417][ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x763a2328>
[2018-08-14 18:11:16,418][ceph_deploy.cli][INFO ] cluster : ceph
[2018-08-14 18:11:16,418][ceph_deploy.cli][INFO ] mon : ['rpi3-4']
[2018-08-14 18:11:16,419][ceph_deploy.cli][INFO ] func : <function mon at 0x763886b0>
[2018-08-14 18:11:16,420][ceph_deploy.cli][INFO ] ceph_conf : None
[2018-08-14 18:11:16,421][ceph_deploy.cli][INFO ] keyrings : None
[2018-08-14 18:11:16,421][ceph_deploy.cli][INFO ] default_release : False
[2018-08-14 18:11:16,424][ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts rpi3-4
[2018-08-14 18:11:16,425][ceph_deploy.mon][DEBUG ] detecting platform for host rpi3-4 ...
[2018-08-14 18:11:16,591][rpi3-4][DEBUG ] connection detected need for sudo
[2018-08-14 18:11:16,742][rpi3-4][DEBUG ] connected to host: rpi3-4
[2018-08-14 18:11:16,746][rpi3-4][DEBUG ] detect platform information from remote host
[2018-08-14 18:11:16,939][rpi3-4][DEBUG ] detect machine type
[2018-08-14 18:11:16,958][rpi3-4][DEBUG ] find the location of an executable
[2018-08-14 18:11:16,961][ceph_deploy.mon][INFO ] distro info: debian 8.0 jessie
[2018-08-14 18:11:16,962][rpi3-4][DEBUG ] determining if provided host has same hostname in remote
[2018-08-14 18:11:16,962][rpi3-4][DEBUG ] get remote short hostname
[2018-08-14 18:11:16,965][rpi3-4][DEBUG ] deploying mon to rpi3-4
[2018-08-14 18:11:16,966][rpi3-4][DEBUG ] get remote short hostname
[2018-08-14 18:11:16,969][rpi3-4][DEBUG ] remote hostname: rpi3-4
[2018-08-14 18:11:16,974][rpi3-4][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf
[2018-08-14 18:11:16,990][rpi3-4][DEBUG ] create the mon path if it does not exist
[2018-08-14 18:11:16,993][rpi3-4][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-rpi3-4/done
[2018-08-14 18:11:16,996][rpi3-4][DEBUG ] create a done file to avoid re-doing the mon deployment
[2018-08-14 18:11:16,999][rpi3-4][DEBUG ] create the init path if it does not exist
[2018-08-14 18:11:17,010][rpi3-4][INFO ] Running command: sudo systemctl enable ceph.target
[2018-08-14 18:11:19,359][rpi3-4][INFO ] Running command: sudo systemctl enable ceph-mon@rpi3-4
[2018-08-14 18:11:21,627][rpi3-4][INFO ] Running command: sudo systemctl start ceph-mon@rpi3-4
[2018-08-14 18:11:24,911][rpi3-4][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.rpi3-4.asok mon_status
[2018-08-14 18:11:25,496][rpi3-4][ERROR ] admin_socket: exception getting command descriptions: [Errno 111] Connection refused
[2018-08-14 18:11:25,497][rpi3-4][WARNING] monitor: mon.rpi3-4, might not be running yet
[2018-08-14 18:11:25,507][rpi3-4][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.rpi3-4.asok mon_status
[2018-08-14 18:11:26,093][rpi3-4][ERROR ] admin_socket: exception getting command descriptions: exception: [Errno 104] Connection reset by peer
[2018-08-14 18:11:26,096][rpi3-4][WARNING] monitor rpi3-4 does not exist in monmap
Thanks in advance.
------------------------------------------------------------------------
Arif Ahmed, PhD Student
MYRIADS Team, IRISA
Rennes, France_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
