|
Thanks Alfredo!
Would that be /etc/ceph/ceph.conf or ~/my-cluster/ceph.conf
Ron Gage
> On February 14, 2014 at 9:31 AM Alfredo Deza <alfredo.deza@xxxxxxxxxxx> wrote: > > > On Fri, Feb 14, 2014 at 9:08 AM, Ron Gage <ron@xxxxxxxxxxx> wrote: > > Please see the post you replied to. The admin keyring is not being created. > > > > Thanks! > > > > Ron > > There is no need to run other commands instead of create-initial. That > is the best way to get started with monitor deployment. > > From ceph-deploy's perspective, everything has been called correctly > and the output from the node looks correct. The next thing I would > suggest is to bump the log levels of the monitors in ceph.conf and > restart them so that you can inspect a bit better what is going on > there. > > In the [global] section of ceph.conf: > > mon debug = 10 > > > > > > > > > On February 14, 2014 at 8:54 AM Srinivasa Rao Ragolu <sragolu@xxxxxxxxxx> > > wrote: > > > > Follow the below link as it is > > > > http://karan-mj.blogspot.fi/2013/12/ceph-installation-part-2.html > > > > Thanks, > > Srinivas. > > > > > > On Fri, Feb 14, 2014 at 7:21 PM, Ron Gage <ron@xxxxxxxxxxx> wrote: > > > > > > > > > > [ceph@cm my-cluster]$ ceph-deploy mon create cm > > [ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy mon create > > cm > > [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts cm > > [ceph_deploy.mon][DEBUG ] detecting platform for host cm ... > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [ceph_deploy.mon][INFO ] distro info: CentOS 6.5 Final > > [cm][DEBUG ] determining if provided host has same hostname in remote > > [cm][DEBUG ] get remote short hostname > > [cm][DEBUG ] deploying mon to cm > > [cm][DEBUG ] get remote short hostname > > [cm][DEBUG ] remote hostname: cm > > [cm][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf > > [cm][DEBUG ] create the mon path if it does not exist > > [cm][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-cm/done > > [cm][DEBUG ] create a done file to avoid re-doing the mon deployment > > [cm][DEBUG ] create the init path if it does not exist > > [cm][DEBUG ] locating the `service` executable... > > [cm][INFO ] Running command: sudo /sbin/service ceph -c /etc/ceph/ceph.conf > > start mon.cm > > [cm][DEBUG ] === mon.cm === > > [cm][DEBUG ] Starting Ceph mon.cm on cm... > > [cm][DEBUG ] Starting ceph-create-keys on cm... > > [cm][WARNIN] No data was received after 7 seconds, disconnecting... > > [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon > > /var/run/ceph/ceph-mon.cm.asok mon_status > > [cm][DEBUG ] > > ******************************************************************************** > > [cm][DEBUG ] status for monitor: mon.cm > > [cm][DEBUG ] { > > [cm][DEBUG ] "election_epoch": 1, > > [cm][DEBUG ] "extra_probe_peers": [], > > [cm][DEBUG ] "monmap": { > > [cm][DEBUG ] "created": "0.000000", > > [cm][DEBUG ] "epoch": 1, > > [cm][DEBUG ] "fsid": "b11875f0-e611-40b9-ae81-e7cba7586ed5", > > [cm][DEBUG ] "modified": "0.000000", > > [cm][DEBUG ] "mons": [ > > [cm][DEBUG ] { > > [cm][DEBUG ] "addr": " 172.24.12.91:6789/0", > > [cm][DEBUG ] "name": "cm", > > [cm][DEBUG ] "rank": 0 > > [cm][DEBUG ] } > > [cm][DEBUG ] ] > > [cm][DEBUG ] }, > > [cm][DEBUG ] "name": "cm", > > [cm][DEBUG ] "outside_quorum": [], > > [cm][DEBUG ] "quorum": [ > > [cm][DEBUG ] 0 > > [cm][DEBUG ] ], > > [cm][DEBUG ] "rank": 0, > > [cm][DEBUG ] "state": "leader", > > [cm][DEBUG ] "sync_provider": [] > > [cm][DEBUG ] } > > [cm][DEBUG ] > > ******************************************************************************** > > [cm][INFO ] monitor: mon.cm is running > > [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon > > /var/run/ceph/ceph-mon.cm.asok mon_status > > [ceph@cm my-cluster]$ ^C > > > > [ceph@cm my-cluster]$ ceph-deploy gatherkeys cm > > [ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy gatherkeys > > cm > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for > > /etc/ceph/ceph.client.admin.keyring > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [cm][DEBUG ] fetch remote file > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > > /etc/ceph/ceph.client.admin.keyring on ['cm'] > > [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for > > /var/lib/ceph/bootstrap-osd/ceph.keyring > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [cm][DEBUG ] fetch remote file > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > > /var/lib/ceph/bootstrap-osd/ceph.keyring on ['cm'] > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for > > /var/lib/ceph/bootstrap-mds/ceph.keyring > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [cm][DEBUG ] fetch remote file > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > > /var/lib/ceph/bootstrap-mds/ceph.keyring on ['cm'] > > [ceph@cm my-cluster]$ > > > > > > > > > > On February 14, 2014 at 8:20 AM Ashish Chandra < > > mail.ashishchandra@xxxxxxxxx> wrote: > > > > Hi Ron, > > Can you try running : > > > > ceph-deploy mon create {ceph-node} > > > > ceph-deploy gatherkeys {ceph-node} > > > > > > as two different commands, instead of running > > > > > > > > > > ceph-deploy mon create-initial > > > > and let me know if the problem still persists. > > > > Actually it is not able to find "client.admin.keyring" file as it has not > > been created yet. > > > > Thanks > > > > Ashish Chandra > > > > > > > > On Fri, Feb 14, 2014 at 6:15 PM, Ron Gage <ron@xxxxxxxxxxx> wrote: > > > > Greetings all! > > I am trying to deploy a proof of concept system for a replicated fault > > tolerant CIFS server cluster using CEPH as the back end. I am having > > significant trouble getting the CEPH part to install correctly. > > Specifically, I can't get the monitor to come up into a useful state due to > > the apparent lack of a keyring or two. Here is the background: > > > > Everything is running on a standalone VMware server with internal storage. > > The idea here is to see it work, not to see how well it performs. > > > > Current configuraton: 4 x OSD, 1 x monitor/manager > > OSD: 2 x CPU, 8 gig memory, internal only networking, 16 gig sda for > > boot/root, 160 gig sdb for CEPH store, Centos 6.5 x64 fully up to date, > > minimal install > > mon/mgr: 2 x CPU, 2 gig memory, internal and external networking, 16 gig > > sda, Centos 6.5 x64 fully up to date, minimal install > > > > Since I have made several attempts already to get this working, I execute > > the following commands from the mgr to reset everything (logged in as user > > ceph) > > ceph-deploy purgedata c1 c2 c3 c4 cm > > ceph-deploy forgetkeys > > ceph-deploy purge c1 c2 c3 c4 cm > > sudo rm -f /etc/ceph/* > > sudo rm -f /var/lib/ceph/mon/*.* > > cd ~/my-cluster > > rm -f * > > > > Now to deploy a straight forward single monitor node: > > ceph-deploy new cm > > > > And to add the public interface to the ceph.conf > > vi ceph.conf > > (add to end of global section) public network = 172.24.0.0/16 > > > > Now to install the ceph software in the 5 nodes: > > ceph-deploy install cm c1 c2 c3 c4 > > (waiting...waiting...waiting...) > > > > And here is where the problems begin - creating the initial monitor: > > [ceph@cm my-cluster]$ ceph-deploy mon create-initial > > [ceph_deploy.cli][INFO ] Invoked (1.3.5): /usr/bin/ceph-deploy mon > > create-initial > > [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts cm > > [ceph_deploy.mon][DEBUG ] detecting platform for host cm ... > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [ceph_deploy.mon][INFO ] distro info: CentOS 6.5 Final > > [cm][DEBUG ] determining if provided host has same hostname in remote > > [cm][DEBUG ] get remote short hostname > > [cm][DEBUG ] deploying mon to cm > > [cm][DEBUG ] get remote short hostname > > [cm][DEBUG ] remote hostname: cm > > [cm][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf > > [cm][DEBUG ] create the mon path if it does not exist > > [cm][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-cm/done > > [cm][DEBUG ] create a done file to avoid re-doing the mon deployment > > [cm][DEBUG ] create the init path if it does not exist > > [cm][DEBUG ] locating the `service` executable... > > [cm][INFO ] Running command: sudo /sbin/service ceph -c /etc/ceph/ceph.conf > > start mon.cm > > [cm][DEBUG ] === mon.cm === > > [cm][DEBUG ] Starting Ceph mon.cm on cm... > > [cm][DEBUG ] Starting ceph-create-keys on cm... > > [cm][WARNIN] No data was received after 7 seconds, disconnecting... > > [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon > > /var/run/ceph/ceph-mon.cm.asok mon_status > > [cm][DEBUG ] > > ******************************************************************************** > > [cm][DEBUG ] status for monitor: mon.cm > > [cm][DEBUG ] { > > [cm][DEBUG ] "election_epoch": 1, > > [cm][DEBUG ] "extra_probe_peers": [], > > [cm][DEBUG ] "monmap": { > > [cm][DEBUG ] "created": "0.000000", > > [cm][DEBUG ] "epoch": 1, > > [cm][DEBUG ] "fsid": "b11875f0-e611-40b9-ae81-e7cba7586ed5", > > [cm][DEBUG ] "modified": "0.000000", > > [cm][DEBUG ] "mons": [ > > [cm][DEBUG ] { > > [cm][DEBUG ] "addr": " 172.24.12.91:6789/0", > > [cm][DEBUG ] "name": "cm", > > [cm][DEBUG ] "rank": 0 > > [cm][DEBUG ] } > > [cm][DEBUG ] ] > > [cm][DEBUG ] }, > > [cm][DEBUG ] "name": "cm", > > [cm][DEBUG ] "outside_quorum": [], > > [cm][DEBUG ] "quorum": [ > > [cm][DEBUG ] 0 > > [cm][DEBUG ] ], > > [cm][DEBUG ] "rank": 0, > > [cm][DEBUG ] "state": "leader", > > [cm][DEBUG ] "sync_provider": [] > > [cm][DEBUG ] } > > [cm][DEBUG ] > > ******************************************************************************** > > [cm][INFO ] monitor: mon.cm is running > > [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon > > /var/run/ceph/ceph-mon.cm.asok mon_status > > [ceph_deploy.mon][INFO ] processing monitor mon.cm > > [cm][DEBUG ] connected to host: cm > > [cm][INFO ] Running command: sudo ceph --cluster=ceph --admin-daemon > > /var/run/ceph/ceph-mon.cm.asok mon_status > > [ceph_deploy.mon][INFO ] mon.cm monitor has reached quorum! > > [ceph_deploy.mon][INFO ] all initial monitors are running and have formed > > quorum > > [ceph_deploy.mon][INFO ] Running gatherkeys... > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for > > /etc/ceph/ceph.client.admin.keyring > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [cm][DEBUG ] fetch remote file > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > > /etc/ceph/ceph.client.admin.keyring on ['cm'] > > [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for > > /var/lib/ceph/bootstrap-osd/ceph.keyring > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [cm][DEBUG ] fetch remote file > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > > /var/lib/ceph/bootstrap-osd/ceph.keyring on ['cm'] > > [ceph_deploy.gatherkeys][DEBUG ] Checking cm for > > /var/lib/ceph/bootstrap-mds/ceph.keyring > > [cm][DEBUG ] connected to host: cm > > [cm][DEBUG ] detect platform information from remote host > > [cm][DEBUG ] detect machine type > > [cm][DEBUG ] fetch remote file > > [ceph_deploy.gatherkeys][WARNIN] Unable to find > > /var/lib/ceph/bootstrap-mds/ceph.keyring on ['cm'] > > [ceph@cm my-cluster]$ > > > > > > From my untrained eye, it looks like several key files aren't either being > > created or copied to the expected place. > > > > Contents of ~/my-cluster: > > [ceph@cm my-cluster]$ ls > > ceph.conf ceph.log ceph.mon.keyring epel-release-6-8.noarch.rpm > > > > Contents of /var/lib/ceph/mon/ceph-cm: > > > > [ceph@cm ceph-cm]$ ls -R .: done keyring store.db sysvinit > > > > ./store.db: 000016.sst 000021.sst CURRENT LOG 000018.sst 000022.log > > LOCK MANIFEST-000020 > > > > Contents of /etc/ceph: > > > > [ceph@cm ceph]$ ls > > ceph.client.admin.keyring.5704.tmp ceph.conf rbdmap > > > > As you can plainly see, I have followed the directions in the "quick start" > > deployment guide on ceph.com > > (http://ceph.com/docs/master/start/quick-ceph-deploy/#create-a-cluster). > > > > What am I missing? Are the directions incomplete (or wrong)? > > > > > > > > HELP! > > > > > > > > Ron Gage > > Westland, Michigan > > > > Please be sure to check out My Photography Website > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > Ron Gage > > Westland, Michigan > > > > Please be sure to check out My Photography Website > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > Ron Gage > > Westland, Michigan > > > > Please be sure to check out My Photography Website > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > |
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
