I will be honest. I found with my install that doing the yum install outside of Ceph-deploy on the Ceph node itself works much better. It does require all the packages and dependencies to be available on the node, but it saved me a lot of debugging hassle and time. In fact, I am fast developing the opinion that doing most things outside of Ceph-deploy (certainly on my Red Hat build) is the easier option.
__________________________________________________________________________
Alistair Whittle | Storage Engineer | Global Technology Infrastructure and Services
Tel +44 (0) 20 3134 5933 | Mobile +44 (0) 77883 55220 | Email alistair.whittle@xxxxxxxxxxxx
Barclays, 10 South Colonnade, Canary Wharf, London, E14 4PU
barclays.com
Advance Out of Office Notification: 14-18th October (Training)
-----Original Message-----
From: Gruher, Joseph R [joseph.r.gruher@xxxxxxxxx]
Sent: Tuesday, October 29, 2013 06:48 PM GMT Standard Time
To: Trivedi, Narendra; Whittle, Alistair: Investment Bank (LDN); ceph-users@xxxxxxxxxxxxxx
Subject: RE: ceph-deploy problems on CentOS-6.4
Now you need to set the Curl proxy in /root/.curlrc (might also set it in ~/.curlrc for good measure).
To ceph-deploy on CentOS 6.4 I had to set all these proxies:
Curl: /root/.curlrc
rpm: /root/.rpmmacros
wget: /etc/wgetrc
yum: /etc/yum.conf
From: Trivedi, Narendra [mailto:Narendra.Trivedi@xxxxxxxxxx]
Sent: Tuesday, October 29, 2013 11:45 AM
To: alistair.whittle@xxxxxxxxxxxx; Gruher, Joseph R; ceph-users@xxxxxxxxxxxxxx
Subject: RE: ceph-deploy problems on CentOS-6.4
I made the changes and it seems now it can successfully wget the rpm but getting a different error now:
[root@ceph-admin-node-centos-6-4 my-cluster]# ceph-deploy install ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 ceph-node3-osd1-centos-6-4
[ceph_deploy.cli][INFO ] Invoked (1.2.7): /usr/bin/ceph-deploy install ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 ceph-node3-osd1-centos-6-4
[ceph_deploy.install][DEBUG ] Installing stable version dumpling on cluster ceph hosts ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 ceph-node3-osd1-centos-6-4
[ceph_deploy.install][DEBUG ] Detecting platform for host ceph-node1-mon-centos-6-4 ...
[ceph_deploy.sudo_pushy][DEBUG ] will use a remote connection without sudo
[ceph_deploy.install][INFO ] Distro info: CentOS 6.4 Final
[ceph-node1-mon-centos-6-4][INFO ] installing ceph on ceph-node1-mon-centos-6-4
[ceph-node1-mon-centos-6-4][INFO ] adding EPEL repository
[ceph-node1-mon-centos-6-4][INFO ] Running command: wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[ceph-node1-mon-centos-6-4][ERROR ] --2013-10-29 13:40:43-- http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[ceph-node1-mon-centos-6-4][ERROR ] Connecting to 10.12.132.208:8080... connected.
[ceph-node1-mon-centos-6-4][ERROR ] Proxy request sent, awaiting response... 200 OK
[ceph-node1-mon-centos-6-4][ERROR ] Length: 14540 (14K) [application/x-rpm]
[ceph-node1-mon-centos-6-4][ERROR ] Saving to: `epel-release-6-8.noarch.rpm'
[ceph-node1-mon-centos-6-4][ERROR ] 0K .......... .... 100% 1.52M=0.009s
[ceph-node1-mon-centos-6-4][ERROR ] Last-modified header invalid -- time-stamp ignored.
[ceph-node1-mon-centos-6-4][ERROR ] 2013-10-29 13:41:42 (1.52 MB/s) - `epel-release-6-8.noarch.rpm' saved [14540/14540]
[ceph-node1-mon-centos-6-4][INFO ] Running command: rpm -Uvh --replacepkgs epel-release-6*.rpm
[ceph-node1-mon-centos-6-4][INFO ] Preparing... ##################################################
[ceph-node1-mon-centos-6-4][INFO ] epel-release ##################################################
[ceph-node1-mon-centos-6-4][ERROR ] warning: epel-release-6-8.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 0608b895: NOKEY
[ceph-node1-mon-centos-6-4][INFO ] Running command: su -c 'rpm --import "https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc"'
[ceph-node1-mon-centos-6-4][ERROR ] Traceback (most recent call last):
[ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib/python2.6/site-packages/ceph_deploy/hosts/centos/install.py", line 23, in install
[ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib/python2.6/site-packages/ceph_deploy/util/decorators.py", line 10, in inner
[ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib/python2.6/site-packages/ceph_deploy/util/wrappers.py", line 6, in remote_call
[ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
[ceph-node1-mon-centos-6-4][ERROR ] raise CalledProcessError(retcode, cmd)
[ceph-node1-mon-centos-6-4][ERROR ] CalledProcessError: Command '['su -c \'rpm --import "https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc"\'']' returned non-zero exit status 1
[ceph-node1-mon-centos-6-4][ERROR ] curl: (7) couldn't connect to host
[ceph-node1-mon-centos-6-4][ERROR ] error: https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc: import read failed(2).
[ceph_deploy][ERROR ] RuntimeError: Failed to execute command: su -c 'rpm --import "https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc"'
Does anyone know why?
Thanks a lot in advance!
Nar
From:alistair.whittle@xxxxxxxxxxxx [mailto:alistair.whittle@xxxxxxxxxxxx]
Sent: Tuesday, October 29, 2013 12:08 PM
To: joseph.r.gruher@xxxxxxxxx; Trivedi, Narendra; ceph-users@xxxxxxxxxxxxxx
Subject: RE: ceph-deploy problems on CentOS-6.4
You also want to make sure that if you are using a proxy your proxy settings are maintained through sudo.
With my deployment I had to add a line to my sudoers file to specify that the https_proxy and http_proxy settings are maintained. Didn’t work otherwise.
Defaults env_keep += "http_proxy https_proxy"
From:ceph-users-bounces@xxxxxxxxxxxxxx [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Gruher, Joseph R
Sent: Tuesday, October 29, 2013 4:55 PM
To: Trivedi, Narendra; ceph-users@xxxxxxxxxxxxxx
Subject: Re: [ceph-users] ceph-deploy problems on CentOS-6.4
If you are behind a proxy try configuring the wget proxy through /etc/wgetrc. I had a similar problem where I could complete wget commands manually but they would fail in ceph-deploy until I configured the wget proxy in that manner.
From: ceph-users-bounces@xxxxxxxxxxxxxx [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Trivedi, Narendra
Sent: Tuesday, October 29, 2013 9:51 AM
To: ceph-users@xxxxxxxxxxxxxx
Subject: [ceph-users] ceph-deploy problems on CentOS-6.4
Hi All,
I am a newbie to ceph. I am installing ceph (dumpling release) using ceph-deploy (issued from my admin node) on one monitor and two OSD nodes running CentOS 6.4 (64-bit) using followed instructions in the link below:
http://ceph.com/docs/master/start/quick-ceph-deploy/
My setup looks exactly like the diagram. I followed pre-flight instructions exacty as outlined in the link below:
http://ceph.com/docs/master/start/quick-start-preflight/
The ceph-deploy takes forever and then throws up the following error:
2013-10-28 17:32:35,903 [ceph_deploy.cli][INFO ] Invoked (1.2.7): /usr/bin/ceph-deploy new ceph-node1-mon-centos-6-4
2013-10-28 17:32:35,904 [ceph_deploy.new][DEBUG ] Creating new cluster named ceph
2013-10-28 17:32:35,904 [ceph_deploy.new][DEBUG ] Resolving host ceph-node1-mon-centos-6-4
2013-10-28 17:32:35,904 [ceph_deploy.new][DEBUG ] Monitor ceph-node1-mon-centos-6-4 at 10.12.0.70
2013-10-28 17:32:35,904 [ceph_deploy.new][DEBUG ] Monitor initial members are ['ceph-node1-mon-centos-6-4']
2013-10-28 17:32:35,904 [ceph_deploy.new][DEBUG ] Monitor addrs are ['10.12.0.70']
2013-10-28 17:32:35,905 [ceph_deploy.new][DEBUG ] Creating a random mon key...
2013-10-28 17:32:35,905 [ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf...
2013-10-28 17:32:35,905 [ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring...
2013-10-28 17:33:10,287 [ceph_deploy.cli][INFO ] Invoked (1.2.7): /usr/bin/ceph-deploy install ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 ceph-admin-node-centos-6-4
2013-10-28 17:33:10,287 [ceph_deploy.install][DEBUG ] Installing stable version dumpling on cluster ceph hosts ceph-node1-mon-centos-6-4 ceph-node2-osd0-centos-6-4 ceph-admin-node-centos-6-4
2013-10-28 17:33:10,288 [ceph_deploy.install][DEBUG ] Detecting platform for host ceph-node1-mon-centos-6-4 ...
2013-10-28 17:33:10,288 [ceph_deploy.sudo_pushy][DEBUG ] will use a remote connection without sudo
2013-10-28 17:33:10,626 [ceph_deploy.install][INFO ] Distro info: CentOS 6.4 Final
2013-10-28 17:33:10,626 [ceph-node1-mon-centos-6-4][INFO ] installing ceph on ceph-node1-mon-centos-6-4
2013-10-28 17:33:10,633 [ceph-node1-mon-centos-6-4][INFO ] adding EPEL repository
2013-10-28 17:33:10,633 [ceph-node1-mon-centos-6-4][INFO ] Running command: wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
2013-10-28 19:20:35,893 [ceph-node1-mon-centos-6-4][ERROR ] Traceback (most recent call last):
2013-10-28 19:20:35,894 [ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib/python2.6/site-packages/ceph_deploy/hosts/centos/install.py", line 77, in install_epel
2013-10-28 19:20:35,899 [ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib/python2.6/site-packages/ceph_deploy/util/decorators.py", line 10, in inner
2013-10-28 19:20:35,900 [ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib/python2.6/site-packages/ceph_deploy/util/wrappers.py", line 6, in remote_call
2013-10-28 19:20:35,902 [ceph-node1-mon-centos-6-4][ERROR ] File "/usr/lib64/python2.6/subprocess.py", line 502, in check_call
2013-10-28 19:20:35,903 [ceph-node1-mon-centos-6-4][ERROR ] raise CalledProcessError(retcode, cmd)
2013-10-28 19:20:35,904 [ceph-node1-mon-centos-6-4][ERROR ] CalledProcessError: Command '['wget', 'http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm']' returned non-zero exit status 4
2013-10-28 19:20:35,911 [ceph-node1-mon-centos-6-4][ERROR ] --2013-10-28 17:33:10-- http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
2013-10-28 19:20:35,911 [ceph-node1-mon-centos-6-4][ERROR ] Resolving dl.fedoraproject.org... 209.132.181.25, 209.132.181.26, 209.132.181.27, ...
2013-10-28 19:20:35,912 [ceph-node1-mon-centos-6-4][ERROR ] Connecting to dl.fedoraproject.org|209.132.181.25|:80... failed: Connection timed out.
2013-10-28 19:20:35,912 [ceph-node1-mon-centos-6-4][ERROR ] Connecting to dl.fedoraproject.org|209.132.181.26|:80... failed: Connection timed out.
Interestingly, “wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm” on each nodes (1 mon and 2 OSDs) succeeds without any problem. I have tried everything many times with root user, ceph user etc. but it fails every time! It is very frustrating!
Has anyone else experienced the same or similar problem?
Thanks a lot in advance!
Nar
This message contains information which may be confidential and/or privileged. Unless you are the intended recipient (or authorized to receive for the intended recipient), you may not read, use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail and delete the message and any attachment(s) thereto without retaining any copies.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
This message contains information which may be confidential and/or privileged. Unless you are the intended recipient (or authorized to receive for the intended recipient), you may not read, use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail and delete the message and any attachment(s) thereto without retaining any copies.
_______________________________________________
This message is for information purposes only, it is not a recommendation, advice, offer or solicitation to buy or sell a product or service nor an official confirmation of any transaction. It is directed at persons who are professionals and is not intended for retail customer use. Intended for recipient only. This message is subject to the terms at: www.barclays.com/emaildisclaimer.
For important disclosures, please see: www.barclays.com/salesandtradingdisclaimer regarding market commentary from Barclays Sales and/or Trading, who are active market participants; and in respect of Barclays Research, including disclosures relating to specific issuers, please see http://publicresearch.barclays.com.
_______________________________________________
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
