On Tue, Dec 3, 2013 at 10:21 AM, Mark Nelson <mark.nelson@xxxxxxxxxxx> wrote: > Guys, I don't think we have pre-released packages of anything new that is > going to work on the pi regardless if you use ceph-deploy. Look at our > armhf packages file: > > http://ceph.com/debian-emperor/dists/wheezy/main/binary-armhf/Packages > > Unless I'm mistaken, you're going to have to compile it yourself. I think > Joao was going to try that, not sure if he ever got around to it though. Oh, good point. I was just assuming that because we support the Distro the packages would exist. > > Mark > > > On 12/03/2013 09:03 AM, Alfredo Deza wrote: >> >> On Tue, Dec 3, 2013 at 9:56 AM, Shlomo Dubrowin <shlomo@xxxxxxxxxxxx> >> wrote: >>> >>> Alfredo, >>> >>> I started that way, but I run into an error: >>> >>> $ ceph-deploy install baxter >>> [ceph_deploy.cli][INFO ] Invoked (1.3.3): /usr/bin/ceph-deploy install >>> baxter >>> [ceph_deploy.install][DEBUG ] Installing stable version emperor on >>> cluster >>> ceph hosts baxter >>> [ceph_deploy.install][DEBUG ] Detecting platform for host baxter ... >>> [baxter][DEBUG ] connected to host: baxter >>> [baxter][DEBUG ] detect platform information from remote host >>> [baxter][DEBUG ] detect machine type >>> [ceph_deploy.install][INFO ] Distro info: debian 7.0 wheezy >>> [baxter][INFO ] installing ceph on baxter >>> [baxter][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive >>> apt-get -q install --assume-yes ca-certificates >>> [baxter][DEBUG ] Reading package lists... >>> [baxter][DEBUG ] Building dependency tree... >>> [baxter][DEBUG ] Reading state information... >>> [baxter][DEBUG ] ca-certificates is already the newest version. >>> [baxter][DEBUG ] 0 upgraded, 0 newly installed, 0 to remove and 85 not >>> upgraded. >>> [baxter][INFO ] Running command: sudo wget -O release.asc >>> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc >>> [baxter][WARNIN] --2013-12-03 16:32:45-- >>> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc >>> [baxter][WARNIN] Resolving ceph.com (ceph.com)... 208.113.241.137 >>> [baxter][WARNIN] Connecting to ceph.com >>> (ceph.com)|208.113.241.137|:443... >>> connected. >>> [baxter][WARNIN] HTTP request sent, awaiting response... 200 OK >>> [baxter][WARNIN] Length: unspecified [text/plain] >>> [baxter][WARNIN] Saving to: `release.asc' >>> [baxter][WARNIN] >>> [baxter][WARNIN] 0K . >>> 1.06M=0.002s >>> [baxter][WARNIN] >>> [baxter][WARNIN] 2013-12-03 16:32:53 (1.06 MB/s) - `release.asc' saved >>> [1752] >>> [baxter][WARNIN] >>> [baxter][INFO ] Running command: sudo apt-key add release.asc >>> [baxter][DEBUG ] OK >>> [baxter][DEBUG ] add ceph deb repo to sources.list >>> [baxter][INFO ] Running command: sudo apt-get -q update >>> [baxter][DEBUG ] Get:1 http://mirrordirector.raspbian.org wheezy >>> Release.gpg >>> [490 B] >>> [baxter][DEBUG ] Hit http://raspberrypi.collabora.com wheezy Release.gpg >>> [baxter][DEBUG ] Get:2 http://archive.raspberrypi.org wheezy Release.gpg >>> [490 B] >>> [baxter][DEBUG ] Get:3 http://mirrordirector.raspbian.org wheezy Release >>> [14.4 kB] >>> [baxter][DEBUG ] Hit http://raspberrypi.collabora.com wheezy Release >>> [baxter][DEBUG ] Get:4 http://ceph.com wheezy Release.gpg [836 B] >>> [baxter][DEBUG ] Get:5 http://archive.raspberrypi.org wheezy Release >>> [7224 >>> B] >>> [baxter][DEBUG ] Hit http://raspberrypi.collabora.com wheezy/rpi armhf >>> Packages >>> [baxter][DEBUG ] Get:6 http://mirrordirector.raspbian.org wheezy/main >>> armhf >>> Packages [7414 kB] >>> [baxter][DEBUG ] Get:7 http://archive.raspberrypi.org wheezy/main armhf >>> Packages [12.1 kB] >>> [baxter][DEBUG ] Get:8 http://ceph.com wheezy Release [5984 B] >>> [baxter][DEBUG ] Ign http://raspberrypi.collabora.com wheezy/rpi >>> Translation-en >>> [baxter][DEBUG ] Get:9 http://ceph.com wheezy/main armhf Packages [1010 >>> B] >>> [baxter][DEBUG ] Ign http://archive.raspberrypi.org wheezy/main >>> Translation-en >>> [baxter][DEBUG ] Ign http://ceph.com wheezy/main Translation-en >>> [baxter][DEBUG ] Hit http://mirrordirector.raspbian.org wheezy/contrib >>> armhf >>> Packages >>> [baxter][DEBUG ] Hit http://mirrordirector.raspbian.org wheezy/non-free >>> armhf Packages >>> [baxter][DEBUG ] Hit http://mirrordirector.raspbian.org wheezy/rpi armhf >>> Packages >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/contrib >>> Translation-en >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/main >>> Translation-en >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/non-free >>> Translation-en >>> [baxter][DEBUG ] Ign http://mirrordirector.raspbian.org wheezy/rpi >>> Translation-en >>> [baxter][DEBUG ] Fetched 7456 kB in 47s (157 kB/s) >>> [baxter][DEBUG ] Reading package lists... >>> [baxter][INFO ] Running command: sudo env DEBIAN_FRONTEND=noninteractive >>> DEBIAN_PRIORITY=critical apt-get -q -o Dpkg::Options::=--force-confnew >>> --no-install-recommends --assume-yes install -- ceph ceph-mds ceph-common >>> ceph-fs-common gdisk >>> [baxter][WARNIN] E: Unable to locate package ceph-mds >>> [baxter][WARNIN] E: Unable to locate package ceph-fs-common >>> [baxter][DEBUG ] Reading package lists... >>> [baxter][DEBUG ] Building dependency tree... >>> [baxter][DEBUG ] Reading state information... >>> [baxter][ERROR ] RuntimeError: command returned non-zero exit status: 100 >>> [ceph_deploy][ERROR ] RuntimeError: Failed to execute command: env >>> DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get -q -o >>> Dpkg::Options::=--force-confnew --no-install-recommends --assume-yes >>> install >>> -- ceph ceph-mds ceph-common ceph-fs-common gdisk >> >> >> That output looks unexpected. I wonder if it is just a network hiccup >> to fail to get to those two packages. >>> >>> >>> >>> When running manually: >>> >>> $ env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical apt-get -q >>> -o >>> Dpkg::Options::=--force-confnew --no-install-recommends --assume-yes >>> install >>> -- ceph ceph-mds ceph-common ceph-fs-common gdisk >>> E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission >>> denied) >>> E: Unable to lock the administration directory (/var/lib/dpkg/), are you >>> root? >>> >>> >>> Maybe this is supposed to be a sudo command? >> >> >> Yes, always with sudo unless you are root, which is the reason why >> ceph-deploy will not use that in the output. >> >>> >>> $ sudo env DEBIAN_FRONTEND=noninteractive DEBIAN_PRIORITY=critical >>> apt-get >>> -q -o Dpkg::Options::=--force-confnew --no-install-recommends >>> --assume-yes >>> install -- ceph ceph-mds ceph-common ceph-fs-common gdisk >>> Reading package lists... >>> Building dependency tree... >>> Reading state information... >>> E: Unable to locate package ceph-mds >>> E: Unable to locate package ceph-fs-common >>> >>> >>> This was the stage where I tried to install ceph with just apt-get >>> install >>> manually. >> >> >> If you run a `sudo apt-get update` does that come back correctly or >> does it complain? >> >>> >>> Shlomo >>> >>> >>> ----------------------------- >>> Shlomo Dubrowin >>> >>> The Solution to the water crisis in Israel: >>> >>> # According to WikiPedia, the Kinneret can hold >>> # 4 km^3, so FULL here is in cubit meters >>> FULL="4000" >>> while [ "$LEVEL" -lt "$FULL" ]; do >>> cat /sea/med /sea/red |\ >>> grep -vi "salt" |\ >>> tee /sea/dead /lake/kinneret >>> LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print $1}'` >>> done >>> >>> >>> On Tue, Dec 3, 2013 at 4:29 PM, Alfredo Deza <alfredo.deza@xxxxxxxxxxx> >>> wrote: >>>> >>>> >>>> On Tue, Dec 3, 2013 at 9:21 AM, Shlomo Dubrowin <shlomo@xxxxxxxxxxxx> >>>> wrote: >>>>> >>>>> Alfredo, >>>>> >>>>> Thank you for your response. I simply did apt-get install ceph on the >>>>> nodes. >>>>> >>>>> My /etc/apt/sources.list.d/ceph.list contains: >>>>> >>>>> deb http://ceph.com/debian-emperor/ wheezy main >>>>> >>>> >>>> Was that added manually? ceph-deploy can take care of handling the >>>> sources list for you. >>>> >>>> It is possible that you have something else in that machine that is >>>> saying that 0.43 is the version you should >>>> be getting. >>>> >>>> Have you tried to install with ceph-deploy?: >>>> >>>> ceph-deploy install baxter >>>> >>>> >>>> >>>>> >>>>> and the versions I received are what I got. >>>>> >>>>> Shlomo >>>>> >>>>> ----------------------------- >>>>> Shlomo Dubrowin >>>>> >>>>> The Solution to the water crisis in Israel: >>>>> >>>>> # According to WikiPedia, the Kinneret can hold >>>>> # 4 km^3, so FULL here is in cubit meters >>>>> FULL="4000" >>>>> while [ "$LEVEL" -lt "$FULL" ]; do >>>>> cat /sea/med /sea/red |\ >>>>> grep -vi "salt" |\ >>>>> tee /sea/dead /lake/kinneret >>>>> LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print >>>>> $1}'` >>>>> done >>>>> >>>>> >>>>> On Tue, Dec 3, 2013 at 4:15 PM, Alfredo Deza <alfredo.deza@xxxxxxxxxxx> >>>>> wrote: >>>>>> >>>>>> >>>>>> On Tue, Dec 3, 2013 at 8:55 AM, Shlomo Dubrowin <shlomo@xxxxxxxxxxxx> >>>>>> wrote: >>>>>>> >>>>>>> I'm trying to deploy Ceph on a group of Raspberry Pis using the >>>>>>> procedure >>>>>>> documented in: http://ceph.com/docs/master/start/quick-ceph-deploy/ >>>>>>> >>>>>>> There used to be a site: >>>>>>> http://ceph.com/docs/master/start/quick-start/ >>>>>>> but >>>>>>> that page is no longer valid. >>>>>>> >>>>>>> The first thing I noticed is that the command lsb_release -sc >>>>>>> specified >>>>>>> in >>>>>>> the Ceph Deploy setup provides n/a even though the answer should be >>>>>>> wheezy. >>>>>>> I manually changed the /etc/apt/sources.list.d/ceph.list to specify >>>>>>> wheezy. >>>>>>> >>>>>>> I've installed ceph-deploy and setup the keys to communicate between >>>>>>> the >>>>>>> deploy Raspberry Pi and all the Raspberry Pis including the system >>>>>>> reunning >>>>>>> ceph-deploy. >>>>>>> >>>>>>> When I tried to run ceph-deploy install <node1> the installation >>>>>>> failed. >>>>>>> I went to each node and installed ceph manually and I see the >>>>>>> versions >>>>>>> are: >>>>>>> >>>>>>> $ ceph-deploy --version >>>>>>> 1.3.3 >>>>>>> >>>>>>> $ ceph --version >>>>>>> ceph version 0.43 (commit:9fa8781c0147d66fcef7c2dd0e09cd3c69747d37) >>>>>>> >>>>>>> >>>>>>> All the nodes have the same ceph version. >>>>>> >>>>>> >>>>>> That looks like a *very* old ceph version. Is there any reason you are >>>>>> using 0.43 as opposed to the latest one? >>>>>> >>>>>> How did you installed Ceph on those nodes? >>>>>>> >>>>>>> >>>>>>> When I try to run a command, I get errors: >>>>>>> >>>>>>> $ ceph-deploy mon create baxter >>>>>>> [ceph_deploy.cli][INFO ] Invoked (1.3.3): /usr/bin/ceph-deploy mon >>>>>>> create >>>>>>> baxter >>>>>>> [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts baxter >>>>>>> [ceph_deploy.mon][DEBUG ] detecting platform for host baxter ... >>>>>>> [baxter][DEBUG ] connected to host: baxter >>>>>>> [baxter][DEBUG ] detect platform information from remote host >>>>>>> [baxter][DEBUG ] detect machine type >>>>>>> [ceph_deploy.mon][INFO ] distro info: debian 7.0 wheezy >>>>>>> [baxter][DEBUG ] determining if provided host has same hostname in >>>>>>> remote >>>>>>> [baxter][DEBUG ] get remote short hostname >>>>>>> [baxter][DEBUG ] deploying mon to baxter >>>>>>> [baxter][DEBUG ] get remote short hostname >>>>>>> [baxter][DEBUG ] remote hostname: baxter >>>>>>> [baxter][DEBUG ] write cluster configuration to >>>>>>> /etc/ceph/{cluster}.conf >>>>>>> [baxter][DEBUG ] create the mon path if it does not exist >>>>>>> [baxter][DEBUG ] checking for done path: >>>>>>> /var/lib/ceph/mon/ceph-baxter/done >>>>>>> [baxter][DEBUG ] done path does not exist: >>>>>>> /var/lib/ceph/mon/ceph-baxter/done >>>>>>> [baxter][INFO ] creating keyring file: >>>>>>> /var/lib/ceph/tmp/ceph-baxter.mon.keyring >>>>>>> [baxter][DEBUG ] create the monitor keyring file >>>>>>> [baxter][INFO ] Running command: sudo ceph-mon --cluster ceph --mkfs >>>>>>> -i >>>>>>> baxter --keyring /var/lib/ceph/tmp/ceph-baxter.mon.keyring >>>>>>> [baxter][WARNIN] too many arguments: [--cluster,ceph] >>>>>>> [baxter][WARNIN] usage: ceph-mon -i monid [--mon-data=pathtodata] >>>>>>> [flags] >>>>>>> [baxter][WARNIN] --debug_mon n >>>>>>> [baxter][WARNIN] debug monitor level (e.g. 10) >>>>>>> [baxter][WARNIN] --mkfs >>>>>>> [baxter][WARNIN] build fresh monitor fs >>>>>>> [baxter][DEBUG ] --conf/-c Read configuration from the given >>>>>>> configuration file >>>>>>> [baxter][DEBUG ] -d Run in foreground, log to stderr. >>>>>>> [baxter][DEBUG ] -f Run in foreground, log to usual >>>>>>> location. >>>>>>> [baxter][DEBUG ] --id/-i set ID portion of my name >>>>>>> [baxter][DEBUG ] --name/-n set name (TYPE.ID) >>>>>>> [baxter][DEBUG ] --version show version and quit >>>>>>> [baxter][DEBUG ] >>>>>>> [baxter][DEBUG ] --debug_ms N >>>>>>> [baxter][DEBUG ] set message debug level (e.g. 1) >>>>>>> [baxter][ERROR ] RuntimeError: command returned non-zero exit status: >>>>>>> 1 >>>>>>> [ceph_deploy.mon][ERROR ] Failed to execute command: ceph-mon >>>>>>> --cluster >>>>>>> ceph >>>>>>> --mkfs -i baxter --keyring /var/lib/ceph/tmp/ceph-baxter.mon.keyring >>>>>>> [ceph_deploy][ERROR ] GenericError: Failed to create 1 monitors >>>>>>> >>>>>>> >>>>>>> If I try to run the same command as the user ceph on the local >>>>>>> machine, >>>>>>> I >>>>>>> get the same error: >>>>>>> >>>>>>> $ ceph-mon --cluster ceph --mkfs -i baxter --keyring >>>>>>> /var/lib/ceph/tmp/ceph-baxter.mon.keyring >>>>>>> failed to open log file '/var/log/ceph/mon.baxter.log': (13) >>>>>>> Permission >>>>>>> denied >>>>>>> too many arguments: [--cluster,ceph] >>>>>>> usage: ceph-mon -i monid [--mon-data=pathtodata] [flags] >>>>>>> --debug_mon n >>>>>>> debug monitor level (e.g. 10) >>>>>>> --mkfs >>>>>>> build fresh monitor fs >>>>>>> --conf/-c Read configuration from the given configuration file >>>>>>> -d Run in foreground, log to stderr. >>>>>>> -f Run in foreground, log to usual location. >>>>>>> --id/-i set ID portion of my name >>>>>>> --name/-n set name (TYPE.ID) >>>>>>> --version show version and quit >>>>>>> >>>>>>> --debug_ms N >>>>>>> set message debug level (e.g. 1) >>>>>>> >>>>>>> >>>>>>> >>>>>>> This makes me think that the ceph-deploy installed doesn't work with >>>>>>> the >>>>>>> ceph installed on each node. >>>>>>> >>>>>>> So I guess I have 2 questions: >>>>>>> >>>>>>> How can I determine which ceph-deploy goes with which ceph and how >>>>>>> can I >>>>>>> correct this apparent mismatch? >>>>>> >>>>>> >>>>>> I don't think there was a ceph-deploy per-se that would've worked for >>>>>> that Ceph version. >>>>>> >>>>>> I believe that ceph-deploy is very backwards compatible for a few Ceph >>>>>> versions. At the very least we should >>>>>> fully support the latest 3 major versions and we special case a few >>>>>> commands depending on the version installed. >>>>>> >>>>>> But like I mentioned, the Ceph version you have installed seems like a >>>>>> very very old one. >>>>>> >>>>>>> Should I be looking at just using local ceph commands to configure >>>>>>> this >>>>>>> by >>>>>>> hand? Are there instructions on how to do this by hand? >>>>>> >>>>>> >>>>>> If you must have that version I don't think there is any option here >>>>>> but to do it by hand and avoid ceph-deploy. >>>>>> >>>>>> For docs, I think that might be even more difficult to do as you would >>>>>> need to go to the tag for that version and look at the docs >>>>>> for that tag in the repository. >>>>>> >>>>>> Then again, what is the reason for using that version? >>>>>> >>>>>>> >>>>>>> Thank you. >>>>>>> >>>>>>> Shlomo >>>>>>> >>>>>>> ----------------------------- >>>>>>> Shlomo Dubrowin >>>>>>> >>>>>>> The Solution to the water crisis in Israel: >>>>>>> >>>>>>> # According to WikiPedia, the Kinneret can hold >>>>>>> # 4 km^3, so FULL here is in cubit meters >>>>>>> FULL="4000" >>>>>>> while [ "$LEVEL" -lt "$FULL" ]; do >>>>>>> cat /sea/med /sea/red |\ >>>>>>> grep -vi "salt" |\ >>>>>>> tee /sea/dead /lake/kinneret >>>>>>> LEVEL=`du -c /sea/dead /lake/kinneret | grep total | awk '{print >>>>>>> $1}'` >>>>>>> done >>>>>>> >>>>>>> _______________________________________________ >>>>>>> ceph-users mailing list >>>>>>> ceph-users@xxxxxxxxxxxxxx >>>>>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>>>>>> >>>>> >>>>> >>> >>> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |