On Tue, Dec 7, 2010 at 6:33 PM, Jeff Wu <cpwu@xxxxxxxxxxxxx> wrote:
>
>
>
>
>
>
> å 2010-12-08äç 09:53 +0800ïBrian Chrismanåéï
>> I've built and installed RPMs for ceph for RHEL6beta.
>> I've placed the below ceph.conf in /etc/ceph on both of my test nodes
>> (test10, test11).
>>
>> I build a ceph filesystem and hide the key
>> mkcephfs -c /etc/ceph/ceph.conf -a --mkbtrfs -k /etc/ceph/keyring.bin
>> cauthtool --print-key /etc/ceph/keyring.bin > /etc/ceph/secret
>> chmod 600 /etc/ceph/secret
>> scp -p /etc/ceph/secret test11:/etc/ceph
>>
>> Then I start the daemons on each node:
>> service ceph start
>>
>> My daemons start up on both nodes with 'service ceph -a start'
>> root   Â3199   1 Â0 17:18 ?    Â00:00:00 /usr/local/bin/cmon -i
>> 0 -c /tmp/ceph.conf.5365
>> root   Â3228   1 Â0 17:18 ?    Â00:00:00 /usr/local/bin/cmds -i
>> test10 -c /tmp/ceph.conf.5365
>> root   Â3285   1 Â0 17:18 ?    Â00:00:00 /usr/local/bin/cosd -i
>> 0 -c /tmp/ceph.conf.5365
>> (similar output on other node)
>>
>> I attempt to mount the ceph filesystem on test10 (using test11's IP):
>> mount -t ceph -o name=admin,secretfile=/etc/ceph/keyring.bin
>> 10.200.98.111:/ /mnt/ceph
>> mount error 5 = Input/output error
>>
>
> Hi ,i take the followng steps , fail to mount,too.
>
> ## save auth secret to a file:
> $cauthtool --print-key /etc/ceph/keyring.bin > /etc/ceph/secret
> chmod 600 /etc/ceph/secret
> ## future ,support,read a secret from a file
> $ mount -t ceph 172.16.50.10:6789:/foo /mnt/ceph
> Â -o name=admin,secretfile=secret
>
> but ,the following steps ,mount ceph is successful.
>
>
> # enable cephx, add a user and secret
> Â Â$ mount -t ceph -o name=admin,secret=<secret>
> Â Â Â 1.2.3.4:/ /mnt/ceph
> $ mount -t ceph 172.16.50.10:6791:/foo /mnt/ceph
> Â Â-o name=admin,secret=âAQArWtdMiI1uDRAAVbNRMeiwsjK+DEMeB7ewLg==â
>
> # cauthtool --list keyring.bin
> client.admin
> Âkey: AQArWtdMiI1uDRAAVbNRMeiwsjK+DEMeB7ewLg==
> auid: 0
> caps: [mds] allow
> caps: [mon] allow *
> caps: [osd] allow *
Same issue.. being that I'm seeing mon daemons timing out while
attempting to connect, I'm guessing there's an issue here other than
crypto..
[root@test10 ~]# mount -t ceph -o
name=admin,secret=AQC63/5MEMaTLBAAuiYetlbkHBjj0WNtAjOsQg==
10.200.98.111:/ /mnt/ceph
mount error 5 = Input/output error
[root@test10 ~]# cauthtool --list /etc/ceph/keyring.bin
client.admin
key: AQC63/5MEMaTLBAAuiYetlbkHBjj0WNtAjOsQg==
auid: 0
caps: [mds] allow
caps: [mon] allow *
caps: [osd] allow *
[root@test10 ~]#
>
>
>
>
>
>> /var/log/messages seems to show me what the problem is:
>> Dec Â7 17:45:19 test10 kernel: libceph: mon0 10.200.98.111:6789
>> connection failed
>> (a few more of those before mount fails)
>> on test11, the daemon is up and listening on that port:
>> tcp    Â0   Â0 10.200.98.111:6789     Â10.200.98.111:56805
>> Â Â ÂESTABLISHED 7781/cmon
>>
>> And here's /var/log/ceph/mon.1.log on test11 (the .111 node)
>> 2010-12-07 17:36:20.136921 --- 7780 opened log /var/log/ceph/mon.1.log ---
>> ceph version 0.24~rc (commit:378d13df9505e4ea9a32f42cb713cdcf7aaccda0)
>> 2010-12-07 17:36:20.137164 7f80c51e3720 store(/data/mon1) mount
>> 2010-12-07 17:36:20.138241 7f80c51e3720 mon.1@1(starting) e1 init fsid
>> 1b4cabdb-30d2-752d-005f-517a7fa982f8
>> 2010-12-07 17:36:20.165407 7f80c51e3720 log [INF] : mon.1 calling new
>> monitor election
>> 2010-12-07 17:36:20.192343 7f80c51e1710 -- 10.200.98.111:6789/0 >>
>> 10.200.98.110:6789/0 pipe(0x1cafd20 sd=6 pgs=0 cs=0 l=0).fault first
>> fault
>>
>> And for test10 (the .110 node)
>> 2010-12-07 17:36:49.183357 --- 5767 opened log /var/log/ceph/mon.0.log ---
>> ceph version 0.24~rc (commit:378d13df9505e4ea9a32f42cb713cdcf7aaccda0)
>> 2010-12-07 17:36:49.183545 7ff24669c720 store(/data/mon0) mount
>> 2010-12-07 17:36:49.184556 7ff24669c720 mon.0@0(starting) e1 init fsid
>> 1b4cabdb-30d2-752d-005f-517a7fa982f8
>> 2010-12-07 17:36:49.600650 7ff24669c720 log [INF] : mon.0 calling new
>> monitor election
>> 2010-12-07 17:36:49.645875 7ff24669a710 -- 10.200.98.110:6789/0 >>
>> 10.200.98.111:6789/0 pipe(0xac7d20 sd=6 pgs=0 cs=0 l=0).fault first
>> fault
>>
>>
>> Does this mean my cmon on 111 is getting into a state where it's not
>> receiving incoming connections?
>> Any suggestions on where to go from here?
>>
>> thanks,
>> Brian Chrisman
>>
>>
>>
>> ----- ceph.conf in /etc/ceph -----
>> ; From sample:
>> [global]
>> Â Â Â auth supported = cephx
>>
>> [mon]
>> Â Â Â mon data = /data/mon$id
>>
>> Â Â Â ; logging, for debugging monitor crashes, in order of
>> Â Â Â ; their likelihood of being helpful :)
>> Â Â Â ;debug ms = 1
>> Â Â Â ;debug mon = 20
>> Â Â Â ;debug paxos = 20
>> Â Â Â ;debug auth = 20
>>
>> [mon0]
>> Â Â Â host = test10
>> Â Â Â mon addr = 10.200.98.110:6789
>>
>> [mon1]
>> Â Â Â host = test11
>> Â Â Â mon addr = 10.200.98.111:6789
>>
>> [mds]
>> Â Â Â keyring = /data/keyring.$name
>> Â Â Â ;debug ms = 1
>> Â Â Â ;debug mds = 20
>>
>> [mds.test10]
>> Â Â Â host = test10
>>
>> [mds.test11]
>> Â Â Â host = test11
>>
>> [osd]
>> Â Â Â osd data = /data/osd$id
>> Â Â Â osd journal = /data/osd$id/journal
>> Â Â Â osd journal size = 1000 ; journal size, in megabytes
>> Â Â Â ;debug ms = 1
>> Â Â Â ;debug osd = 20
>> Â Â Â ;debug filestore = 20
>> Â Â Â ;debug journal = 20
>>
>> [osd0]
>> Â Â Â host = test10
>> Â Â Â btrfs devs = /dev/sdd4
>>
>> [osd1]
>> Â Â Â host = test11
>> Â Â Â btrfs devs = /dev/sdd4
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@xxxxxxxxxxxxxxx
>> More majordomo info at Âhttp://vger.kernel.org/majordomo-info.html
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
