How are you invoking Hadoop? Also, I forgot to ask, are you using the wrappers located in github.com/ceph/hadoop-common (or the jar linked to on http://ceph.com/docs/master/cephfs/hadoop/)? On Mon, Sep 23, 2013 at 12:05 PM, Rolando Martins <rolando.martins@xxxxxxxxx> wrote: > Hi Noah, > I enabled the debugging and got: > > 2013-09-23 18:59:34.705894 7f0b58de7700 -1 monclient(hunting): ERROR: > missing keyring, cannot use cephx for authentication > 2013-09-23 18:59:34.706106 7f0b58de7700 20 client.-1 trim_cache size 0 max 0 > 2013-09-23 18:59:34.706225 7f0b58de7700 10 jni: ceph_mount: exit ret -2 > > I have the ceph.client.admin.keyring file in /etc/ceph and I tried > with and without the > parameter in core-site.xml. Unfortunately without success:( > > Thanks, > Rolando > > > <property> > <name>fs.ceph.impl</name> > <value>org.apache.hadoop.fs.ceph.CephFileSystem</value> > </property> > > <property> > <name>fs.default.name</name> > <value>ceph://hyrax1:6789/</value> > </property> > > <property> > <name>ceph.conf.file</name> > <value>/hyrax/hadoop-ceph/ceph/ceph.conf</value> > </property> > > <property> > <name>ceph.root.dir</name> > <value>/</value> > </property> > <property> > <name>ceph.auth.keyring</name> > <value>/hyrax/hadoop-ceph/ceph/ceph.client.admin.keyring</value> > </property> > > On Mon, Sep 23, 2013 at 2:24 PM, Noah Watkins <noah.watkins@xxxxxxxxxxx> wrote: >> Shoot, I thought I had it figured out :) >> >> There is a default admin user created when you first create your >> cluster. After a typical install via ceph-deploy, there should be a >> file called 'ceph.client.admin.keyring', usually sibling to ceph.conf. >> If this is in a standard location (e.g. /etc/ceph) you shouldn't need >> the keyring option, otherwise point 'ceph.auth.keyring' at that >> keyring file. You shouldn't need both the keyring and the keyfile >> options set, but it just depends on how your authentication / users >> are all setup. >> >> The easiest thing to do if that doesn't solve your problem is probably >> to turn on logging so we can see what is blowing up. >> >> In your ceph.conf you can add 'debug client = 20' and 'debug >> javaclient = 20' to the client section. You may also need to set the >> log file 'log file = /path/...'. You don't need to do this on all your >> nodes, just one node where you get the failure. >> >> - Noah >> >>> Thanks, >>> Rolando >>> >>> P.S.: I have the cephFS mounted locally, so the cluster is ok. >>> >>> cluster d9ca74d0-d9f4-436d-92de-762af67c6534 >>> health HEALTH_OK >>> monmap e1: 9 mons at >>> {hyrax1=10.10.10.10:6789/0,hyrax2=10.10.10.12:6789/0,hyrax3=10.10.10.15:6789/0,hyrax4=10.10.10.13:6789/0,hyrax5=10.10.10.16:6789/0,hyrax6=10.10.10.14:6789/0,hyrax7=10.10.10.18:6789/0,hyrax8=10.10.10.17:6789/0,hyrax9=10.10.10.11:6789/0}, >>> election epoch 6, quorum 0,1,2,3,4,5,6,7,8 >>> hyrax1,hyrax2,hyrax3,hyrax4,hyrax5,hyrax6,hyrax7,hyrax8,hyrax9 >>> osdmap e30: 9 osds: 9 up, 9 in >>> pgmap v2457: 192 pgs: 192 active+clean; 10408 bytes data, 44312 MB >>> used, 168 GB / 221 GB avail >>> mdsmap e4: 1/1/1 up {0=hyrax1=up:active} >>> >>> >>> <property> >>> <name>fs.ceph.impl</name> >>> <value>org.apache.hadoop.fs.ceph.CephFileSystem</value> >>> </property> >>> >>> <property> >>> <name>fs.default.name</name> >>> <value>ceph://hyrax1:6789/</value> >>> </property> >>> >>> <property> >>> <name>ceph.conf.file</name> >>> <value>/hyrax/hadoop-ceph/ceph/ceph.conf</value> >>> </property> >>> >>> <property> >>> <name>ceph.root.dir</name> >>> <value>/</value> >>> </property> >>> >>> <property> >>> <name>ceph.auth.keyfile</name> >>> <value>/hyrax/hadoop-ceph/ceph/admin.secret</value> >>> </property> >>> >>> <property> >>> <name>ceph.auth.keyring</name> >>> <value>/hyrax/hadoop-ceph/ceph/ceph.mon.keyring</value> >>> </property> >>> >>> On Mon, Sep 23, 2013 at 11:42 AM, Noah Watkins <noah.watkins@xxxxxxxxxxx> wrote: >>>>> <property> >>>>> <name>ceph.root.dir</name> >>>>> <value>/mnt/mycephfs</value> >>>>> </property> >>>> >>>> This is probably causing the issue. Is this meant to be a local mount >>>> point? The 'ceph.root.dir' property specifies the root directory >>>> /inside/ CephFS, and the Hadoop implementation doesn't require a local >>>> CephFS mount--it uses a client library to interact with the file >>>> system. >>>> >>>> The default value for this property is "/", so you can probably just >>>> remove this from your config file unless your CephFS directory >>>> structure is carved up in a special way. >>>> >>>>> <property> >>>>> <name>ceph.conf.file</name> >>>>> <value>/hyrax/hadoop-ceph/ceph/ceph.conf</value> >>>>> </property> >>>>> <property> >>>>> <name>ceph.auth.keyfile</name> >>>>> <value>/hyrax/hadoop-ceph/ceph/admin.secret</value> >>>>> </property> >>>>> >>>>> <property> >>>>> <name>ceph.auth.keyring</name> >>>>> <value>/hyrax/hadoop-ceph/ceph/ceph.mon.keyring</value> >>>>> </property> >>>> >>>> These files will need to be available locally on every node Hadoop >>>> runs on. I think the error below will occur after these are loaded, so >>>> it probably isn't your issue, though I don't recall exactly at which >>>> point different configuration files are loaded. >>>> >>>>> <property> >>>>> <name>fs.hdfs.impl</name> >>>>> <value>org.apache.hadoop.fs.ceph.CephFileSystem</value> >>>>> </property> >>>> >>>> I don't think this is part of the problem you are seeing, but this >>>> 'fs.hdfs.impl' property should probably be removed. We aren't >>>> overriding HDFS, just replacing it. >>>> >>>>> <property> >>>>> <name>ceph.mon.address</name> >>>>> <value>hyrax1:6789</value> >>>>> </property> >>>> >>>> This was already specified in your 'fs.default.name' property. I don't >>>> think that duplicating it is an issue, but I should probably update >>>> the documentation to make it clear that the monitor only needs to be >>>> listed once. >>>> >>>> Thanks! >>>> Noah _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |