Gregory Farnum <greg@...> writes: > > On Thu, Mar 19, 2015 at 5:57 PM, Ridwan Rashid <ridwan064@...> wrote: > > Hi, > > > > I have a 5 node ceph(v0.87) cluster and am trying to deploy hadoop with > > cephFS. I have installed hadoop-1.1.1 in the nodes and changed the > > conf/core-site.xml file according to the ceph documentation > > http://ceph.com/docs/master/cephfs/hadoop/ but after changing the file the > > namenode is not starting (namenode can be formatted) but the other > > services(datanode, jobtracker, tasktracker) are running in hadoop. > > > > The default hadoop works fine but when I change the core-site.xml file as > > above I get the following bindException as can be seen from the namenode log: > > > > > > 2015-03-19 01:37:31,436 ERROR > > org.apache.hadoop.hdfs.server.namenode.NameNode: java.net.BindException: > > Problem binding to node1/10.242.144.225:6789 : Cannot assign requested address > > > > > > I have one monitor for the ceph cluster (node1/10.242.144.225) and I > > included in the core-site.xml file ceph://10.242.144.225:6789 as the value > > of fs.default.name. The 6789 port is the default port being used by the > > monitor node of ceph, so that may be the reason for the bindException but > > the ceph documentation mentions that it should be included like this in the > > core-site.xml file. It would be really helpful to get some pointers to where > > I am doing wrong in the setup. > > I'm a bit confused. The NameNode is only used by HDFS, and so > shouldn't be running at all if you're using CephFS. Nor do I have any > idea why you've changed anything in a way that tells the NameNode to > bind to the monitor's IP address; none of the instructions that I see > can do that, and they certainly shouldn't be. > -Greg > Hi Greg, I want to run a hadoop job (e.g. terasort) and want to use cephFS instead of HDFS. In "Using Hadoop with cephFS" documentation in http://ceph.com/docs/master/cephfs/hadoop/ if you look into the Hadoop configuration section, the first property fs.default.name has to be set as the ceph URI and in the notes it's mentioned as ceph://[monaddr:port]/. My core-site.xml of hadoop conf looks like this <configuration> <property> <name>fs.default.name</name> <value>ceph://10.242.144.225:6789</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/app/hadoop/tmp</value> <description>A base for other temporary directories.</description> </property> <property> <name>fs.ceph.impl</name> <value>org.apache.hadoop.fs.ceph.CephFileSystem</value> <description> </description> </property> <property> <name>ceph.conf.file</name> <value>/etc/ceph/ceph.conf</value> </property> <property> <name>ceph.root.dir</name> <value>/</value> </property> <property> <name>ceph.mon.address</name> <value>10.242.144.225:6789</value> <description>This is the primary monitor node IP address in our installation.</description> </property> <property> <name>ceph.auth.id</name> <value>admin</value> </property> <property> <name>ceph.auth.keyring</name> <value>/etc/ceph/ceph.client.admin.keyring</value> </property> <property> <name>ceph.object.size</name> <value>67108864</value> </property> <property> <name>ceph.data.pools</name> <value>data</value> </property> <property> <name>ceph.localize.reads</name> <value>true</value> </property> </configuration> _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |