About cephfs with hadoop

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi, all
I am trying to use cephfs as a replacement drop-in for hadoop hdfs. Maily configure steps according to doc here : 
http://docs.ceph.com/docs/master/cephfs/hadoop/ 

I am using a 3 node hadoop 2.7.1 cluster. Noting the official doc recommend using 1.1.x stable release, I am not sure
if using 2.x release would cause some unexpected trouble. Anyway, I think I had configured right for cephfs and ceph
storage cluster goes well. 

When I am trying to start the hadoop cluster, namenode failed with the following exception: 
  2015-09-23 14:47:39,600 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode.
       java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): ceph://172.16.50.18:6789/ is not of scheme 'hdfs'.
at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:477)
at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:461)
at org.apache.hadoop.hdfs.server.namenode.NameNode.getRpcServerAddress(NameNode.java:512)
at org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(NameNode.java:612)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:632)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:811)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:795)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1488)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1554)
   

Quite weird here cause the Namenode check seems to NOT ALLOW any schema expect default hdfs. Then how would cephfs-hadoop work actually?

Really need your help here. 
Best,
Sun.



CertusNet


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux