I am running into an issues connecting hadoop to my ceph cluster and I'm sure I am missing something but can't figure it out.
I have a Ceph cluster with MDS running fine and I can do a basic mount perfectly normal.
I have hadoop fs -ls with basic file:/// working well.
Info:
ceph cluster version 0.61.7
Ubuntu Server 13.04 x86_64
hadoop 1.2.1-1 deb install (stable now I did try 1.1.2 same issue)
libcephfs-java both hadoop-cephfs.jar and libcephfs.jar show up in "hadoop classpath"
libcephfs-jni with symlink trick /usr/share/hadoop/lib/native/Linux-amd64-64 listed here http://thread.gmane.org/gmane.comp.file-systems.ceph.user/1788/focus=1806 and the LD_LIBRARY_PATH in hadoop-env.sh
When I try to setup the ceph mount within Hadoop I get an exception
$ hadoop fs -ls
Exception in thread "main" java.lang.NoClassDefFoundError: com/ceph/fs/CephPoolException
at org.apache.hadoop.fs.ceph.CephFileSystem.initialize(CephFileSystem.java:96)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1446)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:67)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1464)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:263)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:124)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:247)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.hadoop.fs.FsShell.ls(FsShell.java:583)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:1812)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1916)
Caused by: java.lang.ClassNotFoundException: com.ceph.fs.CephPoolException
Followed the tutorial here
core-site.xml settings
...
<property>
<name>fs.ceph.impl</name>
<value>org.apache.hadoop.fs.ceph.CephFileSystem</value>
</property>
<property>
<name>fs.default.name</name>
<value>ceph://192.168.1.11:6789</value>
</property>
<property>
<name>ceph.data.pools</name>
<value>hadoop1</value>
</property>
<property>
<name>ceph.auth.id</name>
<value>admin</value>
</property>
<property>
<name>ceph.auth.keyfile</name>
<value>/etc/ceph/admin.secret</value>
</property>
Any Help Appreciated
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
