Hi ceph-devel, I came up with the following problems when I tried to let Hadoop use Ceph. Could you give any advice? Thanks firstly! I installed ceph-0.20.2 and hadoop.0.20.205.0 with the hadoop.0.20.205.0-cephfs1.patch successfully. Both ceph and hadoop(using HDFS) can work well separately. Besides, I could run command like 'hadoop fs -ls /', 'hadoop fs -put /localfilepath /cephfilepath' correctly. But when I run a mapreduce demo: wordcount with a small file containing about two thousand words, reduce task crashed reporting information like: A fatal error has been detected by the Java Runtime Environment:...Problematic frame:C [libceph.so.1.0.0+0x898e7]... I thought it was because ceph version is a little older compared with hadoop version. So I tried ceph-0.41.0 and ceph-0.30.0. However, when I run command 'hadoop fs -ls /' with ceph-0.41.0, a FATAL error occurred: FATAL ceph.CephFileSystem: initialize:Ceph initialization failed! After tracing the source code, I found the failure was due to client failing to authenticate and establish communication with ceph. I was not quite sure about the reason leading to this failure. And when I tried ceph-0.30.0, I got the following when I run 'hadoop fs -ls /', a little like the first one: A fatal error has been detected by the Java Runtime Environment:...Problematic frame:C [libceph.so.1.0.0+0x929fe]... I found that ceph-0.41.0 can make libcephfs.so directly, which hadoop looking for. While ceph-0.20.2 and ceph-0.30.0 just make libceph.so out. So I used 'ln -s libceph.so libcephfs.so' to bypass this problem. The above is what I got when I tried to let Hadoop use Ceph. And ceph-0.20.2 led me to the best point among the three versions. I did this according to the paper 'Ceph as a scalable alternative to the Hadoop Distributed File System' and the webpage 'http://ceph.newdream.net/wiki/Hadoop_FileSystem'. The webpage did not point out the version of Ceph, which I think is important to my trial. Now, I really need your suggestion. And thanks for your attention. Hoping to get your reply soon! Best,Xiaobing -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
