how to use hadoop when system is CephFS

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 




Hi all !
    I think I suceed using cephFS with hadoop, but I still face lost question for  that  I am a newer in hadoop and ceph!
    first, my ceph version is 0.62. java is 1.6.0_45 .hadoop is 1.1.2  I think suceed because:
      # hadoop fs -ls /
      drwxrwxrwx - root 0 2013-10-16 10:57 /libcephfs_junit
   but, when I use the commond  hadoop dfsadmin -report, nothing printed on the terminal
  []   #hadoop dfsadmin -report
  []  #
   and then , I try  some mapreduce's work!
   #hadoop jar hadoop-examples-1.1.2.jar wordcount /tmp/wordcount/word.txt /tmp/wordcount/out
   .........INFO ceph.CephFileSystem:...........
...
   ..........WARN Snappy.Load Dnappy: Snappy native library not loaded.
....
            INFO:map 0% reduce 0%
 and  Nothing was print on!  I wait for it  more then 10 Minute and the word.txt content less the 10 words.

by the way, the jps restut is:
#jps
2913     secondaryNameNode
17888   TaskTracker
4581 JPS
2974  JobTracker

and my hadoop  conf is
core-site.xml
hadoop.tmp.dir = /usr/hadoop/tmp
fs.ceph.impl =org.apache.hadoop.fs.ceph.CephFileSystem
fs.default.name = ceph://192.168.58.129/
ceph.conf.file = /etc/ceph/ceph.conf

mapres-site.xml
mapred.job.tracket=http://192.168.58.132:9001

ndfs-site.xml
dfs.replication=1


thanks
peng







  


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux