Re: guidance with hadoop on ceph

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Noah,

Thanks for your response. As I said, I am new to ceph, and deployed it using the 5-min quickstart only. After going through the reply, I tried to build ceph from source, as described in the official site. After cloning the source, I ran the following:
cd ceph
./autogen.sh
./configure --with-jdk-dir=/usr/lib/jvm/java-7-oracle --enable-cephfs-java
sudo make install

As expected, the executables were placed in /usr/local/bin. I created a ceph.conf file in /etc/ceph/ceph.conf, and a dir for each of the daemons in /var/lib/ceph folder.

How do I deploy this cluster? When I tried running mkcephfs, it displayed the following:

temp dir is /tmp/mkcephfs.Pqu0j1MQsc
preparing monmap in /tmp/mkcephfs.Pqu0j1MQsc/monmap
/usr/local/bin/monmaptool --create --clobber --add a 10.72.148.209 --print /tmp/mkcephfs.Pqu0j1MQsc/monmap
/usr/local/bin/monmaptool: monmap file /tmp/mkcephfs.Pqu0j1MQsc/monmap
/usr/local/bin/monmaptool: generated fsid fc40c434-06c6-40f6-aa32-e7c5691adb3a
epoch 0
fsid fc40c434-06c6-40f6-aa32-e7c5691adb3a
last_changed 2013-02-15 16:40:19.700856
created 2013-02-15 16:40:19.700856
0: 10.72.148.209:6789/0 mon.a
/usr/local/bin/monmaptool: writing epoch 0 to /tmp/mkcephfs.Pqu0j1MQsc/monmap (1 monitors)
Building generic osdmap from /tmp/mkcephfs.Pqu0j1MQsc/conf
/usr/local/bin/osdmaptool: osdmap file '/tmp/mkcephfs.Pqu0j1MQsc/osdmap'
/usr/local/bin/osdmaptool: writing epoch 1 to /tmp/mkcephfs.Pqu0j1MQsc/osdmap
Generating admin key at /tmp/mkcephfs.Pqu0j1MQsc/keyring.admin
creating /tmp/mkcephfs.Pqu0j1MQsc/keyring.admin
Building initial monitor keyring
cat: /tmp/mkcephfs.Pqu0j1MQsc/key.*: No such file or directory

Where am I wrong and what should I do to deploy this cluster and do some basic operations on it? Should I paste my ceph.conf file here?

Thanks


On Wed, Feb 13, 2013 at 9:11 PM, Noah Watkins <jayhawk@xxxxxxxxxxx> wrote:
Hi Varun,

The main Ceph/Hadoop development is being done in a new location now.

  http://github.com/ceph/hadopo-common cephfs/branch-1.0

The Java bindings for Ceph are contained in the main Ceph tree. They are also included in the Debian PPA releases, but in the short term the dependency for Hadoop is on the master branch.

So, in summary:

1. Grab the Hadoop branch above
2. Get the Ceph Java bindings
    - Build ceph with --eanble-cephfs-java
    - Or use the master PPA
3. Build Hadoop

This is a listing of configuration options that the Hadoop bindings recognize

  http://ceph.com/docs/master/cephfs/hadoop/

And there is a lot more documentation in the pipe. I'll ping the mailing list once it gets posted.

Let us know if you have any issues getting started.

Thanks,
Noah


On Feb 13, 2013, at 1:35 AM, Varun Chandramouli <varun.c37@xxxxxxxxx> wrote:

> Hi all,
>
> I am new to ceph (installed latest version of ceph via the 5-minute installation guide). I am facing a problem while trying to run hadoop on ceph instead of hdfs. According to the steps given here: http://static.usenix.org/publications/login/2010-08/openpdfs/maltzahn.pdf , I installed the hadoop patch given here: https://issues.apache.org/jira/browse/HADOOP-6253 on hadoop-1.0.3.
>
> However, I am not able to find libhadoopceph on my system, and so, am not able to give a path for fs.ceph.libDir. Can anyone tell me where I am wrong?
>
> Thank You
> Varun
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




--
Varun Chandramouli
Birla Institute of Technology & Science
http://in.linkedin.com/in/chandramoulivarun
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux