Re: Hadoop on Ceph error

[Bishoy Mikhael <b.s.mikhael@xxxxxxxxx>]

here is my core-site.xml file

<?xml version="1.0" encoding="UTF-8"?>

<?xml-stylesheet type="text/xsl" href="">

<configuration>

   <property>

      <name>fs.default.name</name>

      <value>ceph://host01:6789/</value>

   </property>

   <property>

      <name>fs.defaultFS</name>

      <value>ceph://host01:6789</value>

   </property>

   <property>

      <name>io.file.buffer.size</name>

      <value>131072</value>

   </property>

   <property>

      <name>hadoop.tmp.dir</name>

      <value>/mnt/hadoop/hadoop_tmp</value>

   </property>

   <property>

      <name>ceph.conf.file</name>

      <value>/etc/ceph/ceph.conf</value>

   </property>

   <property>

      <name>fs.AbstractFileSystem.ceph.impl</name>

      <value>org.apache.hadoop.fs.ceph.CephFs</value>

   </property>

   <property>

      <name>fs.ceph.impl</name>

      <value>org.apache.hadoop.fs.ceph.CephFileSystem</value>

   </property>

   <property>

      <name>ceph.auth.keyring</name>

      <value>/etc/ceph/ceph.client.admin.keyring</value>

   </property>

   <property>

      <name>ceph.object.size</name>

      <value>67108864</value>

   </property>

   <property>

      <name>ceph.data.pools</name>

      <value>hadoop_data</value>

   </property>

   <property>

      <name>ceph.localize.reads</name>

      <value>true</value>

   </property>

</configuration>

here is the permissions for the client keyring

# ls -lh /etc/ceph/ceph.client.admin.keyring

-rw-------. 1 root root 137 Jan 12 00:14 /etc/ceph/ceph.client.admin.keyring

here is the permission for ceph.conf

# ls -lh /etc/ceph/ceph.conf 

-rw-r--r--. 1 root root 3.0K Jan 12 00:14 /etc/ceph/ceph.conf

how to set debug option in ceph.conf.options with Hadoop xml config file?

On Thu, Jan 18, 2018 at 7:54 PM, Jean-Charles Lopez <jelopez@xxxxxxxxxx> wrote:

Hi,

What’s your Hadoop xml config file like?

Have you checked the permissions of the ceph.conf and keyring file?

In case all good may be consider setting debug option in ceph.conf.options with Hadoop xml config file

JC

On Jan 18, 2018, at 16:55, Bishoy Mikhael <b.s.mikhael@xxxxxxxxx> wrote:

Hi All,

I've a tiny Ceph 12.2.2 cluster setup with three nodes, 17 OSDs, 3 MON,MDS,MGR (spanned across the three nodes).

Hadoop 2.7.3 is configured on only one of the three nodes as follows:

- Hadoop binaries was extracted to /opt/hadoop/bin/

- Hadoop config files where at /opt/hadoop/etc/hadoop/

- Hadoop-cephfs.jar was downloaded from http://download.ceph.com/tarballs/ to /opt/hadoop/lib/ which the last update to it was on 12-Mar-2013

- The following symbolic links have been done:

# ln -s /usr/lib64/libcephfs_jni.so.1.0.0 /usr/lib64/libcephfs_jni.so

# cp /usr/lib64/libcephfs_jni.so.1.0.0 /opt/hadoop/lib/native/

# ln -s /opt/hadoop/lib/native/libcephfs_jni.so.1.0.0 /opt/hadoop/lib/native/libcephfs_jni.so.1

# ln -s /opt/hadoop/lib/native/libcephfs_jni.so.1.0.0 /opt/hadoop/lib/native/libcephfs_jni.so

# ln -s /usr/share/java/libcephfs.jar /opt/hadoop/lib/

The following modification to Hadoop-config.sh has been done:

/opt/hadoop/libexec/hadoop-config.sh

# CLASSPATH initially contains $HADOOP_CONF_DIR

CLASSPATH="${HADOOP_CONF_DIR}:/opt/hadoop/lib/libcephfs.jar:/opt/hadoop/lib/hadoop-cephfs.jar"

So writes and reads to/from Ceph using HDFS CLI works fine, but when I use hadoop Java library I get the following error:

ERROR HdfsTraveller:58 - com.ceph.fs.CephNotMountedException: not mounted

fileSystem.globStatus(path)

FileSystem.globStatus in hdfs api

ceph returns null pointer

Any idea what's going on? is it a configuration problem? is it a Ceph problem? Did anybody see that error before?

Regards,

Bishoy

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com