Hi Sage, After pulling the changes, the problem has been resolved. Still, we are getting problem in ceph-mon command. It fails to create ceph-mon demon. # ceph-mon -c ceph.conf -i 0 ** WARNING: Ceph is still under development. Any feedback can be directed ** ** at ceph-devel@xxxxxxxxxxxxxxx or http://ceph.newdream.net/. ** starting mon.0 rank 0 at 192.168.56.101:6789/0 mon_data /root/workera/CEPH/mondata/data/mon.0 fsid 531eb76e-9181-41c4-b162-66b05cfda952 So we getting timeout when we try to mount ceph client (both fuse and kernel client). Regards, ~Pankaj Singh On Thu, Dec 8, 2011 at 10:07 PM, Sage Weil <sage@xxxxxxxxxxxx> wrote: > Hi Pankaj, > > I pushed a fix for this to the stable branch, commit > ca002a3389877f5e150659649e27e7ae59d7d402. > > Let us know if that doesn't fix it for you! > sage > > > > On Thu, 8 Dec 2011, Sage Weil wrote: >> > On Wed, Dec 7, 2011 at 12:40 PM, pankaj singh <psingh.ait@xxxxxxxxx> wrote: >> > > Hi, >> > > >> > > I and my friend Rohan tries to setup ceph on our machine. We are new >> > > to ceph. We have got some issues during ceph setup. >> > > We have followed following steps. >> > > >> > > 1. downloaded the code from Git repo. >> > > 2. compile the code. ./configure ; make >> > > 3. make install >> > > 4. created ceph.conf file. >> > > /------------------ >> > > [global] >> > > ; log dir = out >> > > ; logger dir = log >> > > ; chdir = "" >> > > ; pid file = out/$type$id.pid >> > > ; auth supported = cephx >> > > ; keyring = /etc/ceph/$name.keyring >> > > [mon] >> > > mon data = /root/workera/CEPH/mondata/data/mon.$id >> > > >> > > [osd] >> > > osd data = /root/workera/CEPH/mondata/data/osd.$id >> > > osd journal = /root/workera/CEPH/mondata/data/osd.$id.journal >> > > osd journal size = 1000 >> > > >> > > [mon.0] >> > > host = alpha >> > > mon addr = 192.168.56.102:6789 >> > > >> > > [osd.0] >> > > host = alpha >> > > btrfs devs = /root/workera/CEPH/mondata/data/osd0 >> > > >> > > [mds] >> > > debug mds = 1 >> > > [mds.alpha] >> > > host = alpha >> > > /---------------------- >> > > 5. Created the necessary directories. mkdir -p >> > > /root/workera/CEPH/mondata/data >> > > 6. created a file disk "osd0" with btrfs filesystem. >> > > 7. Run mkcephfs --allhosts -c ceph.conf >> > > This is the error what i got: >> > > >> > > /--------------------------- >> > > temp dir is /tmp/mkcephfs.rvWtFT0Ddi >> > > preparing monmap in /tmp/mkcephfs.rvWtFT0Ddi/monmap >> > > /usr/local/bin/monmaptool --create --clobber --add 0 >> > > 192.168.56.102:6789 --print /tmp/mkcephfs.rvWtFT0Ddi/monmap >> > > /usr/local/bin/monmaptool: monmap file /tmp/mkcephfs.rvWtFT0Ddi/monmap >> > > /usr/local/bin/monmaptool: generated fsid >> > > fa4b96f9-9c95-424c-b9b0-d55b83ce1d59 >> > > epoch 0 >> > > fsid fa4b96f9-9c95-424c-b9b0-d55b83ce1d59 >> > > last_changed 2011-12-04 05:35:34.443071 >> > > created 2011-12-04 05:35:34.443071 >> > > 0: 192.168.56.102:6789/0 mon.0 >> > > /usr/local/bin/monmaptool: writing epoch 0 to >> > > /tmp/mkcephfs.rvWtFT0Ddi/monmap (1 monitors) >> > > === osd.0 === >> > > 2011-12-04 05:35:34.785176 b70bc710 created object store >> > > /root/workera/CEPH/mondata/data/osd.0 journal >> > > /root/workera/CEPH/mondata/data/osd.0.journal for osd.0 fsid >> > > fa4b96f9-9c95-424c-b9b0-d55b83ce1d59 >> > > creating private key for osd.0 keyring >> > > /tmp/mkcephfs.rvWtFT0Ddi/keyring.osd.0 >> > > creating /tmp/mkcephfs.rvWtFT0Ddi/keyring.osd.0 >> > > === mds.alpha === >> > > creating private key for mds.alpha keyring >> > > /tmp/mkcephfs.rvWtFT0Ddi/keyring.mds.alpha >> > > creating /tmp/mkcephfs.rvWtFT0Ddi/keyring.mds.alpha >> > > Building generic osdmap from /tmp/mkcephfs.rvWtFT0Ddi/conf >> > > /usr/local/bin/osdmaptool: osdmap file '/tmp/mkcephfs.rvWtFT0Ddi/osdmap' >> >> If you run just this command >> >> osdmaptool --create-from-conf /tmp/osdmap -c ceph.conf >> >> do you get the same error? >> >> sage >> >> >> >> > > *** glibc detected *** /usr/local/bin/osdmaptool: free(): invalid >> > > pointer: 0x081e84a8 *** >> > > ======= Backtrace: ========= >> > > /lib/libc.so.6(+0x6ed8b)[0xb71e6d8b] >> > > /lib/libc.so.6(cfree+0x9b)[0xb71eb28b] >> > > /usr/local/bin/osdmaptool(crush_calc_straw+0x26c)[0x8135cdc] >> > > /usr/local/bin/osdmaptool(crush_make_straw_bucket+0xd7)[0x8135e07] >> > > /usr/local/bin/osdmaptool(crush_make_bucket+0xb0)[0x8135ed0] >> > > /usr/local/bin/osdmaptool(_ZN12CrushWrapper10add_bucketEiiiiiPiS0_+0x46)[0x811430c] >> > > /usr/local/bin/osdmaptool(_ZN6OSDMap32build_simple_crush_map_from_confEP11CephContextR12CrushWrapperRSt3mapIiPKcSt4lessIiESaISt4pairIKiS6_EEE+0x102)[0x8111f34] >> > > /usr/local/bin/osdmaptool(_ZN6OSDMap22build_simple_from_confEP11CephContextjR6uuid_diii+0x588)[0x8111d88] >> > > /usr/local/bin/osdmaptool(main+0xc80)[0x80f4d8f] >> > > /lib/libc.so.6(__libc_start_main+0xf3)[0xb7191003] >> > > /usr/local/bin/osdmaptool[0x80f3ed1] >> > > ======= Memory map: ======== >> > > 08048000-081cf000 r-xp 00000000 08:02 802104 /usr/local/bin/osdmaptool >> > > 081cf000-081d1000 r--p 00187000 08:02 802104 /usr/local/bin/osdmaptool >> > > 081d1000-081d3000 rw-p 00189000 08:02 802104 /usr/local/bin/osdmaptool >> > > 081d3000-081ff000 rw-p 00000000 00:00 0 [heap] >> > > b68b2000-b68b3000 ---p 00000000 00:00 0 >> > > b68b3000-b7178000 rw-p 00000000 00:00 0 >> > > b7178000-b72de000 r-xp 00000000 08:02 525765 /lib/libc-2.14.1.so >> > > b72de000-b72e0000 r--p 00165000 08:02 525765 /lib/libc-2.14.1.so >> > > b72e0000-b72e1000 rw-p 00167000 08:02 525765 /lib/libc-2.14.1.so >> > > b72e1000-b72e4000 rw-p 00000000 00:00 0 >> > > b72e4000-b7300000 r-xp 00000000 08:02 525771 /lib/libgcc_s.so.1 >> > > b7300000-b7301000 r--p 0001b000 08:02 525771 /lib/libgcc_s.so.1 >> > > b7301000-b7302000 rw-p 0001c000 08:02 525771 /lib/libgcc_s.so.1 >> > > b7302000-b732b000 r-xp 00000000 08:02 525493 /lib/libm-2.14.1.so >> > > b732b000-b732c000 r--p 00028000 08:02 525493 /lib/libm-2.14.1.so >> > > b732c000-b732d000 rw-p 00029000 08:02 525493 /lib/libm-2.14.1.so >> > > b732d000-b740f000 r-xp 00000000 08:02 270052 >> > > /usr/lib/libstdc++.so.6.0.16 >> > > b740f000-b7413000 r--p 000e2000 08:02 270052 >> > > /usr/lib/libstdc++.so.6.0.16 >> > > b7413000-b7414000 rw-p 000e6000 08:02 270052 >> > > /usr/lib/libstdc++.so.6.0.16 >> > > b7414000-b741c000 rw-p 00000000 00:00 0 >> > > b741c000-b7811000 r-xp 00000000 08:02 293697 >> > > /usr/lib/libcryptopp.so.9.0.0 >> > > b7811000-b7847000 r--p 003f5000 08:02 293697 >> > > /usr/lib/libcryptopp.so.9.0.0 >> > > b7847000-b7849000 rw-p 0042b000 08:02 293697 >> > > /usr/lib/libcryptopp.so.9.0.0 >> > > b7849000-b784d000 rw-p 00000000 00:00 0 >> > > b784d000-b7854000 r-xp 00000000 08:02 525478 /lib/librt-2.14.1.so >> > > b7854000-b7855000 r--p 00006000 08:02 525478 /lib/librt-2.14.1.so >> > > b7855000-b7856000 rw-p 00007000 08:02 525478 /lib/librt-2.14.1.so >> > > b7856000-b785a000 r-xp 00000000 08:02 525907 /lib/libuuid.so.1.3.0 >> > > b785a000-b785b000 r--p 00003000 08:02 525907 /lib/libuuid.so.1.3.0 >> > > b785b000-b785c000 rw-p 00004000 08:02 525907 /lib/libuuid.so.1.3.0 >> > > b785c000-b7873000 r-xp 00000000 08:02 525926 /lib/libpthread-2.14.1.so >> > > b7873000-b7874000 r--p 00016000 08:02 525926 /lib/libpthread-2.14.1.so >> > > b7874000-b7875000 rw-p 00017000 08:02 525926 /lib/libpthread-2.14.1.so >> > > b7875000-b7877000 rw-p 00000000 00:00 0 >> > > b788c000-b788f000 rw-p 00000000 00:00 0 >> > > b788f000-b78ae000 r-xp 00000000 08:02 525496 /lib/ld-2.14.1.so >> > > b78ae000-b78af000 r--p 0001f000 08:02 525496 /lib/ld-2.14.1.so >> > > b78af000-b78b0000 rw-p 00020000 08:02 525496 /lib/ld-2.14.1.so >> > > bfdfc000-bfe1d000 rw-p 00000000 00:00 0 [stack] >> > > ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] >> > > *** Caught signal (Aborted) ** >> > > in thread b7176710 >> > > ceph version 0.39-28-g4a0b00a >> > > (commit:4a0b00a0f29a87965925e0b44c997bece96b9936) >> > > 1: (ceph::BackTrace::BackTrace(int)+0x36) [0x8107cc6] >> > > 2: /usr/local/bin/osdmaptool() [0x81079bd] >> > > 3: [0xffffe400] >> > > 4: (gsignal()+0x45) [0xb71a58c5] >> > > 5: (abort()+0x175) [0xb71a71d5] >> > > 6: (()+0x685ca) [0xb71e05ca] >> > > 7: (()+0x6ed8b) [0xb71e6d8b] >> > > 8: (cfree()+0x9b) [0xb71eb28b] >> > > 9: (crush_calc_straw()+0x26c) [0x8135cdc] >> > > 10: (crush_make_straw_bucket()+0xd7) [0x8135e07] >> > > 11: (crush_make_bucket()+0xb0) [0x8135ed0] >> > > 12: (CrushWrapper::add_bucket(int, int, int, int, int, int*, >> > > int*)+0x46) [0x811430c] >> > > 13: (OSDMap::build_simple_crush_map_from_conf(CephContext*, >> > > CrushWrapper&, std::map<int, char const*, std::less<int>, >> > > std::allocator<std::pair<int const, char const*> > >&)+0x102) >> > > [0x8111f34] >> > > 14: (OSDMap::build_simple_from_conf(CephContext*, unsigned int, >> > > uuid_d&, int, int, int)+0x588) [0x8111d88] >> > > 15: (main()+0xc80) [0x80f4d8f] >> > > 16: (__libc_start_main()+0xf3) [0xb7191003] >> > > 17: /usr/local/bin/osdmaptool() [0x80f3ed1] >> > > /usr/local/sbin/mkcephfs: line 405: 26267 Aborted >> > > (core dumped) $BINDIR/osdmaptool --create-from-conf $dir/osdmap -c >> > > $conf >> > > /------------------------------------- >> > > >> > > This error probably occurs because of following reason >> > > In function crush_calc_straw while allocating "reverse" malloc getting >> > > zero as the argument because "bucket->h.size" happens to be zero. So >> > > while trying to free this variable causing trouble. >> > > >> > > So, We tried a work around by putting the check for zero size in >> > > the start of function, which would return from the function if it is zero. >> > > >> > > After this mkcephfs ran successfully but still >> > > >> > > getting issues while trying to run ceph-mon. >> > > Now ceph-mon command is unable to create ceph-mon daemon and hence when we >> > > >> > > try to mount with the fuse interface or using the kernel client it >> > > does not gets mounted stating error connection failed. >> > > >> > > Here is my machine info: >> > > Platform = OpenSuse 12.1 >> > > kernel version = linux-3.1.0-1.2 >> > > >> > > -- >> > > Regards, >> > > Pankaj SIngh >> > > Phone No: 8826266889 >> > >> > >> > >> > -- >> > Regards, >> > Pankaj SIngh >> > Phone No: 8826266889 >> > -- >> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> > the body of a message to majordomo@xxxxxxxxxxxxxxx >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> > >> > -- Regards, Pankaj SIngh Phone No: 8826266889 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
