HI Huang, Thanks for replying quick. Firstly we are getting trouble while running mfcephfs command. We are getting following error : " free(): invalid pointer: 0x081e84a8 ". We have found that crush_calc_straw while allocating "reverse" malloc getting zero as the argument because "bucket->h.size" happens to be zero. So while trying to free this variable causing error. We tried a work around by putting the check for zero size in the start of function, which would return from the function if it is zero. So Here is the output of "ceph -w -d" when i tried to run ceph-mon.command //-------------------------------------- # ceph -w -d 2011-12-04 19:33:40.117691 b3f1ab70 -- :/13129 >> 192.168.56.101:6789/0 pipe(0x84995b0 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:33:43.120378 b3e1ab70 -- :/13129 >> 192.168.56.101:6789/0 pipe(0x849b008 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:33:46.118933 b3f1ab70 -- :/13129 >> 192.168.56.101:6789/0 pipe(0x84995b0 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:33:49.120646 b3e1ab70 -- :/13129 >> 192.168.56.101:6789/0 pipe(0x8499fa8 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:33:52.121898 b3f1ab70 -- :/13129 >> 192.168.56.101:6789/0 pipe(0x84995b0 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:33:55.121663 b3e1ab70 -- 192.168.56.101:0/13129 >> 192.168.56.101:6789/0 pipe(0x84995b0 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:33:58.123115 b3f1ab70 -- 192.168.56.101:0/13129 >> 192.168.56.101:6789/0 pipe(0x8499930 sd=3 pgs=0 cs=0 l=0).fault first fault 2011-12-04 19:34:01.123607 b3e1ab70 -- 192.168.56.101:0/13129 >> 192.168.56.101:6789/0 pipe(0x84995b0 sd=3 pgs=0 cs=0 l=0).fault first fault //------------------------------------------ here is the output of ceph-mon //------------------------------------- # ceph-mon -c ceph.conf -i 0 ** WARNING: Ceph is still under development. Any feedback can be directed ** ** at ceph-devel@xxxxxxxxxxxxxxx or http://ceph.newdream.net/. ** starting mon.0 rank 0 at 192.168.56.101:6789/0 mon_data /root/workera/CEPH/mondata/data/mon.0 fsid 8fc7782d-0738-42f6-b552-3c0b3d826373 //------------------------------------ Regards, ~Pankaj Singh On Thu, Dec 8, 2011 at 3:21 PM, huang jun <hjwsm1989@xxxxxxxxx> wrote: > 2011/12/8 pankaj singh <psingh.ait@xxxxxxxxx>: >> Hi Guys, >> >> Any help would be appreciable... > hi,sorry for yesterday's answer. > I want to try my best of this problem. > so, can you show me the "ceph -w", the former is messy. > >> Regards, >> ~Pankaj Singh >> >> On Wed, Dec 7, 2011 at 12:40 PM, pankaj singh <psingh.ait@xxxxxxxxx> wrote: >>> Hi, >>> >>> I and my friend Rohan tries to setup ceph on our machine. We are new >>> to ceph. We have got some issues during ceph setup. >>> We have followed following steps. >>> >>> 1. downloaded the code from Git repo. >>> 2. compile the code. ./configure ; make >>> 3. make install >>> 4. created ceph.conf file. >>> /------------------ >>> [global] >>> ; log dir = out >>> ; logger dir = log >>> ; chdir = "" >>> ; pid file = out/$type$id.pid >>> ; auth supported = cephx >>> ; keyring = /etc/ceph/$name.keyring >>> [mon] >>> mon data = /root/workera/CEPH/mondata/data/mon.$id >>> >>> [osd] >>> osd data = /root/workera/CEPH/mondata/data/osd.$id >>> osd journal = /root/workera/CEPH/mondata/data/osd.$id.journal >>> osd journal size = 1000 >>> >>> [mon.0] >>> host = alpha >>> mon addr = 192.168.56.102:6789 >>> >>> [osd.0] >>> host = alpha >>> btrfs devs = /root/workera/CEPH/mondata/data/osd0 >>> >>> [mds] >>> debug mds = 1 >>> [mds.alpha] >>> host = alpha >>> /---------------------- >>> 5. Created the necessary directories. mkdir -p >>> /root/workera/CEPH/mondata/data >>> 6. created a file disk "osd0" with btrfs filesystem. >>> 7. Run mkcephfs --allhosts -c ceph.conf >>> This is the error what i got: >>> >>> /--------------------------- >>> temp dir is /tmp/mkcephfs.rvWtFT0Ddi >>> preparing monmap in /tmp/mkcephfs.rvWtFT0Ddi/monmap >>> /usr/local/bin/monmaptool --create --clobber --add 0 >>> 192.168.56.102:6789 --print /tmp/mkcephfs.rvWtFT0Ddi/monmap >>> /usr/local/bin/monmaptool: monmap file /tmp/mkcephfs.rvWtFT0Ddi/monmap >>> /usr/local/bin/monmaptool: generated fsid >>> fa4b96f9-9c95-424c-b9b0-d55b83ce1d59 >>> epoch 0 >>> fsid fa4b96f9-9c95-424c-b9b0-d55b83ce1d59 >>> last_changed 2011-12-04 05:35:34.443071 >>> created 2011-12-04 05:35:34.443071 >>> 0: 192.168.56.102:6789/0 mon.0 >>> /usr/local/bin/monmaptool: writing epoch 0 to >>> /tmp/mkcephfs.rvWtFT0Ddi/monmap (1 monitors) >>> === osd.0 === >>> 2011-12-04 05:35:34.785176 b70bc710 created object store >>> /root/workera/CEPH/mondata/data/osd.0 journal >>> /root/workera/CEPH/mondata/data/osd.0.journal for osd.0 fsid >>> fa4b96f9-9c95-424c-b9b0-d55b83ce1d59 >>> creating private key for osd.0 keyring >>> /tmp/mkcephfs.rvWtFT0Ddi/keyring.osd.0 >>> creating /tmp/mkcephfs.rvWtFT0Ddi/keyring.osd.0 >>> === mds.alpha === >>> creating private key for mds.alpha keyring >>> /tmp/mkcephfs.rvWtFT0Ddi/keyring.mds.alpha >>> creating /tmp/mkcephfs.rvWtFT0Ddi/keyring.mds.alpha >>> Building generic osdmap from /tmp/mkcephfs.rvWtFT0Ddi/conf >>> /usr/local/bin/osdmaptool: osdmap file '/tmp/mkcephfs.rvWtFT0Ddi/osdmap' >>> *** glibc detected *** /usr/local/bin/osdmaptool: free(): invalid >>> pointer: 0x081e84a8 *** >>> ======= Backtrace: ========= >>> /lib/libc.so.6(+0x6ed8b)[0xb71e6d8b] >>> /lib/libc.so.6(cfree+0x9b)[0xb71eb28b] >>> /usr/local/bin/osdmaptool(crush_calc_straw+0x26c)[0x8135cdc] >>> /usr/local/bin/osdmaptool(crush_make_straw_bucket+0xd7)[0x8135e07] >>> /usr/local/bin/osdmaptool(crush_make_bucket+0xb0)[0x8135ed0] >>> /usr/local/bin/osdmaptool(_ZN12CrushWrapper10add_bucketEiiiiiPiS0_+0x46)[0x811430c] >>> /usr/local/bin/osdmaptool(_ZN6OSDMap32build_simple_crush_map_from_confEP11CephContextR12CrushWrapperRSt3mapIiPKcSt4lessIiESaISt4pairIKiS6_EEE+0x102)[0x8111f34] >>> /usr/local/bin/osdmaptool(_ZN6OSDMap22build_simple_from_confEP11CephContextjR6uuid_diii+0x588)[0x8111d88] >>> /usr/local/bin/osdmaptool(main+0xc80)[0x80f4d8f] >>> /lib/libc.so.6(__libc_start_main+0xf3)[0xb7191003] >>> /usr/local/bin/osdmaptool[0x80f3ed1] >>> ======= Memory map: ======== >>> 08048000-081cf000 r-xp 00000000 08:02 802104 /usr/local/bin/osdmaptool >>> 081cf000-081d1000 r--p 00187000 08:02 802104 /usr/local/bin/osdmaptool >>> 081d1000-081d3000 rw-p 00189000 08:02 802104 /usr/local/bin/osdmaptool >>> 081d3000-081ff000 rw-p 00000000 00:00 0 [heap] >>> b68b2000-b68b3000 ---p 00000000 00:00 0 >>> b68b3000-b7178000 rw-p 00000000 00:00 0 >>> b7178000-b72de000 r-xp 00000000 08:02 525765 /lib/libc-2.14.1.so >>> b72de000-b72e0000 r--p 00165000 08:02 525765 /lib/libc-2.14.1.so >>> b72e0000-b72e1000 rw-p 00167000 08:02 525765 /lib/libc-2.14.1.so >>> b72e1000-b72e4000 rw-p 00000000 00:00 0 >>> b72e4000-b7300000 r-xp 00000000 08:02 525771 /lib/libgcc_s.so.1 >>> b7300000-b7301000 r--p 0001b000 08:02 525771 /lib/libgcc_s.so.1 >>> b7301000-b7302000 rw-p 0001c000 08:02 525771 /lib/libgcc_s.so.1 >>> b7302000-b732b000 r-xp 00000000 08:02 525493 /lib/libm-2.14.1.so >>> b732b000-b732c000 r--p 00028000 08:02 525493 /lib/libm-2.14.1.so >>> b732c000-b732d000 rw-p 00029000 08:02 525493 /lib/libm-2.14.1.so >>> b732d000-b740f000 r-xp 00000000 08:02 270052 >>> /usr/lib/libstdc++.so.6.0.16 >>> b740f000-b7413000 r--p 000e2000 08:02 270052 >>> /usr/lib/libstdc++.so.6.0.16 >>> b7413000-b7414000 rw-p 000e6000 08:02 270052 >>> /usr/lib/libstdc++.so.6.0.16 >>> b7414000-b741c000 rw-p 00000000 00:00 0 >>> b741c000-b7811000 r-xp 00000000 08:02 293697 >>> /usr/lib/libcryptopp.so.9.0.0 >>> b7811000-b7847000 r--p 003f5000 08:02 293697 >>> /usr/lib/libcryptopp.so.9.0.0 >>> b7847000-b7849000 rw-p 0042b000 08:02 293697 >>> /usr/lib/libcryptopp.so.9.0.0 >>> b7849000-b784d000 rw-p 00000000 00:00 0 >>> b784d000-b7854000 r-xp 00000000 08:02 525478 /lib/librt-2.14.1.so >>> b7854000-b7855000 r--p 00006000 08:02 525478 /lib/librt-2.14.1.so >>> b7855000-b7856000 rw-p 00007000 08:02 525478 /lib/librt-2.14.1.so >>> b7856000-b785a000 r-xp 00000000 08:02 525907 /lib/libuuid.so.1.3.0 >>> b785a000-b785b000 r--p 00003000 08:02 525907 /lib/libuuid.so.1.3.0 >>> b785b000-b785c000 rw-p 00004000 08:02 525907 /lib/libuuid.so.1.3.0 >>> b785c000-b7873000 r-xp 00000000 08:02 525926 /lib/libpthread-2.14.1.so >>> b7873000-b7874000 r--p 00016000 08:02 525926 /lib/libpthread-2.14.1.so >>> b7874000-b7875000 rw-p 00017000 08:02 525926 /lib/libpthread-2.14.1.so >>> b7875000-b7877000 rw-p 00000000 00:00 0 >>> b788c000-b788f000 rw-p 00000000 00:00 0 >>> b788f000-b78ae000 r-xp 00000000 08:02 525496 /lib/ld-2.14.1.so >>> b78ae000-b78af000 r--p 0001f000 08:02 525496 /lib/ld-2.14.1.so >>> b78af000-b78b0000 rw-p 00020000 08:02 525496 /lib/ld-2.14.1.so >>> bfdfc000-bfe1d000 rw-p 00000000 00:00 0 [stack] >>> ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] >>> *** Caught signal (Aborted) ** >>> in thread b7176710 >>> ceph version 0.39-28-g4a0b00a >>> (commit:4a0b00a0f29a87965925e0b44c997bece96b9936) >>> 1: (ceph::BackTrace::BackTrace(int)+0x36) [0x8107cc6] >>> 2: /usr/local/bin/osdmaptool() [0x81079bd] >>> 3: [0xffffe400] >>> 4: (gsignal()+0x45) [0xb71a58c5] >>> 5: (abort()+0x175) [0xb71a71d5] >>> 6: (()+0x685ca) [0xb71e05ca] >>> 7: (()+0x6ed8b) [0xb71e6d8b] >>> 8: (cfree()+0x9b) [0xb71eb28b] >>> 9: (crush_calc_straw()+0x26c) [0x8135cdc] >>> 10: (crush_make_straw_bucket()+0xd7) [0x8135e07] >>> 11: (crush_make_bucket()+0xb0) [0x8135ed0] >>> 12: (CrushWrapper::add_bucket(int, int, int, int, int, int*, >>> int*)+0x46) [0x811430c] >>> 13: (OSDMap::build_simple_crush_map_from_conf(CephContext*, >>> CrushWrapper&, std::map<int, char const*, std::less<int>, >>> std::allocator<std::pair<int const, char const*> > >&)+0x102) >>> [0x8111f34] >>> 14: (OSDMap::build_simple_from_conf(CephContext*, unsigned int, >>> uuid_d&, int, int, int)+0x588) [0x8111d88] >>> 15: (main()+0xc80) [0x80f4d8f] >>> 16: (__libc_start_main()+0xf3) [0xb7191003] >>> 17: /usr/local/bin/osdmaptool() [0x80f3ed1] >>> /usr/local/sbin/mkcephfs: line 405: 26267 Aborted >>> (core dumped) $BINDIR/osdmaptool --create-from-conf $dir/osdmap -c >>> $conf >>> /------------------------------------- >>> >>> This error probably occurs because of following reason >>> In function crush_calc_straw while allocating "reverse" malloc getting >>> zero as the argument because "bucket->h.size" happens to be zero. So >>> while trying to free this variable causing trouble. >>> >>> So, We tried a work around by putting the check for zero size in >>> the start of function, which would return from the function if it is zero. >>> >>> After this mkcephfs ran successfully but still >>> >>> getting issues while trying to run ceph-mon. >>> Now ceph-mon command is unable to create ceph-mon daemon and hence when we >>> >>> try to mount with the fuse interface or using the kernel client it >>> does not gets mounted stating error connection failed. >>> >>> Here is my machine info: >>> Platform = OpenSuse 12.1 >>> kernel version = linux-3.1.0-1.2 >>> >>> -- >>> Regards, >>> Pankaj SIngh >>> Phone No: 8826266889 >> >> >> >> -- >> Regards, >> Pankaj SIngh >> Phone No: 8826266889 >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@xxxxxxxxxxxxxxx >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > -- > huangjun > ChuYunHanZhi WuHan Network Storage System Ltd > WuHan GuangGu Road -- Regards, Pankaj SIngh Phone No: 8826266889 -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
