On Wed, Sep 22, 2010 at 8:14 PM, DongJin Lee <dongjin.lee@xxxxxxxxxxxxxx> wrote: > Question 1. > After the latest git update, i.e., 22nd-Sept (unstable) > The 'make' breaks down. Here's the last line. > >> /bin/bash ../libtool --tag=CXX --mode=link g++ -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -latomic_ops -o cmon cmon.o SimpleMessenger.o libmon.a libcrush.a libcommon.a -lpthread -lm -lcrypto >> libtool: link: g++ -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic -g -O2 -o cmon cmon.o SimpleMessenger.o -latomic_ops libmon.a libcrush.a libcommon.a -lpthread -lm -lcrypto >> g++ -DHAVE_CONFIG_H -I. -Wall -D__CEPH__ -D_FILE_OFFSET_BITS=64 -D_REENTRANT -D_THREAD_SAFE -rdynamic "-fno-builtin-malloc -fno-builtin-calloc -fno-builtin-realloc -fno-builtin-free" -g -O2 -MT cosd-cosd.o -MD -MP -MF .deps/cosd-cosd.Tpo -c -o cosd-cosd.o `test -f 'cosd.cc' || echo './'`cosd.cc >> cosd.cc: In function ‘int main(int, const char**)’: >> cosd.cc:65: error: ‘IsHeapProfilerRunning’ was not declared in this scope >> cosd.cc:310: warning: ignoring return value of ‘int chdir(const char*)’, declared with attribute warn_unused_result >> make[2]: *** [cosd-cosd.o] Error 1 >> make[2]: Leaving directory `/home/aaa/ceph/ceph/src' >> make[1]: *** [all] Error 2 >> make[1]: Leaving directory `/home/aaa/ceph/ceph/src' >> make: *** [all-recursive] Error 1 >> aaa@s3:~/ceph/ceph$ Hmmm -- it looks like the scripts are split on whether you have tcmalloc installed in your system, and I don't see any obvious issues when I go over them. Do you have tcmalloc installed? Did you try a "make clean; make" cycle? > Question 2. > Performance issue: I'm mainly interested in the speed, all > authentications have been disabled. > > Very simple /etc/ceph/ceph.conf > >>> [global] >>> pid file = /var/run/ceph/$name.pid >>> debug ms = 3 >>> sudo = true >>> [osd] >>> [osd0] >>> host = s1 >>> btrfs devs = /dev/sdb >>> osd data = /data/osd0 >>> osd journal = /data/osd0/journal >>> osd journal size = 1000 >>> [osd1] >>> host = s2 >>> btrfs devs = /dev/sdb >>> osd data = /data/osd1 >>> osd journal = /data/osd1/journal >>> osd journal size = 1000 >>> [mds] >>> [mds0] >>> host = s3 >>> [mon] >>> debug ms = 3 >>> [mon0] >>> host = s4 >>> mon data = /data/mon0 >>> mon addr = 192.168.50.5:6789 > > Ceph seems to work okay (mounted at /ceph 5th PC) > I ran fio benchmark (4 tests, random read/write sequential read/write). (4k) > > I find that 'read' seems just okay,, random read = 55 iops, sequential > read = 3000 iops (but I don't think it is that fast?) > But I find that 'write' is very slow, I get both random/sequential > write about 7 to 10 iops. I really suspect something is wrong. :( > Are there any tweaks that I must do to increase the iops? I'm not > really interested in the large block size. > > Also, when I was just copying a large file to it, the speed quickly > deteriorated, it first reported about 40MB/s but later, goes to less > than 8MB/s or worse, sometimes freeze.. > Were there further setups that I need to configure? such as > crush/replication levels? (I haven't touched any of these) Wow, these levels seem oddly slow. What filesystem are those disks running? What tool are you using to test IOPS, and what's your network setup? If you can move the journal onto a separate device it will help. -Greg -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
