I am trying to use my Dell 2850 Server as a data pump using sgp_dd to perform large sequential read from my target through sg devices. One instance of sgp_dd uses about 50% of the CPU, so running a second only causes the CPU to become very busy and my read performance suffers as a result. A third and a fourth instance only make matters worse.
I tried using sgm_dd, which is not multithreaded, to run the same test. Though the speed is not quite as high as sgp_dd, it only uses a small amount of CPU resources. That lead me to believe that either sgp_dd has a problem in its multi-threading or maybe the POSIX threads. I'm really not sure at this point what to do next.
Under Windows 2003 running IOMeter, my target can be saturated at my expected bandwidth across 4 FC4 ports. That shows that my target and server are capable of delivering the speed.
I tried both the SMP and non-SMP kernels with the same results.
Here's my configuration using the non-SMP kernel:
My target w/ 4 FC4 host ports
Dell 2850 server Dual processor 1GB memory
Fedora Core 3 Distro updated to kernel 2.6.11-1.27_FC3SMP and 2.6.11-1.27_FC3
swap size 2GB
2 Emulex LP11000 FC4 Dual HBAs, each on independant PCI busses
Emulex driver version: 2.6-8.0.16.6_x2 compiled and installed for the new kernels.
uname -a
Linux 2.6.11-1.27_FC3 #1 Tue May 17 20:27:37 EDT 2005 i686 i686 i386 GNU/Linux
gcc -v
Reading specs from /usr/lib/gcc/i386-redhat-linux/3.4.3/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-java-awt=gtk --host=i386-redhat-linux
Thread model: posix
gcc version 3.4.3 20050227 (Red Hat 3.4.3-22.fc3)
lsscsi -g [0:0:0:0] disk SEAGATE ST336607LC DS09 /dev/sda /dev/sg0 [0:0:6:0] process PE/PV 1x6 SCSI BP 1.0 - /dev/sg1 [14:0:0:0] disk E1.0 /dev/sdb /dev/sg2 [15:0:0:0] disk E1.0 /dev/sdc /dev/sg3 [16:0:0:0] disk E1.0 /dev/sdd /dev/sg4 [17:0:0:0] disk E1.0 /dev/sde /dev/sg5
cat /proc/scsi/sg/allow_dio 0
I tried using a value of 1 for allow_dio, but it had no effect.
Using sg3_utils-1.14
Running sgp_dd like this: sgp_dd if=/dev/sg2 of=/dev/null bs=512 bpt=4096 thr=6 time=1
6 threads, 2M transfers, actually gives 2M commands sizes
[root@racerx proc]# cat cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 4
model name : Intel(R) Xeon(TM) CPU 3.00GHz
stepping : 1
cpu MHz : 2993.167
cache size : 1024 KB
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 5
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips : 5947.39
-
: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
