We tried the extents+mballoc+delalloc patches suggested by Andreas and found that it made a significant improvement in our benchmark - write bandwidth increased from 144 MBps to 214 MBps. We are at about 85% of the bandwidth that one can get writing to an ext2 file which in turn is about 82% of the bandwidth one can get writing to the block device. We are analyzing our traces to determine the cause of these differences. So far, we see that during writes to the ext3 file lun writes periodically wait for 5 reads while in the case of writes to ext2 file lun writes periodically wait for only one read. Workload: Single threaded 512 KB writes to a new file. RedHat 4 U1 2.6.16.8 kernel (2.6.9 based kernel) Block Device 308 MBps 306 MBps Ext2 file 267 255 Ext3 138 144 Ext3 with patches N/A 216 Ext3 with patches, journal on separate LUN 215 Himanshu -----Original Message----- From: Andreas Dilger [mailto:adilger@xxxxxxxxxxxxx] Sent: Wednesday, June 21, 2006 4:54 PM To: Sinha, Himanshu Cc: ext3-users@xxxxxxxxxx Subject: Re: Limited write bandwidth from ext3 On Jun 19, 2006 14:18 -0400, Sinha_Himanshu@xxxxxxx wrote: > We measured the write bandwidth for writes to the block device > corresponding to the lun (e.g. /dev/sdb), a file in an ext2 filesystem > and to a file in an ext3 file system. > Write b/w for 512 KB writes > Block device 312 MBps > Ext2 file 247 MBps > Ext3 file 130 MBps > > We are looking for ways to improve the ext3 file write bandwidth. Have a look at the extents+mballoc+delalloc patches from Alex Tomas: ftp://ftp.lustre.org/pub/people/alex/2.6.16.8/ Mount the filesystem with "-o extents,mballoc,delalloc" to enable this. They noticably improve IO performance while also reducing the CPU load for ext3. The extent patches are approved by all of the ext3 developers and will be supported upstream fairly soon (in the kernel and e2fsprogs), and mballoc+delalloc will follow on afterward. NOTE: the extents on-disk format is incompatible with older kernels, so at this stage consider it "for benchmarking only". Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. _______________________________________________ Ext3-users@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/ext3-users
