sudden (big) performance drop in writes

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Our server has suddenly become extremely slow in writes. With

% dd if=/dev/zero of=zero bs=2M count=2000

I get only 1.4MB/s in an almost idle machine. It has a raid6 with 6
disks, running 3.3.7. 3.4.2 doesn't improve matters.

The important point is that it became so slow yesterday. No hardware
has changed and all disks are fine. Reading from the filesystem is
unaffected, at more than 85MB/s. The problem happens both in the root
and home partitions.

During the dd the disk utilization measured by sar is small:

DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
sdb     93.30    254.40    931.20     12.71      0.57      6.10      4.16     38.77
sdc    100.50    244.00    970.40     12.08      0.76      7.59      3.80     38.20
sdd    101.40    261.60    927.20     11.72      0.68      6.68      3.74     37.90
sde     86.80    300.00    780.80     12.45      0.69      7.93      4.29     37.20
sdf     82.90    315.20    810.40     13.58      0.55      6.60      4.39     36.37
sda     96.70    220.00    984.00     12.45      0.47      4.87      3.72     35.94

So it seems that it's not a disk problem. Both filesystems reached 90%
usage in space but less than 10% in inodes shortly before the slowness
appeared. Now they're at 57% and 77% but still crawl. Could this be
related?
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux