On Mon 01-08-16 12:52:40, Ralf-Peter Rohbeck wrote: > On 01.08.2016 12:43, Michal Hocko wrote: > > On Mon 01-08-16 12:35:51, Ralf-Peter Rohbeck wrote: > > > On 01.08.2016 12:26, Michal Hocko wrote: > > [...] > > > > the amount of dirty pages is much smaller as well as the anonymous > > > > memory. The biggest portion seems to be in the page cache. The memory > > > The page cache will always be full if I'm writing at full steam to multiple > > > drives, no? > > Yes, the memory full of page cache is not unusual. The large portion of > > that memory being dirty/writeback can be a problem. That is why we have > > a dirty memory throttling which slows down (throttles) writers to keep > > the amount reasonable. What is your dirty throttling setup? > > $ grep . /proc/sys/vm/dirty* > > > > and what is your storage setup? > > root@fs:~# grep . /proc/sys/vm/dirty* > /proc/sys/vm/dirty_background_bytes:0 > /proc/sys/vm/dirty_background_ratio:10 > /proc/sys/vm/dirty_bytes:0 > /proc/sys/vm/dirty_expire_centisecs:3000 > /proc/sys/vm/dirty_ratio:20 With your 8G of RAM this can be quite a lot of dirty data at once. Is your storage able to write that back in a reasonable time? I mean this shouldn't cause the OOM killer but it can lead to some unexpected stalls especially when there are a lot of writers AFAIU. dirty_bytes knob should help to define a better cap. > /proc/sys/vm/dirtytime_expire_seconds:43200 > /proc/sys/vm/dirty_writeback_centisecs:500 > > > Storage setup: > > root@fs:~# lsscsi > [0:2:0:0] disk LSI MR9271-8iCC 3.29 /dev/sda > [0:2:1:0] disk LSI MR9271-8iCC 3.29 /dev/sdb > [9:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdf > [10:0:0:0] disk Seagate Backup+ Desk 050B /dev/sdc > [11:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdd > [12:0:0:0] disk Seagate Backup+ Desk 050B /dev/sde > [13:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdg > [14:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdl > [15:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdh > [16:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdi > [17:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdm > [18:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdj > [19:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdk > > sda is a 6x 1TB RAID5 and sdb is a single 480GB SSD, both on a MegaRAID > controller. > > The rest are 4TB USB drives that I'm experimenting with. Which devices did you write when hitting the OOM killer? -- Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@xxxxxxxxx. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>