On 08/01/16 13:09, Michal Hocko wrote:
On Mon 01-08-16 12:52:40, Ralf-Peter Rohbeck wrote:
On 01.08.2016 12:43, Michal Hocko wrote:
On Mon 01-08-16 12:35:51, Ralf-Peter Rohbeck wrote:
On 01.08.2016 12:26, Michal Hocko wrote:
[...]
the amount of dirty pages is much smaller as well as the anonymous
memory. The biggest portion seems to be in the page cache. The memory
The page cache will always be full if I'm writing at full steam to multiple
drives, no?
Yes, the memory full of page cache is not unusual. The large portion of
that memory being dirty/writeback can be a problem. That is why we have
a dirty memory throttling which slows down (throttles) writers to keep
the amount reasonable. What is your dirty throttling setup?
$ grep . /proc/sys/vm/dirty*
and what is your storage setup?
root@fs:~# grep . /proc/sys/vm/dirty*
/proc/sys/vm/dirty_background_bytes:0
/proc/sys/vm/dirty_background_ratio:10
/proc/sys/vm/dirty_bytes:0
/proc/sys/vm/dirty_expire_centisecs:3000
/proc/sys/vm/dirty_ratio:20
With your 8G of RAM this can be quite a lot of dirty data at once. Is
your storage able to write that back in a reasonable time? I mean this
shouldn't cause the OOM killer but it can lead to some unexpected stalls
especially when there are a lot of writers AFAIU. dirty_bytes knob
should help to define a better cap.
The main filesystems are on the MegaRAID and can do 500-600 MB/s.
Writing to the USB drives only pushes about 90MB/s per drive.
/proc/sys/vm/dirtytime_expire_seconds:43200
/proc/sys/vm/dirty_writeback_centisecs:500
Storage setup:
root@fs:~# lsscsi
[0:2:0:0] disk LSI MR9271-8iCC 3.29 /dev/sda
[0:2:1:0] disk LSI MR9271-8iCC 3.29 /dev/sdb
[9:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdf
[10:0:0:0] disk Seagate Backup+ Desk 050B /dev/sdc
[11:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdd
[12:0:0:0] disk Seagate Backup+ Desk 050B /dev/sde
[13:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdg
[14:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdl
[15:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdh
[16:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdi
[17:0:0:0] disk TOSHIBA External USB 3.0 5438 /dev/sdm
[18:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdj
[19:0:0:0] disk Seagate Expansion Desk 9400 /dev/sdk
sda is a 6x 1TB RAID5 and sdb is a single 480GB SSD, both on a MegaRAID
controller.
The rest are 4TB USB drives that I'm experimenting with.
Which devices did you write when hitting the OOM killer?
sdc, sdd and sde each at max speed, with a little bit of garden variety
IO on sda and sdb.
----------------------------------------------------------------------
The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@xxxxxxxxx. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@xxxxxxxxx";> email@xxxxxxxxx </a>