On 06/05/2014 04:32 PM, Brio wrote:
So here's where I'm stuck. How can reading a file not leave it in the
Linux cache? I'd expect it to enter the inactive list (which is about
80GB), so I'd expect another 80GB would need to be read before it would
be its turn to be evicted.... which should take a long time if my
maximum read speed is 100MB/s.
So here's the thing. The Linux page reclamation code is *extremely
broken* in everything before 3.11. Take a look at this, then realize
that this is *only one patch* from several that target the memory
manager weightings:
http://linux-kernel.2935.n7.nabble.com/patch-v2-0-3-mm-improve-page-aging-fairness-between-zones-nodes-td696105.html
This is especially true of the 3.2 kernel you're using. It's extremely
aggressive about ageing pages out of memory when there's high memory
pressure from frequent disk reads. Chances of promotion into the active
set is dismal, so you end up with a constant churn between inactive and
disk. Worse, if you kick it hard enough by having too many PostgreSQL
backends using memory, it'll actively purge the active set while still
failing to promote the inactive set.
The dev that linked me to this patch said he tested it against 3.10,
meaning it probably went into 3.11 or 3.12. So I personally wouldn't
trust anything before 3.13. :p
Since you're using Ubuntu 12.04, I strongly suggest upgrading your core
to 12.04.4 and apply the linux-generic-lts-saucy pseudo-package to at
least get onto the 3.11 instead. The 3.2 kernel is pants-on-head
retarded; we've had a lot more luck with 3.8 and above.
Cheers!
--
Shaun Thomas
OptionsHouse, LLC | 141 W. Jackson Blvd. | Suite 800 | Chicago IL, 60604
312-676-8870
sthomas@xxxxxxxxxxxxxxxx
______________________________________________
See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email
