I'm trying to explain swapping out behavior that is causing unpredictability in our app. We’re running redhat kernel 2.6.32-431 (yes older) on a host that has 24GB of physical memory and a swap space of 4GB. Swappiness is set to 10, min_free_kbytes is 90112. Over time free memory drop down to ~180MB due to filesystem usage over a few hours, which is immediately followed by 2GB or 4GB of memory being reclaimed. We expect the free memory to be used by the file system cache, and also expect kswapd to be triggered when min_free_kbytes is breached. However what was not expected was the 2-4GB of memory being reclaimed. Our understanding is once free memory hits high water mark which is 2 x min_free_kbytes, kswapd duty cycle finishes. 2-3GB is usually the file system cache pages, however the other 1-2GB are anonymous pages. It’s a issue for us to see the anonymous pages swapped out because they correspond to a process (JVM) whose performance is important to us. This process virtual and resident size is static at 15GB. Why is kswapd so aggressive in reclaiming pages when it clearly reclaimed more than high water immediately after the FS cache was flushed? Is this by design?