Re: kernel page caching questions

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Mulyadi,

Thanks for the response.

Mulyadi Santosa wrote:
On Sun, Aug 8, 2010 at 22:13, Philip Downer <phil@xxxxxxxxx> wrote:
Hi folks,

I'm developing a kernel module (sorry under nda till product release so
no code available) which is currently allocating ~6MB of buffers (3 x
2MB buffers) with pci_alloc_consistent for dma usage, it will be less
later on but for now the hardware requires this.

Sorry to snip out a lot of your original post. Kindly allow me to
share my idea here:

You seems to allocate large continous memory block, precisely in DMA
range, am I right? Assuming we're talking about the "old" DMA which
can address only the first 16 MiB of physical RAM, then I think that's
the problem.

Actually I should have specified this, the pci card has 32bit addressing so no, it's not just the first 16MB of RAM and pci_alloc_consistent in this case appears to consistently allocate memory in address ranges much higher than the first 16MB

Quite likely, your function fails to allocate that large physically
continous block of RAM. I stress the word "physically continous".
During run time, as you probably aware too, allocation and
deallocation works hundreds..maybe thousands or more. Thus
fragmentation is inevitable. Effect? You bet...hard....er to get
physically contigous block of RAM.

Yes, but the reason it has trouble allocating memory is because of the unnecessary page caching caused by my application which is streaming data from the device to disk. If the cached memory is dumped or the size of the page cache is restricted, then the kernel can allocate the requested memory but I'm trying to avoid the page caching of the streaming data entirely.

So, again if I am right interpreting your situation, you have several solutions:
1. allocate it during boot.... i forgot the func names...but it will
works better since RAM is still pristine situation that time

Actually I'd rather avoid this because it's not something to undertake lightly and there are not yet any standard functions to do this, see http://lwn.net/Articles/396702/

2. allocate smaller... and somehow strings them together. So instead
of 6 MiB, how about 512 KiB times 12?

As already stated, I'm not requesting 6MB (IME it's not actually possible to allocate more than 4MB of contiguous memory), but 3 separate buffers of 2MB which individually need to be contiguous (hardware restrictions).

More about it, maybe you can read my article
http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html.
It talks about user mode mem allocation, but IMO it applies in kernel
too...

Thanks, thats an interesting article, but I think it's not applicable in this case.

Philip


--
To unsubscribe from this list: send an email with
"unsubscribe kernelnewbies" to ecartis@xxxxxxxxxxxx
Please read the FAQ at http://kernelnewbies.org/FAQ



[Index of Archives]     [Newbies FAQ]     [Linux Kernel Mentors]     [Linux Kernel Development]     [IETF Annouce]     [Git]     [Networking]     [Security]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux RAID]     [Linux SCSI]     [Linux ACPI]
  Powered by Linux