On Thu, Jan 13, 2005 at 23:20:31 -0800, prasanna wakhare wrote: > Hi all, > I have a simple question in mind. I want to introduce > client side caching in any network or cluster file > system. The caches are in-memory cache not secondary > cache as in AFS but as sprite. > My question is all the cache is in kernel space and if > large file happens to mmap or read/write say 2 GB etc. > What would be consequence if there is great limit on > my cache. > We know kernel space virtual address are from 3GB to > 4GB. If i start allocating cache block as soon as i'm > getting file from storage node and if my cache size is > 5GB then this will hang the system as kernel cant have > that much addresses refer to. And in that case we need > either CPU to address 64 bit or something like that. > But even then there is limit on cacheing the file > data. > So how much shall i keep my cache size to have best > performance. > I hope i'm pretty clear in what i asked? You are clear... in that you have never heared of page cache. The page cache is handled in the generic_file_read and generic_file_write (and generic_file_mmap or somesuch IIRC) functions, if you use them for appropriate methods in file_operations. These functions require you to implement the address_space_operations methods, namely readpage, writepage, prepare_write and commit_write. The file is never downloaded all at once. It is downloaded page-by-page on demand (with some readahead). So only the data actualy requested are downloaded and they are dropped when they are not accessed for some time. Kernel uses all pages it does not need for anything else for caching files. In that, it considers all pages equal. No matter where they came from (except it of course affect thow they are writeen out), when a page is needed, kernel chooses some page that was not accessed in some period of time and drops it (writing dirty data in the process). There is a small amount of pages kept really free so the allocation does not need to wait for the write-out. And the key point in all the above is, that it's not your business to manage the cache. Leave it to the VM subsystem -- it does a better job. Btw: The kernel address space is 1G large on 32 bit paltforms, but it's not a limit of anything -- pages (for pagecache) are addressed using the high memory mechanizm if there are not enough addresses. ------------------------------------------------------------------------------- Jan 'Bulb' Hudec <bulb@xxxxxx>
Attachment:
signature.asc
Description: Digital signature
