Search Postgresql Archives

Re: Splitting queries across servers

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Max wrote:
And you absolutely want a 64-bit OS with that much memory. Even on our
development server with just 3GB of RAM, we see quite big jumps in
performance after moving from 32-bit Linux to 64-bit Linux. I'd hate to
think about 64GB of RAM being swapped in and out using PAE.



What's PAE ?

32-bit OS can only address up to 4GB of RAM. Of that 4GB, operating systems usually split up the address range as 2GB for userspace and 2GB for kernel. Depending on what OS/version you use, you might be able to change this to 3/1 -- and I even recall a wacky mode under Linux that can get you 4/4 (probably using PAE to swap both areas in and out).


So if you have more than 2/3/4GB of memory, the OS has to create a "window" under the 4GB space and "swap" memory > 4GB in/out. It's not really an actual swap of course -- that would absolutely cripple performance to be unusable. It's more like an moving window that can point to memory areas > 4GB. Still, there is a performance hit. If process one needs memory at 6-7GB and process two needs it at 10-11GB, there's going to be some contention for that window.

Here's a question for the developers: what's the memory consumption
difference when you move a 3GB database from a 32 bit machine to a 64 ?
given that the whole thing must be in RAM.

Isn't the whole data taking up more RAM because pointers are now 64 bits
instead of 32 ?

Yes and no. Yes in that the memory used directly by Postgres takes twice the amount of space. No in that the optimal setup for Postgres is to have small settings for shared_buffers and leave the lion's share of memory for the OS cache.


I have a bunch of servers -- both 32-bit and 64-bit. The 32-bit servers @ 10K shared_buffers use 72MB of RAM. The 64-bit servers @ 10K shared_buffers use 144MB of RAM. If these servers only had 256MB, it would be a big deal. But since they're all in the gigabytes, the difference between 3500MB of cache versus 3428 is rather insignificant.

Now if files on 64-bit OSes took up twice the space as files on 32-bit OSes, the cache could only hold half the amount of data. But as far as I can see, they don't. I don't think even the blocksizes are twice the amount because most filesystems under Linux can already store files > 4GB.

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Postgresql Jobs]     [Postgresql Admin]     [Postgresql Performance]     [Linux Clusters]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Postgresql & PHP]     [Yosemite]
  Powered by Linux