It looks like it was a problem with NFS. We're not really sure what was wrong with it but once we failed over to an iSCSI mount for the data everything is running just fine.
On Thu, Dec 27, 2012 at 6:12 PM, Tom Lane <tgl@xxxxxxxxxxxxx> wrote:
Nikolas Everett <nik9000@xxxxxxxxx> writes:OK...
> On Thu, Dec 27, 2012 at 4:33 PM, Tom Lane <tgl@xxxxxxxxxxxxx> wrote:
>> Nikolas Everett <nik9000@xxxxxxxxx> writes:
>>> We straced the backend during the explain and it looked like the open
>>> commands were taking several seconds each.
>> Kind of makes me wonder if you have a whole lot of tables ("whole lot"
>> in this context probably means tens of thousands) and are storing the
>> database on a filesystem that doesn't scale well to lots of files in one
>> directory. If that's the explanation, the reason the 8.3 installation
>> was okay was likely that it was stored on a more modern filesystem.
> We have 1897 files for our largest database which really isn't a whole lot.
Now I'm wondering about network glitches or NFS configuration problems.
> The old servers were EXT3 over FC to a NetApp running RHEL5 PPC. The new
> servers are on NFS to the same NetApp running RHEL5 Intel.
This is a bit outside my expertise unfortunately, but it seems clear
that your performance issue is somewhere in that area.
regards, tom lane
