Search Postgresql Archives

Re: ERROR: could not open relation

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



One other detail: pg_autovacuum is running on this system.

I just noticed this from Tom's "Autovacuum loose ends" post from earlier today:

"The code does not make a provision to ignore temporary tables.
Although vacuum.c and analyze.c will disregard the request to touch
such tables, it'd probably be better to recognize the situation further
upstream.  In particular it seems that autovacuum will continually throw
ANALYZE requests for a temp table due to lack of stats."

Is this an issue with the current pg_autovacuum implementation? Is it somehow involved?

Though I feel like we're a little closer to solving this mystery, I want to target the correct vacuuming process with a fix. I have a feeling that explicitly dropping the temp tables in the process that also calls VACUUM prior to the VACUUM is a good short-term fix, but I also want to know whether continuing to run pg_autovacuum with this architecture is a bad idea. If so, we can revert to not using temp tables at all.

Further, why have we only noticed it once when this version of code (and PostgreSQL) has been running for weeks?

--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC

Strategic Open Source: Open Your i™

http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005

On Jul 14, 2005, at 11:42 AM, Thomas F. O'Connell wrote:

Sorry, I didn't have the evidence about the bgwriter before. It was based on conjecture on IRC last night and newly gathered evidence from this morning.

Here's a list of current postgres processes on the box.

postgres 1186 2.8 5.0 437812 417624 ? S Jul13 22:37 postgres: writer process postgres 1187 0.3 0.0 5940 2688 ? S Jul13 2:54 postgres: stats buffer process postgres 1188 3.1 0.1 13456 8856 ? S Jul13 25:16 postgres: stats collector process

My assumption is that it's typically the case that these three processes generally get allocated sequential pids when postgres starts.

In the postgres log, we see these two types of errors, which were the only ones that did not report an IP address:

2005-07-12 01:53:31 CDT 13390 :LOG:  statistics buffer is full
2005-07-13 17:44:51 CDT 13389 :ERROR: could not open relation 1663/32019395/94144936: No such file or directory

So if we assume that pid 13390 referred to the stats collector from yesterday, then presumably 13389 was the bgwriter.

Unfortunately, this is a system where the interloper is superuser (and, yes, changing this has been a TODO). But even so, I need help understanding how one backend could access the temp table of another. Which is what brings me to vacuum or some other system process as a culprit.

Recognizing that the application code will execute as superuser in postgres, here is what is happening in a session:

Several temporary tables (some of which inherit from actual tables) are constructed.
Data is loaded in.
If the data includes updates, in the same session, a VACUUM is performed, else an ANALYZE is performed.

So we know these things:

1. This import process was running.
2. It had started the vacuum, which occurs in the same session as temp tables that inherit from the table being vacuumed. 3. bgwriter reported an error about a missing relation file (I guess this is a strong suspicion more than knowledge, but we strongly suspect).

So could this be somehow related to the fact that VACUUM, as a result of the inheritance relationship in the temp tables, is explicitly attempting to access them?

--
Thomas F. O'Connell
Co-Founder, Information Architect
Sitening, LLC

Strategic Open Source: Open Your i™

http://www.sitening.com/
110 30th Avenue North, Suite 6
Nashville, TN 37203-6320
615-260-0005

On Jul 14, 2005, at 11:07 AM, Tom Lane wrote:


Do you have some evidence that the bgwriter was what was reporting the
error?  You didn't say that before.

The bgwriter only works on dirty shared buffers, so the only way this
could be happening is if a page of a temp table had gotten loaded into a shared buffer, which isn't supposed to happen really. Is it possible
that you had some backend deliberately trying to read a temp table
created by another backend?  (You don't have to assume that the
interloper tried to modify the table; a mere SELECT could have created
the dirty-buffer condition due to hint-bit update.  You do have to
assume that the interloper was superuser, though, else permissions
would have stopped him from accessing someone else's temp table.)

            regards, tom lane



---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
      choose an index scan if your joining column's datatypes do not
      match



---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

              http://www.postgresql.org/docs/faq


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Index of Archives]     [Postgresql Jobs]     [Postgresql Admin]     [Postgresql Performance]     [Linux Clusters]     [PHP Home]     [PHP on Windows]     [Kernel Newbies]     [PHP Classes]     [PHP Books]     [PHP Databases]     [Postgresql & PHP]     [Yosemite]
  Powered by Linux