Re: A very long running query....

[Ioannis Anagnostopoulos <ioannis@xxxxxxxxxx>]

On 20/07/2012 22:33, Claudio Freire wrote:

On Fri, Jul 20, 2012 at 6:27 PM, Ioannis Anagnostopoulos
<ioannis@xxxxxxxxxx> wrote:

On 20/07/2012 22:23, Claudio Freire wrote:

On Fri, Jul 20, 2012 at 6:19 PM, Ioannis Anagnostopoulos
<ioannis@xxxxxxxxxx> wrote:

"        ->  Nested Loop  (cost=0.00..20942.93 rows=53 width=144) (actual
time=62.174..17783236.718 rows=387105 loops=1)"
"              Join Filter: (feed_all_y2012m07.message_copies.msg_id =
feed_all_y2012m07.ship_pos_messages.msg_id)"
"              ->  Append  (cost=0.00..19057.93 rows=53 width=33) (actual
time=62.124..5486473.545 rows=387524 loops=1)"

Misestimated row counts... did you try running an analyze, or upping
statistic targets?

I have run analyse every so often. I think the problem is that as I get 16K
new rows every minutes, the "stats" are always out... Possible?

Looking at this:

"                    ->  Index Scan using
idx_message_copies_wk2_date_src_pos_partial on message_copies_wk2
message_copies  (cost=0.00..19057.93 rows=52 width=32) (actual
time=62.124..5486270.845 rows=387524 loops=1)"
"                          Index Cond: ((date_trunc('day'::text,
msg_date_rec) = '2012-07-17 00:00:00'::timestamp without time zone)
AND (src_id = 1))"
"                          Filter: ((date_part('day'::text,
msg_date_rec) = 17::double precision) AND (NOT (((((pos_georef1)::text
|| (pos_georef2)::text) || (pos_georef3)::text) ||
(pos_georef4)::text) IS NULL)) AND (((((pos_georef1)::text ||
(pos_georef2)::text) || (pos_georef3)::text) || (pos_georef4)::text)
<> ''::text))"

It's very possible.

I think pg 9.1 had a fix for that, but I'm not sure it will help in
your case, I'd have to know what that index looks like.

Here is the index:

CREATE INDEX idx_message_copies_wk2_date_src_pos_partial
  ON feed_all_y2012m07.message_copies_wk2
  USING btree
  (date_trunc('day'::text, msg_date_rec), src_id, (((pos_georef1::text || pos_georef2::text) || pos_georef3::text) || pos_georef4::text))
TABLESPACE archive
  WHERE (((pos_georef1::text || pos_georef2::text) || pos_georef3::text) || pos_georef4::text) IS NOT NULL OR NOT (((pos_georef1::text || pos_georef2::text) || pos_georef3::text) || pos_georef4::text) = ''::text;