Deadlock on "select ... for update"?

[Craig James <craig_james@xxxxxxxxxxxxxx>]

Several times recently one of our databases has gotten stuck with the following situation:

postgres=# select datname, procpid, usename, current_query from pg_stat_activity where current_query != '<IDLE>';
  datname   | procpid | usename  |                                             current_query
------------+---------+----------+--------------------------------------------------------------------------------------------------------
 emolecules |   13503 | customerdb | select tableid from hitlist_table_pool where hitlistid <= 0 for update
 emolecules |   32082 | customerdb | select tableid from hitlist_table_pool where hitlistid <= 0 for update
 emolecules |   17974 | customerdb | select tableid from hitlist_table_pool where hitlistid <= 0 for update
 emolecules |   31299 | customerdb | select tableid from hitlist_table_pool where hitlistid = 0 limit 1 for update
 emolecules |   30247 | customerdb | select tableid from hitlist_table_pool where hitlistid = 0 limit 1 for update
 postgres   |    1705 | postgres | select datname, procpid, usename, current_query from pg_stat_activity where current_query != '<IDLE>';
 emolecules |   28866 | customerdb | <IDLE> in transaction
 emolecules |   21394 | customerdb | select tableid from hitlist_table_pool where hitlistid = 0 limit 1 for update
 emolecules |   22237 | customerdb | select tableid from hitlist_table_pool where hitlistid = 0 limit 1 for update
(9 rows)

It's obvious that they're all waiting ... but for what?  The "<IDLE>" process looks like the culprit, but how do I figure out what it's doing?

The next time this happens, what queries can I run to help diagnose what's going on?

This is PG 8.4.4 on Ubuntu 10.

Thanks,

Craig