On Sat, May 19, 2018 at 11:24:57AM -0500, Justin Pryzby wrote: > On Sat, May 19, 2018 at 11:08:23AM -0400, Tom Lane wrote: > > Justin Pryzby <pryzby@xxxxxxxxxxxxx> writes: > > > I'll defer fixing this for awhile in case someone wants me to save a copy of > > > the relation/toast/index. From last time, I recall this just needs the right > > > combination of REINDEX/VACUUM/ANALYZE, and the only complication was me > > > needing to realize the right combination of affected DB(s). > > > > If you could come up with such a sequence that causes the problem > > reproducibly, that would be of huge interest, and probably lead to > > a fix promptly. But I don't think that we can do much by looking > > at the post-mortem state --- once the toast rows are gone, they're > > gone, especially if the table's been vacuumed since. > > This is unlikely to allow reproducing it, but for sake of completeness here's a > fuller log. I'll try to trigger on another DB. Did not take long... [pryzbyj@database ~]$ while :; do for db in `psql postgres -Atc "SELECT datname FROM pg_database WHERE datallowconn"`; do for t in pg_statistic pg_attrdef pg_constraint; do echo "$db.$t..."; PGOPTIONS=-cstatement_timeout='9s' psql $db -qc "VACUUM FULL $t"; done; done; done ... postgres.pg_statistic... postgres.pg_attrdef... postgres.pg_constraint... template1.pg_statistic... template1.pg_attrdef... template1.pg_constraint... ts.pg_statistic... ERROR: canceling statement due to statement timeout ts.pg_attrdef... ts.pg_constraint... postgres.pg_statistic... ERROR: missing chunk number 0 for toast value 3372855171 in pg_toast_2619 I'm running this again on another DB, but I wonder if that's enough for anyone else to reproduce it with some consistency ? I think that took something like 10min before failing. Justin