On 1/18/22 2:19 AM, Victor Sudakov wrote:
Tom Lane wrote:
Victor Sudakov <vas@xxxxxxxxxx> writes:
Tom Lane wrote:
Well, yeah, actually. An ill-advised query will blow out your backend's
memory consumption, potentially leading to a SIGKILL from the dreaded OOM
killer[1] (if you're on Linux), resulting in a backend crash and cluster
restart.
Why should "SELECT COUNT(*) FROM t" ever consume more than work_mem
even if t has 10000 partitions?
Sure, COUNT(*)'s runtime memory consumption is negligible.
But you're not thinking about overhead --- specifically,
1. 10000 relcache entries for the base tables.
2. If you have N indexes per table, N*10000 relcache entries for
the indexes. (The planner will probably have looked at all those
indexes, even if it didn't find any use for them.)
3. 10000 SeqScan plan nodes and associated rangetable entries,
4. Likewise, 10000 instances of executor per-node state.
5. 10000 lock table entries (both shared and local lock tables).
6. Probably a few per-relation things I didn't think of.
I see your point about all that query-related stuff. I hope the
testing of queries in a staging environment should help to detect such
situations.
What about the system catalogs however? Will the extra 10000
tables and 500000 indexes negatively impact the performance of the
system catalogs? Are there any caveats you could think of?
EXPLAIN plans are going to be hilariously gigantic, which means that query
planning would take a loooong time, And the query planner (in v12, at
least) can generate some pretty bad plans in partitioned tables; I bet there
are edge cases in the QP code that don't work well with 10000 partitions and
50000 indices.
--
Angular momentum makes the world go 'round.
