Hi, I have an issue with a rather large CASE WHEN and I cannot figure out why it is so slow... First, here is my test query : SELECT CASE WHEN dwh_company_id = 1 THEN CASE WHEN wv.source ~ '^$' THEN 'Not tracked' WHEN wv.source ~ '^1$' THEN 'Not tracked1' WHEN wv.source ~ '^2$' THEN 'Not tracked2' WHEN wv.source ~ '^3$' THEN 'Not tracked3' WHEN wv.source ~ '^4$' THEN 'Not tracked4' WHEN wv.source ~ '^5$' THEN 'Not tracked5' WHEN wv.source ~ '^6$' THEN 'Not tracked6' WHEN wv.source ~ '^7$' THEN 'Not tracked7' WHEN wv.source ~ '^8$' THEN 'Not tracked8' WHEN wv.source ~ '^9$' THEN 'Not tracked9' WHEN wv.source ~ '^10$' THEN 'Not tracked10' WHEN wv.source ~ '^11$' THEN 'Not tracked11' WHEN wv.source ~ '^12$' THEN 'Not tracked12' WHEN wv.source ~ '^13$' THEN 'Not tracked13' WHEN wv.source ~ '^14$' THEN 'Not tracked14' WHEN wv.source ~ '^15$' THEN 'Not tracked15' WHEN wv.source ~ '^16$' THEN 'Not tracked16' WHEN wv.source ~ '^17$' THEN 'Not tracked17' WHEN wv.source ~ '^18$' THEN 'Not tracked18' WHEN wv.source ~ '^19$' THEN 'Not tracked19' WHEN wv.source ~ '^20$' THEN 'Not tracked20' WHEN wv.source ~ '^21$' THEN 'Not tracked21' WHEN wv.source ~ '^22$' THEN 'Not tracked22' WHEN wv.source ~ '^23$' THEN 'Not tracked23' WHEN wv.source ~ '^24$' THEN 'Not tracked24' WHEN wv.source ~ '^25$' THEN 'Not tracked25' WHEN wv.source ~ '^26$' THEN 'Not tracked26' WHEN wv.source ~ '^27$' THEN 'Not tracked27' WHEN wv.source ~ '^28$' THEN 'Not tracked28' --WHEN wv.source ~ '^29$' THEN 'Not tracked29' WHEN wv.source ~ '^30$' THEN 'Not tracked30' WHEN wv.source ~ '^31$' THEN 'Not tracked31' WHEN wv.source ~ '^32$' THEN 'Not tracked32' END ELSE 'Others' END as channel FROM ( SELECT wv.id, wv.ga_id, split_part(wv.ga_source_medium, ' / ', 1) as source, ga.dwh_source_id, s.dwh_company_id FROM marketing.web_visits wv INNER JOIN dwh_metadata.google_analytics ga ON ga.ga_id = wv.ga_id INNER JOIN dwh_manager.sources s ON ga.dwh_source_id =s.dwh_source_id --WHERE s.dwh_company_id = 1 LIMIT 100000 ) wv This is a pretty simple case, my subquery (or CTE when using WITH statement) should return 5 fields with more or less this structure : Id : character(32) Ga_id : bigint Source : character(32) Medium : character(32) dwh_company_id : bigint On top of which I apply a case when statement… Now the weird thing is, using this query I notice a significant drop in performance as the “case when” is getting bigger. If I run the query as if, I get the following exec plain and execution time: Subquery Scan on wv (cost=6.00..29098.17 rows=100000 width=36) (actual time=0.828..22476.917 rows=100000 loops=1) Buffers: shared hit=3136 -> Limit (cost=6.00..11598.17 rows=100000 width=58) (actual time=0.209..133.429 rows=100000 loops=1) Buffers: shared hit=3136 -> Hash Join (cost=6.00..1069811.24 rows=9228690 width=58) (actual time=0.208..119.297 rows=100000 loops=1) Hash Cond: (wv_1.ga_id = ga.ga_id) Buffers: shared hit=3136 -> Seq Scan on web_visits wv_1 (cost=0.00..877005.78 rows=20587078 width=50) (actual time=0.004..18.412 rows=100000 loops=1) Buffers: shared hit=3133 -> Hash (cost=5.50..5.50 rows=40 width=12) (actual time=0.184..0.184 rows=111 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 5kB Buffers: shared hit=3 -> Hash Join (cost=1.88..5.50 rows=40 width=12) (actual time=0.056..0.148 rows=111 loops=1) Hash Cond: (ga.dwh_source_id = s.dwh_source_id) Buffers: shared hit=3 -> Seq Scan on google_analytics ga (cost=0.00..2.89 rows=89 width=8) (actual time=0.007..0.028 rows=111 loops=1) Buffers: shared hit=2 -> Hash (cost=1.39..1.39 rows=39 width=8) (actual time=0.042..0.042 rows=56 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 3kB Buffers: shared hit=1 -> Seq Scan on sources s (cost=0.00..1.39 rows=39 width=8) (actual time=0.005..0.020 rows=56 loops=1) Buffers: shared hit=1 Planning time: 0.599 ms Execution time: 22486.216 ms Then try commenting out only one line in the case when and the query run 10x faster : Subquery Scan on wv (cost=6.00..28598.17 rows=100000 width=36) (actual time=0.839..2460.002 rows=100000 loops=1) Buffers: shared hit=3136 -> Limit (cost=6.00..11598.17 rows=100000 width=58) (actual time=0.210..112.043 rows=100000 loops=1) Buffers: shared hit=3136 -> Hash Join (cost=6.00..1069811.24 rows=9228690 width=58) (actual time=0.209..99.513 rows=100000 loops=1) Hash Cond: (wv_1.ga_id = ga.ga_id) Buffers: shared hit=3136 -> Seq Scan on web_visits wv_1 (cost=0.00..877005.78 rows=20587078 width=50) (actual time=0.004..14.048 rows=100000 loops=1) Buffers: shared hit=3133 -> Hash (cost=5.50..5.50 rows=40 width=12) (actual time=0.184..0.184 rows=111 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 5kB Buffers: shared hit=3 -> Hash Join (cost=1.88..5.50 rows=40 width=12) (actual time=0.058..0.146 rows=111 loops=1) Hash Cond: (ga.dwh_source_id = s.dwh_source_id) Buffers: shared hit=3 -> Seq Scan on google_analytics ga (cost=0.00..2.89 rows=89 width=8) (actual time=0.007..0.025 rows=111 loops=1) Buffers: shared hit=2 -> Hash (cost=1.39..1.39 rows=39 width=8) (actual time=0.042..0.042 rows=56 loops=1) Buckets: 1024 Batches: 1 Memory Usage: 3kB Buffers: shared hit=1 -> Seq Scan on sources s (cost=0.00..1.39 rows=39 width=8) (actual time=0.006..0.021 rows=56 loops=1) Buffers: shared hit=1 Planning time: 0.583 ms Execution time: 2467.484 ms Why this drop in performance for only one (in this simple example) condition ? I do not really understand it. If I add more conditions to the query (let say 1 or 2) it is also getting slower. And it’s not a few ms, it is around 5 sec or so. (which is huge considering I only take in my example 1/500 of my data with LIMIT. Before we deviate from the problem I have (which is why the sudden drop of performance) let me clarify a few things about this query : - The purpose is not to rewrite it, with a join or whatever, the case when actually comes from a function which is auto-generated by another app we have - My example is pretty simple and regex expressions could be replaced by equals, the real case when query contains way more complicated regex - This is subset of my CASE WHEN, it is much bigger, I cut it at the “bottleneck” point for this post. Thanks a lot. Best Regards, Kevin |
