I was able to reproduce in a somewhat contrived way:
sh -c 'ulimit -v 1024000 ; /usr/local/pgsql/bin/postgres -D ./pg12dev5 -cport=1234' &
postgres=# SET work_mem='64kB';SET client_min_messages =debug1;SET log_statement_stats=on;explain(analyze) WITH v AS MATERIALIZED (SELECT * FROM generate_series(1,9999999)i WHERE i%10<10 AND i%11<11 AND i%12<12 AND i%13<13 AND i%14<14 AND i%15<15 AND i%16<16 AND i%17<17 AND i%18<18 AND i%19<19 AND i%20<20 AND i%21<21 ) SELECT * FROM generate_series(1,99)k JOIN v ON k=i;
HashTableContext: 8192 total in 1 blocks; 7696 free (7 chunks); 496 used
hash batch files: 852065104 total in 101318 blocks; 951896 free (20 chunks); 851113208 used
HashBatchContext: 73888 total in 4 blocks; 24280 free (6 chunks); 49608 used
2019-04-21 04:11:02.521 CDT [4156] ERROR: out of memory
2019-04-21 04:11:02.521 CDT [4156] DETAIL: Failed on request of size 8264 in memory context "hash batch files".
