Ben Brehmer wrote:
Since my input data is split into about 200 files (3GB each), I could potentially spawn one load command for each file. What would be the maximum number of input connections Postgres can handle without bogging down?You can expect to easily get one loader process per real CPU going. Beyond that, it depends on how CPU intensive they all are and what the resulting I/O rate out of the combination is. You're probably going to run out of CPU on a loading job long before you hit any of the other limits in this area, and potentially you could run out of disk throughput on a cloud system before that. PostgreSQL isn't going to bog down on a connection basis until you've reached several hundred of them, your loader will be lucky to hit 10 active processes before it grinds to a halt on some physical resources unrelated to general database scaling. -- Greg Smith 2ndQuadrant Baltimore, MD PostgreSQL Training, Services and Support greg@xxxxxxxxxxxxxxx www.2ndQuadrant.com |
