So my experience isn't with pgagent directly, because I have been using my re-written version of it for ~5 years (but at least at one point I had a pretty darn good understanding from doing that rewrite)...please take this with a grain of salt if I am incorrect on anything.
So the agent is only able to connect to a single "db" at a time, so you are correct you'd need to run multiple agents (connected to the different db's) to have things work as you intended.
What is the reason for creating the pgagent extensions in multiple databases? The extension itself is the storage / logs / job definition, so each extra one is totally detached from the rest. You can still use a single pgagent install to connect to many databases to do work, so I am just not sure why you'd need multiple.
