Thanks Michael and Erik for the help, I really appreciate it!
Thanks for explaining the context why PostgreSQL doesn't allow binding against port 0.
I somehow didn't consider looking at the postgres tests, though it makes sense that they need to solve this problem. If I read the perl code correctly though it seems that this could, in theory, cause a race? The script checks first whether the port has been assigned to a test, then binds a socket to check whether it is used by someone else, closes this test socker, and then starts a server process. I guess it's unlikely enough, but isn't there a risk that some other process (that isn't controlled by this perl script) binds to the found port right after this test bind but right before postgres calls bind? I guess it should be rare enough so that it wouldn't cause flaky tests.
I decided to implement the following (this strategy works, though it might be a bit brittle if PostgreSQL changes the error output format in the future):
1. Loop, starting from port 5432, incrementing each iteration
2. Start postgres with the given port
3. Parse the output to check whether postgres either writes a line that ends with "could not create any TCP/IP sockets" (in which case I continue) or with "database system is ready to accept connections" (in which case I break).
This is definitely not the most elegant solution, but it should do for now. At the moment I want to be able to set up everything in one process. In my experience this makes debugging problems a bit easier but comes at the cost of a more complex test driver (I recognize that it is a bit weird that the application layer initializes the runtime environment in this case).
Also, this is a hobby-project and I am more interested in fun learning than reducing work :) Generally I would agree that reusing existing and testing code to run this would be better unless there's a really good reason not to do that.
On Sun, Mar 26, 2023 at 7:27 PM Michael Paquier <michael@xxxxxxxxxxx> wrote:
On Sat, Mar 25, 2023 at 11:01:33AM -0600, Markus Pilman wrote:
> Now the problem is that I need to find a TCP port for each running postgres
> instance. There's multiple ways to do this, but by far the easiest one I
> know is to bind to port 0. So my plan was to start postgres with "-p 0" and
> then parse stdout to figure out which port it actually uses. But that
> doesn't seem to work:
Note that you can find some inspiration about that in the code tree
within src/test/perl/PostgreSQL/Test/Cluster.pm, particularly
get_free_port(), where we have now accumulated a couple of years of
experience in designing something that's rather safe, even if it comes
with its own limits. It is in perl so perhaps you could just reuse it
rather than reinvent the wheel? Of course, still it should not be
complicated to translate that in a different language, but there may
be no need to reinvent the wheel. And seeing your first message with
the requirements you list, this does what you are looking for:
- Create an empty cluster.
- Freely create databases, tablespaces, queries, etc.
- Wipe out the whole.
The test cases around src/test/recovery/t/ could be a good starting
point, as well.
--
Michael
