upgrading cluster with 2 nodes to new major version

[Mariel Cherkassky <mariel.cherkassky@xxxxxxxxx>]

Hey,

I have an environment with 2 nodes (one PRIMARY, one secondary in read only) that has pg 9.6 version with repmr 4.3.

I'm trying to upgrade both nodes to pg11 according to the following doc : https://www.postgresql.org/docs/current/pgupgrade.html

I wanted to get your opinion regarding the rsync command. According to the docs, I need to run the following rsync command in the PRIMARY for each secondary node : 

rsync --archive --delete --hard-links --size-only --no-inc-recursive primary_old_cluster_data_dir primary_new_cluster_data_dir secondary_remote_data_dir

I wanted to run the following rsync command from the secondary : 

rsync --archive --hard-links --size-only --verbose --human-readable --no-inc-recursive --delete  root@primaryIP:PRIMARY_OLD_PGSQL_PATH SECONDARY_PGSQL_PATH

 However,

as it seems, when I run the first command from the primary for every seconday the rsync succeeds and I can start the db without any issues.

On the other hand, after running the second command from the secondary I fail to start the db and I get the following error : 

The database cluster was initialized with PG_CONTROL_VERSION 960, but the server was compiled with PG_CONTROL_VERSION 1100. 

Can anyone explain why is it mandatory to run the first command on the primary ? and why cant I run rsync from the new dir in the primary to the old dir in the secondary ,why I need to specify 3 directories according to the docs ?

Last question - after the upgrade (when I use the first command) my replication slots are deleted. any idea why ? Is it normal after upgrade and I should create them from scratch ?

thanks.