A pleasure. Hope it helps :) Happy to share if you need any more information Zac. Cheers, Tom On Wed, 26 Apr 2023 at 18:14, Dan van der Ster <dan.vanderster@xxxxxxxxx> wrote: > Thanks Tom, this is a very useful post! > I've added our docs guy Zac in cc: IMHO this would be useful in a > "Tips & Tricks" section of the docs. > > -- dan > > ______________________________ > Clyso GmbH | https://www.clyso.com > > > > > On Wed, Apr 26, 2023 at 7:46 AM Thomas Bennett <thomas@xxxxxxxx> wrote: > > > > I would second Joachim's suggestion - this is exactly what we're in the > > process of doing for a client, i.e migrating from Luminous to Quincy. > > However below would also work if you're moving to Nautilus. > > > > The only catch with this plan would be if you plan to reuse any hardware > - > > i.e the hosts running rados gateways and mons, etc. If you have enough > > hardware to spare this is a good plan. > > > > My process: > > > > 1. Stand a new Quincy cluster and tune the cluster. > > 2. Migrate user information, secrets and access keys (using > > radosg-admin in a script). > > 3. Using a combination of rclone and parallel to push data across from > > the old cluster to the new cluster. > > > > > > Below is a bash script I used to capture all the user information on the > > old cluster and I ran it on the new cluster to create users and keep > their > > secrets and keys the same. > > > > # > > for i in $(radosgw-admin user list | jq -r .[]); do > > USER_INFO=$(radosgw-admin user info --uid=$i) > > USER_ID=$(echo $USER_INFO | jq -r '.user_id') > > DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name') > > EMAIL=$(echo $USER_INFO | jq '.email') > > MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)') > > ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key') > > SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key') > > echo "radosgw-admin user create --uid=$USER_ID > > --display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS > > --access-key=$ACCESS --secret-key=$SECRET" | tee -a > > generated.radosgw-admin-user-create.sh > > done > > # > > > > Rclone is a really powerful tool! I lazily set up a backends for each > user, > > by appending below to the for loop in the above script. Below script is > not > > pretty but it does the job: > > # > > echo "" >> generated.rclone.conf > > echo [old-cluster-$USER_ID] >> generated.rclone.conf > > echo type = s3 >> generated.rclone.conf > > echo provider = Ceph >> generated.rclone.conf > > echo env_auth = false >> generated.rclone.conf > > echo access_key_id = $ACCESS >> generated.rclone.conf > > echo secret_access_key = $SECRET >> generated.rclone.conf > > echo endpoint = http://xx.xx.xx.xx:xxxx >> generated.rclone.conf > > echo acl = public-read >> generated.rclone.conf > > echo "" >> generated.rclone.conf > > echo [new-cluster-$USER_ID] >> generated.rclone.conf > > echo type = s3 >> generated.rclone.conf > > echo provider = Ceph >> generated.rclone.conf > > echo env_auth = false >> generated.rclone.conf > > echo access_key_id = $ACCESS >> generated.rclone.conf > > echo secret_access_key = $SECRET >> generated.rclone.conf > > echo endpoint = http://yy.yy.yy.yy:yyyy >> generated.rclone.conf > > echo acl = public-read >> generated.rclone.conf > > echo "" >> generated.rclone.conf > > # > > > > Copy the generated.rclone.conf to the node that is going to act as the > > transfer node (I just used the new rados gateway node) into > > ~/.config/rclone/rclone.conf > > > > Now if you run rclone lsd old-cluser-{user}: (it even tab completes!) > > you'll get a list of all the buckets for that user. > > > > You could even simply rclone sync old-cluser-{user}: new-cluser-{user}: > and > > it should sync all buckets for a user. > > > > Catches: > > > > - Use the scripts carefully - our buckets for this one user are set > > public-read - you might want to check each line of the script if you > use it. > > - Quincy bucket naming convention is stricter than Luminous. I've had > to > > catch some '_' and upper cases and fix them in the command line I > generate > > for copying each bucket. > > - Using rclone will take a long time.Feed a script into parallel sped > > things up for me: > > - # parallel -j 10 < sync-script > > - Watch out for lifecycling! Not sure how to handle this to make sure > > it's captured correctly. > > > > Cheers, > > Tom > > > > On Tue, 25 Apr 2023 at 22:36, Marc <Marc@xxxxxxxxxxxxxxxxx> wrote: > > > > > > > > Maybe he is limited by the supported OS > > > > > > > > > > > > > > I would create a new cluster with Quincy and would migrate the data > from > > > > the old to the new cluster bucket by bucket. Nautilus is out of > support > > > > and > > > > I would recommend at least to use a ceph version that is receiving > > > > Backports. > > > > > > > > huxiaoyu@xxxxxxxxxxxx <huxiaoyu@xxxxxxxxxxxx> schrieb am Di., 25. > Apr. > > > > 2023, 18:30: > > > > > > > > > Dear Ceph folks, > > > > > > > > > > I would like to listen to your advice on the following topic: We > have > > > > a > > > > > 6-node Ceph cluster (for RGW usage only ) running on Luminous > 12.2.12, > > > > and > > > > > now will add 10 new nodes. Our plan is to phase out the old 6 > nodes, > > > > and > > > > > run RGW Ceph cluster with the new 10 nodes on Nautilus version。 > > > > > > > > > > I can think of two ways to achieve the above goal. The first method > > > > would > > > > > be: 1) Upgrade the current 6-node cluster from Luminous 12.2.12 > to > > > > > Nautilus 14.2.22; 2) Expand the cluster with the 10 new nodes, and > > > > then > > > > > re-balance; 3) After rebalance completes, remove the 6 old nodes > from > > > > the > > > > > cluster > > > > > > > > > > The second method would get rid of the procedure to upgrade the > old 6- > > > > node > > > > > from Luminous to Nautilus, because those 6 nodes will be phased out > > > > anyway, > > > > > but then we have to deal with a hybrid cluster with 6-node on > Luminous > > > > > 12.2.12, and 10-node on Nautilus, and after re-balancing, we can > > > > remove the > > > > > 6 old nodes from the cluster. > > > > > > > > > > Any suggestions, advice, or best practice would be highly > appreciated. > > > > > > > > _______________________________________________ > > > ceph-users mailing list -- ceph-users@xxxxxxx > > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > > > > _______________________________________________ > > ceph-users mailing list -- ceph-users@xxxxxxx > > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |