Hi, We solved it by running Micha scripts, plus we needed to run period update and commit commands (for some reason we had to do it in separate commands): radosgw-admin period update radosgw-admin period commit Btw, we added endpoints to json file, but I am not sure these are needed. And I agree with Micha - this should be noticed in upgrade instructions on Ceph site. We run into this trap on our prod env (upgrading Infernalis -> Jewel). Maybe we should test it more next time.. Br, Arvydas -----Original Message----- From: ceph-users [mailto:ceph-users-bounces@xxxxxxxxxxxxxx] On Behalf Of Micha Krause Sent: Wednesday, July 6, 2016 2:46 PM To: ceph-users@xxxxxxxxxxxxxx Subject: Re: Can't create bucket (ERROR: endpoints not configured for upstream zone) Hi, I think I found a Solution for my Problem, here are my findings: This Bug can be easily reproduced in a test environment: 1. Delete all rgw related pools. 2. Start infernalis radosgw to initialize them again. 3. Create user. 4. User creates bucket. 5. Upgrade radosgw to jewel 6. User creates bucket -> fail I found this scary script from Yehuda: https://raw.githubusercontent.com/yehudasa/ceph/wip-fix-default-zone/src/fix-zone which needs to be modified according to http://www.spinics.net/lists/ceph-users/msg27957.html. After the modification, a lot of the script becomes obsolete (in my opinion), and can be rewritten to this (less scary): #!/bin/sh set -x RADOSGW_ADMIN=radosgw-admin echo "Exercise initialization code" $RADOSGW_ADMIN user info --uid=foo # exercise init code (???) echo "Get default zonegroup" $RADOSGW_ADMIN zonegroup get --rgw-zonegroup=default | sed 's/"id":.*/"id": "default",/g' | sed 's/"master_zone.*/"master_zone": "default",/g' > default-zg.json echo "Get default zone" $RADOSGW_ADMIN zone get --zone-id=default > default-zone.json echo "Creating realm" $RADOSGW_ADMIN realm create --rgw-realm=myrealm echo "Creating default zonegroup" $RADOSGW_ADMIN zonegroup set --rgw-zonegroup=default < default-zg.json echo "Creating default zone" $RADOSGW_ADMIN zone set --rgw-zone=default < default-zone.json echo "Setting default zonegroup to 'default'" $RADOSGW_ADMIN zonegroup default --rgw-zonegroup=default echo "Setting default zone to 'default'" $RADOSGW_ADMIN zone default --rgw-zone=default My plan to do this in production is now: 1. Stop all rados-gateways 2. Upgrade rados-gateways to jewel 3. Run less scary script 4. Start rados-gateways This whole thing is a serious problem, there should at least be a clear notice in the Jewel release notes about this. I was lucky to catch this in my test-cluster, I'm sure a lot of people will run into this in production. Micha Krause Am 05.07.2016 um 09:30 schrieb Micha Krause: > *bump* > > Am 01.07.2016 um 13:00 schrieb Micha Krause: >> Hi, >> >> > In Infernalis there was this command: >>> >>> radosgw-admin regions list >>> >>> But this is missing in Jewel. >> >> Ok, I just found out that this was renamed to zonegroup list: >> >> root@rgw01:~ # radosgw-admin --id radosgw.rgw zonegroup list >> read_default_id : -2 { >> "default_info": "", >> "zonegroups": [ >> "default" >> ] >> } >> >> This looks to me like there is indeed only one zonegroup or region configured. >> >> Micha Krause >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |