On Sat, Mar 11, 2017 at 12:21 PM, <cephmailinglist@xxxxxxxxx> wrote: > The upgrade of our biggest cluster, nr 4, did not go without > problems. Since we where expecting a lot of "failed to encode map > e<version> with expected crc" messages, we disabled clog to monitors > with 'ceph tell osd.* injectargs -- --clog_to_monitors=false' so our > monitors would not choke in those messages. The upgrade of the > monitors did go as expected, without any problem, the problems > started when we started the upgrade of the OSDs. In the upgrade > procedure, we had to change the ownership of the files from root to > the user ceph and that process was taking so long on our cluster that > completing the upgrade would take more then a week. We decided to > keep the permissions as they where for now, so in the upstart init > script /etc/init/ceph-osd.conf, we changed '--setuser ceph --setgroup > ceph' to '--setuser root --setgroup root' and fix that OSD by OSD > after the upgrade was completely done For others following this thread who still have the hammer→jewel upgrade ahead: there is a ceph.conf option you can use here; no need to fiddle with the upstart scripts. setuser match path = /var/lib/ceph/$type/$cluster-$id What this will do is it will check which user owns files in the respective directories, and then start your Ceph daemons under the appropriate user and group IDs. In other words, if you enable this and you upgrade from Hammer to Jewel, and your files are still owned by root, your daemons will also continue run as root:root (as they did in hammer). Then, you can stop your OSDs, run the recursive chown, and restart the OSDs one-by-one. When they come back up, they will just automatically switch to running as ceph:ceph. Cheers, Florian
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
