So I'm updating a trusty cluster to Jewel and updating the kernel at the same time.
Got around some mon issues, and that seems ok, but after upgrading one of my OSD nodes, I'm getting these errors in the old log on that node.
0 ceph version 10.2.1 (3a66dd4f30852819c1bdaa8ec23c795d4ad77269), process ceph-osd, pid 22401
2016-05-13 10:51:39.416511 7fe486e73800 0 pidfile_write: ignore empty --pid-file
2016-05-13 10:51:39.423336 7fe486e73800 -1 filestore(/var/lib/ceph/osd/ceph-6) FileStore::mount: unable to access basedir '/var/lib/ceph/osd/ceph-6': (13) Permission denied
2016-05-13 10:51:39.423342 7fe486e73800 -1 osd.6 0 OSD:init: unable to mount object store
2016-05-13 10:51:39.423348 7fe486e73800 -1 ** ERROR: osd init failed: (13) Permission denied
2016-05-13 10:51:39.572099 7f4d86a68800 0 set uid:gid to 1000:1000 (ceph:ceph)
2016-05-13 10:51:39.572126 7f4d86a68800 0 ceph version 10.2.1 (3a66dd4f30852819c1bdaa8ec23c795d4ad77269), process ceph-osd, pid 22530
2016-05-13 10:51:39.573744 7f4d86a68800 0 pidfile_write: ignore empty --pid-file
2016-05-13 10:51:39.580543 7f4d86a68800 -1 filestore(/var/lib/ceph/osd/ceph-6) FileStore::mount: unable to access basedir '/var/lib/ceph/osd/ceph-6': (13) Permission denied
2016-05-13 10:51:39.580549 7f4d86a68800 -1 osd.6 0 OSD:init: unable to mount object store
2016-05-13 10:51:39.580554 7f4d86a68800 -1 ** ERROR: osd init failed: (13) Permission denied
2016-05-13 10:51:39.798196 7fc0daf80800 0 set uid:gid to 1000:1000 (ceph:ceph)
2016-05-13 10:51:39.798222 7fc0daf80800 0 ceph version 10.2.1 (3a66dd4f30852819c1bdaa8ec23c795d4ad77269), process ceph-osd, pid 22724
2016-05-13 10:51:39.799923 7fc0daf80800 0 pidfile_write: ignore empty --pid-file
2016-05-13 10:51:39.806382 7fc0daf80800 -1 filestore(/var/lib/ceph/osd/ceph-6) FileStore::mount: unable to access basedir '/var/lib/ceph/osd/ceph-6': (13) Permission denied
2016-05-13 10:51:39.806387 7fc0daf80800 -1 osd.6 0 OSD:init: unable to mount object store
2016-05-13 10:51:39.806390 7fc0daf80800 -1 ** ERROR: osd init failed: (13) Permission denied
2016-05-13 10:51:39.954085 7feb28741800 0 set uid:gid to 1000:1000 (ceph:ceph)
2016-05-13 10:51:39.954112 7feb28741800 0 ceph version 10.2.1 (3a66dd4f30852819c1bdaa8ec23c795d4ad77269), process ceph-osd, pid 22888
2016-05-13 10:51:39.955839 7feb28741800 0 pidfile_write: ignore empty --pid-file
2016-05-13 10:51:39.962785 7feb28741800 -1 filestore(/var/lib/ceph/osd/ceph-6) FileStore::mount: unable to access basedir '/var/lib/ceph/osd/ceph-6': (13) Permission denied
2016-05-13 10:51:39.962791 7feb28741800 -1 osd.6 0 OSD:init: unable to mount object store
2016-05-13 10:51:39.962796 7feb28741800 -1 ** ERROR: osd init failed: (13) Permission denied
The OSDs are all mounted as expected:
/dev/sdl1 3.7T 1.8T 1.9T 49% /var/lib/ceph/osd/ceph-6
/dev/sdn1 3.7T 2.0T 1.7T 55% /var/lib/ceph/osd/ceph-18
/dev/sdb1 3.7T 2.2T 1.5T 61% /var/lib/ceph/osd/ceph-30
/dev/sdf1 3.7T 2.0T 1.7T 54% /var/lib/ceph/osd/ceph-54
/dev/sdh1 3.7T 1.9T 1.8T 52% /var/lib/ceph/osd/ceph-66
/dev/sde1 3.7T 1.9T 1.8T 51% /var/lib/ceph/osd/ceph-48
/dev/sdd1 3.7T 1.8T 1.9T 49% /var/lib/ceph/osd/ceph-42
/dev/sdk1 3.7T 1.7T 2.0T 46% /var/lib/ceph/osd/ceph-0
/dev/sda1 3.7T 1.9T 1.8T 51% /var/lib/ceph/osd/ceph-24
/dev/sdm1 3.7T 1.9T 1.8T 52% /var/lib/ceph/osd/ceph-12
/dev/sdc1 3.7T 1.7T 2.0T 47% /var/lib/ceph/osd/ceph-36
/dev/sdg1 3.7T 1.8T 1.9T 49% /var/lib/ceph/osd/ceph-60
Any ideas as to what could be going on?
//Tu Holmes
