Hi,
I have a working ceph cluster with storage nodes running Ubuntu 14.04
and ceph hammer 0.94.5.
Now I want to switch to CentOS 7.1 (forget about the reasons for now, I
can explain, but it would be a long story and irrelevant to my question).
I've set the osd noout flag and norebalance,norecover for the time of
reinstall. The new system is installed with the same version of ceph.
I've made a backup of /var/lib/ceph directory (after stopping ceph
services obviously) and kept the osd's intact.
But after reinstall, when I try to start the daemons (the machine run
one monitor and three osd's) I get these messages in the logs:
monitor:
2015-12-09 13:15:28.223872 7f4ccd41b880 0 ceph version 0.94.5
(9764da52395923e0b32908d83a9f7304401fee43), process ceph-mon, pid 5800
2015-12-09 13:15:29.411448 7f4ccd41b880 -1 error opening mon data
directory at '/var/lib/ceph/mon/ceph-cf03': (22) Invalid argument
osds:
2015-12-09 13:11:50.480625 7fac03c7f880 0 ceph version 0.94.5
(9764da52395923e0b32908d83a9f7304401fee43), process ceph-osd, pid 3092
2015-12-09 13:11:50.508803 7fac03c7f880 0
filestore(/var/lib/ceph/osd/ceph-5) backend xfs (magic 0x58465342)
2015-12-09 13:11:50.640410 7fac03c7f880 0
genericfilestorebackend(/var/lib/ceph/osd/ceph-5) detect_features:
FIEMAP ioctl is supported and appears to work
2015-12-09 13:11:50.640429 7fac03c7f880 0
genericfilestorebackend(/var/lib/ceph/osd/ceph-5) detect_features:
FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-12-09 13:11:50.640890 7fac03c7f880 0
genericfilestorebackend(/var/lib/ceph/osd/ceph-5) detect_features:
syncfs(2) syscall fully supported (by glibc and kernel)
2015-12-09 13:11:50.646915 7fac03c7f880 0
xfsfilestorebackend(/var/lib/ceph/osd/ceph-5) detect_feature: extsize is
supported and kernel 3.10.0-229.20.1.el7.x86_64 >= 3.5
2015-12-09 13:11:51.171377 7fac03c7f880 -1
filestore(/var/lib/ceph/osd/ceph-5) Error initializing leveldb :
Corruption: 29 missing files; e.g.:
/var/lib/ceph/osd/ceph-5/current/omap/046388.sst
2015-12-09 13:11:51.171399 7fac03c7f880 -1 osd.5 0 OSD:init: unable to
mount object store
2015-12-09 13:11:51.171404 7fac03c7f880 -1 ** ERROR: osd init failed:
(1) Operation not permitted
can anyone help? I don't see any sst files on any of my other (working)
ceph nodes, the directories are fine with correct permissions..
I can readd this machine from scratch without data loss, but the
rebalancing/recovering will last a week (been there, done that), so I
was hoping I could start the osds with only some data out of date.
Is it possible? What can I do?
Cheers,
J
--
Jacek Jarosiewicz
Administrator Systemów Informatycznych
----------------------------------------------------------------------------------------
SUPERMEDIA Sp. z o.o. z siedzibą w Warszawie
ul. Senatorska 13/15, 00-075 Warszawa
Sąd Rejonowy dla m.st.Warszawy, XII Wydział Gospodarczy Krajowego
Rejestru Sądowego,
nr KRS 0000029537; kapitał zakładowy 42.756.000 zł
NIP: 957-05-49-503
Adres korespondencyjny: ul. Jubilerska 10, 04-190 Warszawa
----------------------------------------------------------------------------------------
SUPERMEDIA -> http://www.supermedia.pl
dostep do internetu - hosting - kolokacja - lacza - telefonia
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
