|
Hello, I am setting up a 9 node ceph cluster. For legacy reasons I'm using Ceph giant (0.87) on Fedora 21. Each OSD node has 4x4TB SATA drives with journals on a separate SSD. The server is an HP XL190 Gen 9 with latest firmware. The issue I'm seeing is that only 2 drives get detected and mounted on almost all the nodes. During the initial creation all drives were created and mounted but now only 2 show up. Usually a partprobe forces the drives to come online but in this case it doesnt. On a reboot a different set of OSDs will get detected. For e.g. of osds 0 and 1 are up, on a reboot osds 0 and 3 will be detected. On another reboot osds 1 and 2 may come up.
$ ceph osd tree # id weight type name up/down reweight
-1 131 root default
-2 14.56 host
xxx-a5-34
0 3.64 osd.0 up 1
1 3.64 osd.1 down 0
2 3.64 osd.2 up 1
3 3.64 osd.3 down 0
-3 14.56 host
xxx-a5-36
4 3.64 osd.4 down 0
5 3.64 osd.5 down 0
6 3.64 osd.6 up 1
7 3.64 osd.7 up 1
-4 14.56 host
xxx-a5-37
8 3.64 osd.8 down 0
9 3.64 osd.9 up 1
10 3.64 osd.10 up 1
11 3.64 osd.11 down 0
-5 14.56 host
xxx-b5-34
12 3.64 osd.12 up 1
13 3.64 osd.13 down 0
14 3.64 osd.14 up 1
15 3.64 osd.15 down 0
-6 14.56 host
xxx-b5-36
16 3.64 osd.16 up 1
17 3.64 osd.17 up 1
18 3.64 osd.18 down 0
19 3.64 osd.19 down 0
-7 14.56 host
xxx-b5-37
20 3.64 osd.20 up 1
21 3.64 osd.21 up 1
22 3.64 osd.22 down 0
23 3.64 osd.23 down 0
-8 14.56 host
xxx-c5-34
24 3.64 osd.24 up 1
25 3.64 osd.25 up 1
26 3.64 osd.26 up 1
27 3.64 osd.27 up 1
-9 14.56 host
xxx-c5-36
28 3.64 osd.28 down 0
29 3.64 osd.29 up 1
30 3.64 osd.30 down 0
31 3.64 osd.31 up 1
-10 14.56 host
xxx-c5-37
32 3.64 osd.32 up 1
33 3.64 osd.33 up 1
34 3.64 osd.34 up 1
35 3.64 osd.35 up 1
Anyone seen this problem before and know what the issue could be?
Thanks
|
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
