On Thu, Mar 26, 2015 at 4:40 PM, Gregory Farnum <greg@xxxxxxxxxxx> wrote:
Has the OSD actually been detected as down yet?
I believe it has, however I can't directly check because "ceph health" starts to hang when I down the second node.
You'll also need to set that min size on your existing pools ("ceph
osd pool <pool> set min_size 1" or similar) to change their behavior;
the config option only takes effect for newly-created pools. (Thus the
"default".)
I've done this, however the behavior is the same:
$ for f in `ceph osd lspools | sed 's/[0-9]//g' | sed 's/,//g'`; do ceph osd pool set $f min_size 1; done
set pool 0 min_size to 1
set pool 1 min_size to 1
set pool 2 min_size to 1
set pool 3 min_size to 1
set pool 4 min_size to 1
set pool 5 min_size to 1
set pool 6 min_size to 1
set pool 7 min_size to 1
$ ceph -w
cluster db460aa2-5129-4aaa-8b2e-43eac727124e
health HEALTH_WARN 1 mons down, quorum 0,1 ceph-node-1,ceph-node-2
monmap e3: 3 mons at {ceph-node-1=192.168.122.121:6789/0,ceph-node-2=192.168.122.131:6789/0,ceph-node-3=192.168.122.141:6789/0}, election epoch 194, quorum 0,1 ceph-node-1,ceph-node-2
mdsmap e94: 1/1/1 up {0=ceph-node-1=up:active}
osdmap e362: 3 osds: 2 up, 2 in
pgmap v5913: 840 pgs, 8 pools, 7441 MB data, 994 objects
25329 MB used, 12649 MB / 40059 MB avail
840 active+clean
2015-03-26 17:23:56.009938 mon.0 [INF] pgmap v5913: 840 pgs: 840 active+clean; 7441 MB data, 25329 MB used, 12649 MB / 40059 MB avail
2015-03-26 17:25:51.042802 mon.0 [INF] pgmap v5914: 840 pgs: 840 active+clean; 7441 MB data, 25329 MB used, 12649 MB / 40059 MB avail; 0 B/s rd, 260 kB/s wr, 13 op/s
2015-03-26 17:25:56.046491 mon.0 [INF] pgmap v5915: 840 pgs: 840 active+clean; 7441 MB data, 25333 MB used, 12645 MB / 40059 MB avail; 0 B/s rd, 943 kB/s wr, 38 op/s
2015-03-26 17:26:01.058167 mon.0 [INF] pgmap v5916: 840 pgs: 840 active+clean; 7441 MB data, 25335 MB used, 12643 MB / 40059 MB avail; 0 B/s rd, 10699 kB/s wr, 621 op/s
<this is where i kill the second OSD>
2015-03-26 17:26:26.778461 7f4ebeffd700 0 monclient: hunting for new mon
2015-03-26 17:26:30.701099 7f4ec45f5700 0 -- 192.168.122.111:0/1007741 >> 192.168.122.141:6789/0 pipe(0x7f4ec0023200 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f4ec0023490).fault
2015-03-26 17:26:42.701154 7f4ec44f4700 0 -- 192.168.122.111:0/1007741 >> 192.168.122.131:6789/0 pipe(0x7f4ec00251b0 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f4ec0025440).fault
And all writes block until I bring back an OSD.
Lee
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
