output is in the attached files
2013/7/17 Gregory Farnum <greg@xxxxxxxxxxx>:
> The maps in the OSDs only would have gotten there from the monitors.
> If a bad map somehow got distributed to the OSDs then cleaning it up
> is unfortunately going to take a lot of work without any well-defined
> processes.
> So if you could just do "ceph osd crush dump" and "ceph osd dump" and
> provide the output from those commands, we can look at what the map
> actually has and go from there.
> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
>
>
> On Tue, Jul 16, 2013 at 3:22 PM, Vladislav Gorbunov <vadikgo@xxxxxxxxx> wrote:
>> Gregory, thank for you help!
>> After all osd servers downed, i'am back rule set for the iscsi pool
>> back to default rule 0:
>> ceph osd pool set iscsi crush_ruleset 0
>> it does not help, all osd not started, except without data, with weight 0.
>> next i remove ruleset iscsi from crush map. It does not help too. And
>> after that i post crushmap to this mail list.
>> Is any method to extract crush map from downed osd server and inject
>> it to the mon server? from /var/lib/ceph/osd/ceph-2/current/omap
>> folder?
>>
>> 2013/7/17 Gregory Farnum <greg@xxxxxxxxxxx>:
>>> I notice that your first dump of the crush map didn't include rule #3.
>>> Are you sure you've injected it into the cluster? Try extracting it
>>> from the monitors and looking at that map directly, instead of a
>>> locally cached version.
>>> You mentioned some problem with OSDs being positioned wrong too, so
>>> you might look at "ceph osd tree" and look at the shape of the map.
>>> But it sounds to me like maybe there's a disconnect between what
>>> you've put into the cluster, and what you're looking at.
>>> -Greg
>>> Software Engineer #42 @ http://inktank.com | http://ceph.com
{ "devices": [
{ "id": 0,
"name": "device0"},
{ "id": 1,
"name": "device1"},
{ "id": 2,
"name": "osd.2"},
{ "id": 3,
"name": "device3"},
{ "id": 4,
"name": "osd.4"},
{ "id": 5,
"name": "osd.5"},
{ "id": 6,
"name": "osd.6"},
{ "id": 7,
"name": "osd.7"},
{ "id": 8,
"name": "osd.8"},
{ "id": 9,
"name": "osd.9"},
{ "id": 10,
"name": "osd.10"},
{ "id": 11,
"name": "osd.11"},
{ "id": 12,
"name": "osd.12"},
{ "id": 13,
"name": "osd.13"},
{ "id": 14,
"name": "osd.14"},
{ "id": 15,
"name": "osd.15"},
{ "id": 16,
"name": "osd.16"},
{ "id": 17,
"name": "osd.17"},
{ "id": 18,
"name": "osd.18"},
{ "id": 19,
"name": "osd.19"},
{ "id": 20,
"name": "osd.20"},
{ "id": 21,
"name": "osd.21"},
{ "id": 22,
"name": "osd.22"},
{ "id": 23,
"name": "osd.23"},
{ "id": 24,
"name": "osd.24"},
{ "id": 25,
"name": "osd.25"},
{ "id": 26,
"name": "osd.26"},
{ "id": 27,
"name": "osd.27"},
{ "id": 28,
"name": "osd.28"},
{ "id": 29,
"name": "osd.29"},
{ "id": 30,
"name": "osd.30"},
{ "id": 31,
"name": "osd.31"},
{ "id": 32,
"name": "osd.32"},
{ "id": 33,
"name": "osd.33"},
{ "id": 34,
"name": "osd.34"},
{ "id": 35,
"name": "osd.35"},
{ "id": 36,
"name": "osd.36"},
{ "id": 37,
"name": "osd.37"},
{ "id": 38,
"name": "osd.38"},
{ "id": 39,
"name": "osd.39"},
{ "id": 40,
"name": "osd.40"},
{ "id": 41,
"name": "osd.41"},
{ "id": 42,
"name": "osd.42"},
{ "id": 43,
"name": "osd.43"},
{ "id": 44,
"name": "osd.44"},
{ "id": 45,
"name": "osd.45"},
{ "id": 46,
"name": "osd.46"},
{ "id": 47,
"name": "osd.47"},
{ "id": 48,
"name": "osd.48"},
{ "id": 49,
"name": "osd.49"},
{ "id": 50,
"name": "osd.50"},
{ "id": 51,
"name": "osd.51"},
{ "id": 52,
"name": "osd.52"},
{ "id": 53,
"name": "osd.53"},
{ "id": 54,
"name": "osd.54"},
{ "id": 55,
"name": "osd.55"},
{ "id": 56,
"name": "osd.56"},
{ "id": 57,
"name": "osd.57"},
{ "id": 58,
"name": "device58"},
{ "id": 59,
"name": "device59"},
{ "id": 60,
"name": "device60"},
{ "id": 61,
"name": "device61"},
{ "id": 62,
"name": "osd.62"},
{ "id": 63,
"name": "osd.63"},
{ "id": 64,
"name": "osd.64"},
{ "id": 65,
"name": "osd.65"},
{ "id": 66,
"name": "osd.66"},
{ "id": 67,
"name": "osd.67"},
{ "id": 68,
"name": "osd.68"},
{ "id": 69,
"name": "osd.69"},
{ "id": 70,
"name": "osd.70"},
{ "id": 71,
"name": "osd.71"},
{ "id": 72,
"name": "osd.72"},
{ "id": 73,
"name": "osd.73"},
{ "id": 74,
"name": "osd.74"},
{ "id": 75,
"name": "osd.75"},
{ "id": 76,
"name": "osd.76"},
{ "id": 77,
"name": "osd.77"},
{ "id": 78,
"name": "osd.78"},
{ "id": 79,
"name": "osd.79"},
{ "id": 80,
"name": "osd.80"},
{ "id": 81,
"name": "osd.81"}],
"types": [
{ "type_id": 0,
"name": "osd"},
{ "type_id": 1,
"name": "host"},
{ "type_id": 2,
"name": "rack"},
{ "type_id": 3,
"name": "row"},
{ "type_id": 4,
"name": "room"},
{ "type_id": 5,
"name": "datacenter"},
{ "type_id": 6,
"name": "pool"}],
"buckets": [
{ "id": -1,
"name": "default",
"type_id": 6,
"type_name": "pool",
"weight": 4301783,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": -3,
"weight": 4301783,
"pos": 0}]},
{ "id": -2,
"name": "iscsi",
"type_id": 6,
"type_name": "pool",
"weight": 0,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": -4,
"weight": 0,
"pos": 0},
{ "id": -5,
"weight": 0,
"pos": 1}]},
{ "id": -3,
"name": "unknownrack",
"type_id": 2,
"type_name": "rack",
"weight": 4301783,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": -9,
"weight": 327680,
"pos": 0},
{ "id": -10,
"weight": 1048576,
"pos": 1},
{ "id": -11,
"weight": 1376256,
"pos": 2},
{ "id": -12,
"weight": 1048576,
"pos": 3},
{ "id": -13,
"weight": 500695,
"pos": 4}]},
{ "id": -4,
"name": "datacenter-cod",
"type_id": 5,
"type_name": "datacenter",
"weight": 0,
"alg": "straw",
"hash": "rjenkins1",
"items": []},
{ "id": -5,
"name": "datacenter-rcod",
"type_id": 5,
"type_name": "datacenter",
"weight": 0,
"alg": "straw",
"hash": "rjenkins1",
"items": []},
{ "id": -9,
"name": "gstore5",
"type_id": 1,
"type_name": "host",
"weight": 327680,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": 2,
"weight": 327680,
"pos": 0}]},
{ "id": -10,
"name": "cstore1",
"type_id": 1,
"type_name": "host",
"weight": 1048576,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": 5,
"weight": 65536,
"pos": 0},
{ "id": 6,
"weight": 65536,
"pos": 1},
{ "id": 7,
"weight": 65536,
"pos": 2},
{ "id": 8,
"weight": 65536,
"pos": 3},
{ "id": 9,
"weight": 65536,
"pos": 4},
{ "id": 10,
"weight": 65536,
"pos": 5},
{ "id": 11,
"weight": 65536,
"pos": 6},
{ "id": 12,
"weight": 65536,
"pos": 7},
{ "id": 13,
"weight": 65536,
"pos": 8},
{ "id": 14,
"weight": 65536,
"pos": 9},
{ "id": 4,
"weight": 65536,
"pos": 10},
{ "id": 47,
"weight": 65536,
"pos": 11},
{ "id": 48,
"weight": 65536,
"pos": 12},
{ "id": 49,
"weight": 65536,
"pos": 13},
{ "id": 50,
"weight": 65536,
"pos": 14},
{ "id": 51,
"weight": 65536,
"pos": 15}]},
{ "id": -11,
"name": "cstore2",
"type_id": 1,
"type_name": "host",
"weight": 1376256,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": 15,
"weight": 65536,
"pos": 0},
{ "id": 16,
"weight": 65536,
"pos": 1},
{ "id": 17,
"weight": 65536,
"pos": 2},
{ "id": 18,
"weight": 65536,
"pos": 3},
{ "id": 19,
"weight": 65536,
"pos": 4},
{ "id": 20,
"weight": 65536,
"pos": 5},
{ "id": 21,
"weight": 65536,
"pos": 6},
{ "id": 22,
"weight": 65536,
"pos": 7},
{ "id": 23,
"weight": 65536,
"pos": 8},
{ "id": 24,
"weight": 65536,
"pos": 9},
{ "id": 41,
"weight": 65536,
"pos": 10},
{ "id": 42,
"weight": 65536,
"pos": 11},
{ "id": 43,
"weight": 65536,
"pos": 12},
{ "id": 44,
"weight": 65536,
"pos": 13},
{ "id": 45,
"weight": 65536,
"pos": 14},
{ "id": 46,
"weight": 65536,
"pos": 15},
{ "id": 52,
"weight": 65536,
"pos": 16},
{ "id": 53,
"weight": 65536,
"pos": 17},
{ "id": 54,
"weight": 65536,
"pos": 18},
{ "id": 55,
"weight": 65536,
"pos": 19},
{ "id": 56,
"weight": 65536,
"pos": 20},
{ "id": 57,
"weight": 0,
"pos": 21}]},
{ "id": -12,
"name": "cstore3",
"type_id": 1,
"type_name": "host",
"weight": 1048576,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": 25,
"weight": 65536,
"pos": 0},
{ "id": 26,
"weight": 65536,
"pos": 1},
{ "id": 27,
"weight": 65536,
"pos": 2},
{ "id": 28,
"weight": 65536,
"pos": 3},
{ "id": 29,
"weight": 65536,
"pos": 4},
{ "id": 30,
"weight": 65536,
"pos": 5},
{ "id": 31,
"weight": 65536,
"pos": 6},
{ "id": 32,
"weight": 65536,
"pos": 7},
{ "id": 33,
"weight": 65536,
"pos": 8},
{ "id": 34,
"weight": 65536,
"pos": 9},
{ "id": 35,
"weight": 65536,
"pos": 10},
{ "id": 36,
"weight": 65536,
"pos": 11},
{ "id": 37,
"weight": 65536,
"pos": 12},
{ "id": 38,
"weight": 65536,
"pos": 13},
{ "id": 39,
"weight": 65536,
"pos": 14},
{ "id": 40,
"weight": 65536,
"pos": 15}]},
{ "id": -13,
"name": "cstore4",
"type_id": 1,
"type_name": "host",
"weight": 500680,
"alg": "straw",
"hash": "rjenkins1",
"items": [
{ "id": 62,
"weight": 36044,
"pos": 0},
{ "id": 63,
"weight": 36044,
"pos": 1},
{ "id": 64,
"weight": 36044,
"pos": 2},
{ "id": 65,
"weight": 36044,
"pos": 3},
{ "id": 66,
"weight": 36044,
"pos": 4},
{ "id": 67,
"weight": 36044,
"pos": 5},
{ "id": 68,
"weight": 36044,
"pos": 6},
{ "id": 69,
"weight": 36044,
"pos": 7},
{ "id": 70,
"weight": 17694,
"pos": 8},
{ "id": 71,
"weight": 17694,
"pos": 9},
{ "id": 72,
"weight": 17694,
"pos": 10},
{ "id": 73,
"weight": 17694,
"pos": 11},
{ "id": 74,
"weight": 17694,
"pos": 12},
{ "id": 75,
"weight": 17694,
"pos": 13},
{ "id": 76,
"weight": 17694,
"pos": 14},
{ "id": 77,
"weight": 17694,
"pos": 15},
{ "id": 78,
"weight": 17694,
"pos": 16},
{ "id": 79,
"weight": 17694,
"pos": 17},
{ "id": 80,
"weight": 17694,
"pos": 18},
{ "id": 81,
"weight": 17694,
"pos": 19}]}],
"rules": [
{ "rule_id": 0,
"rule_name": "data",
"ruleset": 0,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{ "op": "take",
"item": -1},
{ "op": "chooseleaf_firstn",
"num": 0,
"type": "host"},
{ "op": "emit"}]},
{ "rule_id": 1,
"rule_name": "metadata",
"ruleset": 1,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{ "op": "take",
"item": -1},
{ "op": "chooseleaf_firstn",
"num": 0,
"type": "host"},
{ "op": "emit"}]},
{ "rule_id": 2,
"rule_name": "rbd",
"ruleset": 2,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{ "op": "take",
"item": -1},
{ "op": "chooseleaf_firstn",
"num": 0,
"type": "host"},
{ "op": "emit"}]},
{ "rule_id": 3,
"rule_name": "iscsi",
"ruleset": 3,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{ "op": "take",
"item": -2},
{ "op": "chooseleaf_firstn",
"num": 0,
"type": "datacenter"},
{ "op": "emit"}]}]}
epoch 78147
fsid 4f574056-e0db-4e19-88fd-cd6ef3cf2410
created 2012-10-09 16:09:36.414320
modified 2013-07-17 10:55:38.957906
flags
pool 0 'data' rep size 2 min_size 1 crush_ruleset 2 object_hash rjenkins pg_num 128 pgp_num 128 last_change 78129 owner 0 crash_replay_interval 45
pool 1 'metadata' rep size 2 min_size 1 crush_ruleset 1 object_hash rjenkins pg_num 128 pgp_num 128 last_change 78142 owner 0
pool 10 'iscsi-old' rep size 2 min_size 0 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 78141 owner 0
pool 11 'rados' rep size 2 min_size 0 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 78137 owner 0
removed_snaps [1~45e,460~df3,1254~2a2]
pool 12 'proxmox' rep size 2 min_size 0 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 78138 owner 0
removed_snaps [1~9]
pool 13 'rbd' rep size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 512 pgp_num 512 last_change 78143 owner 0
pool 14 'ceph' rep size 2 min_size 0 crush_ruleset 0 object_hash rjenkins pg_num 4192 pgp_num 4192 last_change 78139 owner 0
removed_snaps [1~9]
pool 16 'iscsi' rep size 2 min_size 0 crush_ruleset 0 object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 78140 owner 0 flags 1
max_osd 107
osd.2 down in weight 1 up_from 76083 up_thru 77388 down_at 77421 last_clean_interval [25918,76081) 10.166.10.27:6801/12007 10.166.10.27:6800/12007 10.166.10.27:6802/12007 exists 27875851-3a96-485a-98fc-4da2a2bac04c
osd.4 down out weight 0 up_from 76114 up_thru 77382 down_at 77539 last_clean_interval [75833,76085) 10.166.10.28:6851/18513 10.166.10.28:6816/18513 10.166.10.28:6822/18513 autoout,exists fdfa0c39-70fa-4884-b310-98a8e41010b7
osd.5 down out weight 0 up_from 76098 up_thru 77392 down_at 77457 last_clean_interval [75854,76081) 10.166.10.28:6848/26878 10.166.10.28:6805/26878 10.166.10.28:6810/26878 autoout,exists f2fb7f41-cd70-4459-99f8-565f738e3497
osd.6 down out weight 0 up_from 76133 up_thru 77407 down_at 77548 last_clean_interval [75871,76132) 10.166.10.28:6806/2775 10.166.10.28:6832/2775 10.166.10.28:6833/2775 autoout,exists dbcebfe2-4da8-4c69-8076-f62a848d5b0b
osd.7 down out weight 0 up_from 76114 up_thru 77382 down_at 77552 last_clean_interval [75886,76110) 10.166.10.28:6891/29921 10.166.10.28:6823/29921 10.166.10.28:6824/29921 autoout,exists 6fef6eed-30ab-4c79-96ff-2c6c1711bcbb
osd.8 down out weight 0 up_from 76114 up_thru 77407 down_at 77449 last_clean_interval [75886,76085) 10.166.10.28:6845/15132 10.166.10.28:6813/15132 10.166.10.28:6815/15132 autoout,exists f84ab096-8d98-45b8-abab-4df02521a261
osd.9 down out weight 0 up_from 76058 up_thru 77409 down_at 77541 last_clean_interval [75888,76056) 10.166.10.28:6896/507 10.166.10.28:6838/507 10.166.10.28:6839/507 autoout,exists 85e77b2d-9f1d-437e-881c-77353135a734
osd.10 down out weight 0 up_from 75825 up_thru 77398 down_at 77566 last_clean_interval [60521,75806) 10.166.10.28:6854/1426 10.166.10.28:6855/1426 10.166.10.28:6856/1426 autoout,exists 5ce96f6b-557c-4c61-abd3-8a2696142a1d
osd.11 down out weight 0 up_from 76117 up_thru 77402 down_at 77568 last_clean_interval [75824,76116) 10.166.10.28:6802/4550 10.166.10.28:6827/4550 10.166.10.28:6828/4550 autoout,exists 54538f2d-a4c7-4f1e-ae83-a4619a199eb6
osd.12 down out weight 0 up_from 75827 up_thru 77406 down_at 77445 last_clean_interval [60524,75810) 10.166.10.28:6811/11755 10.166.10.28:6817/11755 10.166.10.28:6842/11755 autoout,exists dd892d01-d193-47d0-8cc8-b93f91d4df38
osd.13 down out weight 0 up_from 75834 up_thru 77407 down_at 77570 last_clean_interval [60529,75813) 10.166.10.28:6863/26101 10.166.10.28:6864/26101 10.166.10.28:6865/26101 autoout,exists 2e0380e9-742d-4814-8c7f-1180f056596c
osd.14 down out weight 0 up_from 76042 up_thru 77394 down_at 77572 last_clean_interval [75849,76040) 10.166.10.28:6830/8056 10.166.10.28:6809/8056 10.166.10.28:6814/8056 autoout,exists b82c8d91-46c4-49f3-aad6-9a7a8454a7ce
osd.15 down out weight 0 up_from 76252 up_thru 77384 down_at 77435 last_clean_interval [76098,76244) 10.166.10.29:6800/24661 10.166.10.29:6807/24661 10.166.10.29:6808/24661 autoout,exists 2b6190ba-6d01-4bce-b5bb-4f2023e57fe7
osd.16 down out weight 0 up_from 76256 up_thru 77382 down_at 77439 last_clean_interval [76114,76247) 10.166.10.29:6810/25105 10.166.10.29:6811/25105 10.166.10.29:6812/25105 autoout,exists 34052f8f-1e25-49c2-907d-811bf4b0419a
osd.17 down out weight 0 up_from 76259 up_thru 77382 down_at 77439 last_clean_interval [76115,76250) 10.166.10.29:6813/25489 10.166.10.29:6815/25489 10.166.10.29:6816/25489 autoout,exists f536a65e-65cc-4a95-993f-9a9e08cd8b26
osd.18 down out weight 0 up_from 76258 up_thru 77382 down_at 77416 last_clean_interval [76126,76250) 10.166.10.29:6819/14614 10.166.10.29:6820/14614 10.166.10.29:6822/14614 autoout,exists 4741c01f-09b8-4bbd-bdac-3f402b632120
osd.19 down out weight 0 up_from 76263 up_thru 77384 down_at 77416 last_clean_interval [76133,76252) 10.166.10.29:6823/5583 10.166.10.29:6825/5583 10.166.10.29:6826/5583 autoout,exists 4ee885ea-8030-4452-8b8f-055541c4f9e8
osd.20 down out weight 0 up_from 76265 up_thru 77384 down_at 77416 last_clean_interval [76131,76254) 10.166.10.29:6828/6760 10.166.10.29:6829/6760 10.166.10.29:6830/6760 autoout,exists cd122be9-0d44-455b-a7e7-8da53dbed4d5
osd.21 down out weight 0 up_from 76268 up_thru 77386 down_at 77439 last_clean_interval [76137,76258) 10.166.10.29:6835/15564 10.166.10.29:6837/15564 10.166.10.29:6840/15564 autoout,exists d68ced55-47c7-4347-83d9-91bfb943e22b
osd.22 down out weight 0 up_from 76269 up_thru 77382 down_at 77417 last_clean_interval [76139,76262) 10.166.10.29:6841/19849 10.166.10.29:6842/19849 10.166.10.29:6844/19849 autoout,exists 2e69025a-31a9-4d2e-aa3d-09191571325c
osd.23 down out weight 0 up_from 76272 up_thru 77390 down_at 77544 last_clean_interval [76140,76264) 10.166.10.29:6846/2767 10.166.10.29:6848/2767 10.166.10.29:6851/2767 autoout,exists 4aeb7022-dbba-413e-94a6-0ac877d3e382
osd.24 down out weight 0 up_from 76276 up_thru 77382 down_at 77417 last_clean_interval [76146,76266) 10.166.10.29:6852/29950 10.166.10.29:6854/29950 10.166.10.29:6855/29950 autoout,exists af5be086-17dd-4760-bb10-891dc01c0ae5
osd.25 down out weight 0 up_from 76211 up_thru 77382 down_at 77576 last_clean_interval [75990,76194) 10.166.10.30:6801/26966 10.166.10.30:6802/26966 10.166.10.30:6803/26966 autoout,exists c70752b9-927b-4e58-8d32-f99aaeab2d8c
osd.26 down out weight 0 up_from 76213 up_thru 77398 down_at 77537 last_clean_interval [75993,76198) 10.166.10.30:6805/27363 10.166.10.30:6806/27363 10.166.10.30:6807/27363 autoout,exists eaafb204-da6b-4db7-a543-cba0cd845d20
osd.27 down out weight 0 up_from 76216 up_thru 77382 down_at 77539 last_clean_interval [75997,76202) 10.166.10.30:6809/27723 10.166.10.30:6810/27723 10.166.10.30:6811/27723 autoout,exists 7202ff23-b1aa-4198-b202-19df74fa87f2
osd.28 down out weight 0 up_from 76220 up_thru 77399 down_at 77453 last_clean_interval [75998,76205) 10.166.10.30:6812/28455 10.166.10.30:6813/28455 10.166.10.30:6814/28455 autoout,exists d6c8899c-8414-4e84-a72f-cceccb6e74fd
osd.29 down out weight 0 up_from 76225 up_thru 77406 down_at 77578 last_clean_interval [76005,76207) 10.166.10.30:6816/28806 10.166.10.30:6817/28806 10.166.10.30:6821/28806 autoout,exists 18fb7223-eda5-423a-a5e7-75eef928e7e0
osd.30 down out weight 0 up_from 76222 up_thru 77410 down_at 77579 last_clean_interval [76003,76209) 10.166.10.30:6824/16704 10.166.10.30:6826/16704 10.166.10.30:6827/16704 autoout,exists c37b8548-c3ff-4bb3-812d-64135b7dfd49
osd.31 down out weight 0 up_from 76225 up_thru 77411 down_at 77580 last_clean_interval [76005,76211) 10.166.10.30:6828/10775 10.166.10.30:6834/10775 10.166.10.30:6835/10775 autoout,exists abe7e8e5-e121-4b90-b6d2-08213cfea869
osd.32 down out weight 0 up_from 76227 up_thru 77383 down_at 77581 last_clean_interval [76008,76213) 10.166.10.30:6825/11218 10.166.10.30:6830/11218 10.166.10.30:6836/11218 autoout,exists 1e1f7824-f0af-406f-82c3-3db926167337
osd.33 down out weight 0 up_from 76227 up_thru 77382 down_at 77583 last_clean_interval [76010,76215) 10.166.10.30:6846/30526 10.166.10.30:6849/30526 10.166.10.30:6850/30526 autoout,exists b9cb4d54-2082-4264-b4b6-9e093b6cd131
osd.34 down out weight 0 up_from 76234 up_thru 77382 down_at 77584 last_clean_interval [76014,76218) 10.166.10.30:6851/28972 10.166.10.30:6852/28972 10.166.10.30:6853/28972 autoout,exists 7b8c6f98-9841-4969-a9d8-413228c2ca40
osd.35 down out weight 0 up_from 76238 up_thru 77382 down_at 77585 last_clean_interval [76016,76222) 10.166.10.30:6854/9752 10.166.10.30:6855/9752 10.166.10.30:6856/9752 autoout,exists d58c2852-00ef-42b3-b659-10b4d6c081d0
osd.36 down out weight 0 up_from 76238 up_thru 77413 down_at 77445 last_clean_interval [76018,76225) 10.166.10.30:6857/27907 10.166.10.30:6858/27907 10.166.10.30:6860/27907 autoout,exists ffbab99a-df00-41c7-85a2-ce3ce90ccdba
osd.37 down out weight 0 up_from 76236 up_thru 77382 down_at 77586 last_clean_interval [76016,76227) 10.166.10.30:6818/28608 10.166.10.30:6819/28608 10.166.10.30:6861/28608 autoout,exists 6f5365b2-16c0-4256-9722-6b249ab4ed7c
osd.38 down out weight 0 up_from 76239 up_thru 77382 down_at 77455 last_clean_interval [76020,76229) 10.166.10.30:6843/29158 10.166.10.30:6862/29158 10.166.10.30:6868/29158 autoout,exists 1e724f00-41ca-4bb1-a0e5-8dda1b975670
osd.39 down out weight 0 up_from 76241 up_thru 77389 down_at 77587 last_clean_interval [76023,76231) 10.166.10.30:6837/32010 10.166.10.30:6871/32010 10.166.10.30:6872/32010 autoout,exists 720ea680-5b20-4cf8-94a6-9539e8dad71c
osd.40 down out weight 0 up_from 76243 up_thru 77383 down_at 77539 last_clean_interval [76026,76233) 10.166.10.30:6800/19726 10.166.10.30:6831/19726 10.166.10.30:6832/19726 autoout,exists 2775d92d-bb0f-4810-bcc1-cc887375eb52
osd.41 down out weight 0 up_from 76280 up_thru 77382 down_at 77417 last_clean_interval [76148,76269) 10.166.10.29:6856/18023 10.166.10.29:6858/18023 10.166.10.29:6860/18023 autoout,exists 1c5fee0f-e284-46ed-beed-0e215079bc33
osd.42 down out weight 0 up_from 76281 up_thru 77382 down_at 77441 last_clean_interval [76150,76271) 10.166.10.29:6861/32427 10.166.10.29:6862/32427 10.166.10.29:6863/32427 autoout,exists ed315085-0978-4393-b4d2-a1f9b3bd7220
osd.43 down out weight 0 up_from 76284 up_thru 77382 down_at 77439 last_clean_interval [76153,76273) 10.166.10.29:6864/566 10.166.10.29:6865/566 10.166.10.29:6866/566 autoout,exists 8f3a69f4-d155-4f66-a40d-0f6bca29b96b
osd.44 down out weight 0 up_from 76286 up_thru 77407 down_at 77439 last_clean_interval [76157,76275) 10.166.10.29:6867/19311 10.166.10.29:6868/19311 10.166.10.29:6870/19311 autoout,exists 9932776c-8a06-4f74-825e-9d4d478bfd82
osd.45 down out weight 0 up_from 76288 up_thru 77382 down_at 77417 last_clean_interval [76157,76277) 10.166.10.29:6872/27657 10.166.10.29:6874/27657 10.166.10.29:6875/27657 autoout,exists bf217359-10be-4b97-bf7a-2b84a5a766c9
osd.46 down out weight 0 up_from 76289 up_thru 77403 down_at 77439 last_clean_interval [76159,76279) 10.166.10.29:6877/20081 10.166.10.29:6879/20081 10.166.10.29:6880/20081 autoout,exists bf479e56-bd1c-4b4b-856b-4f7272ad4333
osd.47 down out weight 0 up_from 76123 up_thru 77382 down_at 77451 last_clean_interval [75841,76122) 10.166.10.28:6812/27273 10.166.10.28:6829/27273 10.166.10.28:6831/27273 autoout,exists 004d5377-f32a-4bbe-8ccb-bf1e1980244c
osd.48 down out weight 0 up_from 76115 up_thru 77382 down_at 77527 last_clean_interval [75844,76114) 10.166.10.28:6878/5806 10.166.10.28:6825/5806 10.166.10.28:6826/5806 autoout,exists 0bbbe5e0-2848-4c20-90c9-d0d8f98484e7
osd.49 down out weight 0 up_from 75852 up_thru 77382 down_at 77574 last_clean_interval [60555,75824) 10.166.10.28:6881/21893 10.166.10.28:6882/21893 10.166.10.28:6883/21893 autoout,exists 6734b11d-f68e-4e2f-bc28-24df2209a3d4
osd.50 down out weight 0 up_from 75857 up_thru 77382 down_at 77499 last_clean_interval [60570,75835) 10.166.10.28:6870/22093 10.166.10.28:6886/22093 10.166.10.28:6887/22093 autoout,exists 80422e6f-1f37-47af-8073-9f85f67d1831
osd.51 down out weight 0 up_from 75864 up_thru 77382 down_at 77449 last_clean_interval [60562,75837) 10.166.10.28:6874/21345 10.166.10.28:6888/21345 10.166.10.28:6889/21345 autoout,exists 124441fc-607d-4276-9517-a007d7b78b34
osd.52 down out weight 0 up_from 76292 up_thru 77384 down_at 77417 last_clean_interval [76161,76281) 10.166.10.29:6814/20995 10.166.10.29:6817/20995 10.166.10.29:6818/20995 autoout,exists 7765370e-2340-4fb1-9268-8e052d338646
osd.53 down out weight 0 up_from 76294 up_thru 77382 down_at 77439 last_clean_interval [76163,76283) 10.166.10.29:6881/3495 10.166.10.29:6882/3495 10.166.10.29:6884/3495 autoout,exists 67f43702-78dc-4107-a0b8-c798df4d3716
osd.54 down out weight 0 up_from 76296 up_thru 77412 down_at 77417 last_clean_interval [76162,76285) 10.166.10.29:6885/27866 10.166.10.29:6886/27866 10.166.10.29:6888/27866 autoout,exists 72e9c539-9559-4a66-aedd-0bf74e7c3a59
osd.55 down out weight 0 up_from 76298 up_thru 77409 down_at 77417 last_clean_interval [76165,76287) 10.166.10.29:6889/17850 10.166.10.29:6891/17850 10.166.10.29:6892/17850 autoout,exists 1eae2c35-a087-444e-b4c7-87805b7237ec
osd.56 up in weight 1 up_from 78114 up_thru 78123 down_at 78113 last_clean_interval [78087,78110) 10.166.10.29:6860/5075 10.166.10.29:6861/5075 10.166.10.29:6862/5075 exists,up f2393dc2-2185-4711-a928-0c839576f815
osd.57 up in weight 1 up_from 78146 up_thru 78113 down_at 78145 last_clean_interval [78112,78128) 10.166.10.29:6800/3908 10.166.10.29:6801/3908 10.166.10.29:6802/3908 exists,up 0f4e17e7-17e2-420d-9ed4-6c8413e229b4
osd.62 down out weight 0 up_from 76088 up_thru 77382 down_at 77449 last_clean_interval [31679,76084) 10.166.10.19:6800/2876 10.166.10.19:6809/2876 10.166.10.19:6810/2876 autoout,exists e0582d43-eb22-4db6-a121-ec88c567ae77
osd.63 down out weight 0 up_from 76190 up_thru 77382 down_at 77419 last_clean_interval [53760,76181) 10.166.10.19:6869/20127 10.166.10.19:6870/20127 10.166.10.19:6871/20127 autoout,exists b3a3d4e6-2466-4ba5-bfba-76a701d424c8
osd.64 down out weight 0 up_from 76193 up_thru 77382 down_at 77480 last_clean_interval [53762,76181) 10.166.10.19:6865/19901 10.166.10.19:6866/19901 10.166.10.19:6868/19901 autoout,exists 033fa952-61da-47ac-bb1c-fddd1889ca60
osd.65 down out weight 0 up_from 76192 up_thru 77382 down_at 77417 last_clean_interval [53758,76181) 10.166.10.19:6861/19860 10.166.10.19:6862/19860 10.166.10.19:6864/19860 autoout,exists b241a0f1-33d0-48a1-8293-0bbb9cc88942
osd.66 down out weight 0 up_from 76190 up_thru 77382 down_at 77443 last_clean_interval [53761,76181) 10.166.10.19:6857/19816 10.166.10.19:6859/19816 10.166.10.19:6860/19816 autoout,exists 9d507a8c-1a7c-4f00-bc6a-bdbc94f71627
osd.67 down out weight 0 up_from 76189 up_thru 77382 down_at 77419 last_clean_interval [53755,76181) 10.166.10.19:6854/19748 10.166.10.19:6855/19748 10.166.10.19:6856/19748 autoout,exists 6411e622-ecd8-4216-b6ff-5b07133a1cb7
osd.68 down out weight 0 up_from 76190 up_thru 77382 down_at 77421 last_clean_interval [53754,76181) 10.166.10.19:6851/19661 10.166.10.19:6852/19661 10.166.10.19:6853/19661 autoout,exists b089591d-a409-4a53-9086-3cc7b598c429
osd.69 down out weight 0 up_from 76193 up_thru 77382 down_at 77439 last_clean_interval [53763,76181) 10.166.10.19:6848/19592 10.166.10.19:6849/19592 10.166.10.19:6850/19592 autoout,exists 294cc19c-66f9-4eee-9c7e-e08c6c0751d1
osd.70 down out weight 0 up_from 76186 up_thru 77382 down_at 77525 last_clean_interval [53751,76181) 10.166.10.19:6845/19502 10.166.10.19:6846/19502 10.166.10.19:6847/19502 autoout,exists 8243bcaa-06b1-4533-8969-020b946bd6b6
osd.71 down out weight 0 up_from 76185 up_thru 77382 down_at 77437 last_clean_interval [53750,76181) 10.166.10.19:6842/19428 10.166.10.19:6843/19428 10.166.10.19:6844/19428 autoout,exists c49bdefe-1152-47c2-89da-9da97348f105
osd.72 down out weight 0 up_from 76187 up_thru 77382 down_at 77443 last_clean_interval [53751,76181) 10.166.10.19:6838/19355 10.166.10.19:6839/19355 10.166.10.19:6841/19355 autoout,exists 57cdbd50-f5b1-4bb1-9f6e-9e2c87ed423c
osd.73 down out weight 0 up_from 76185 up_thru 77382 down_at 77440 last_clean_interval [53750,76181) 10.166.10.19:6835/19311 10.166.10.19:6836/19311 10.166.10.19:6837/19311 autoout,exists ee420806-db99-465f-a6af-800521ab015a
osd.74 down out weight 0 up_from 76186 up_thru 77382 down_at 77435 last_clean_interval [53751,76181) 10.166.10.19:6830/19244 10.166.10.19:6832/19244 10.166.10.19:6833/19244 autoout,exists 188a6c63-327f-4c16-aca1-dcc2f30a3bc5
osd.75 down out weight 0 up_from 76186 up_thru 77382 down_at 77523 last_clean_interval [53752,76181) 10.166.10.19:6826/19167 10.166.10.19:6828/19167 10.166.10.19:6829/19167 autoout,exists eca76040-df66-4357-86ec-127516e21342
osd.76 down out weight 0 up_from 76186 up_thru 77382 down_at 77417 last_clean_interval [53751,76181) 10.166.10.19:6822/19111 10.166.10.19:6823/19111 10.166.10.19:6825/19111 autoout,exists a3cc3b8d-6cf4-4871-8cdc-8542c2fc5b1c
osd.77 down out weight 0 up_from 76185 up_thru 77382 down_at 77494 last_clean_interval [53750,76181) 10.166.10.19:6819/19049 10.166.10.19:6820/19049 10.166.10.19:6821/19049 autoout,exists 52ba31f1-1dfe-47e9-8865-5ef58a84360e
osd.78 down out weight 0 up_from 76186 up_thru 77382 down_at 77492 last_clean_interval [53751,76181) 10.166.10.19:6816/18967 10.166.10.19:6817/18967 10.166.10.19:6818/18967 autoout,exists 02f777fb-64e2-4a00-b69d-6385e977e509
osd.79 down out weight 0 up_from 76185 up_thru 77382 down_at 77417 last_clean_interval [53750,76181) 10.166.10.19:6813/18929 10.166.10.19:6814/18929 10.166.10.19:6815/18929 autoout,exists c039aeb6-4826-4be5-a5da-142336153a5f
osd.80 down out weight 0 up_from 76185 up_thru 77382 down_at 77417 last_clean_interval [53751,76181) 10.166.10.19:6806/18883 10.166.10.19:6807/18883 10.166.10.19:6812/18883 autoout,exists f3f0b1bb-7a7d-483d-8441-d9e3f1283cda
osd.81 down out weight 0 up_from 76184 up_thru 77382 down_at 77417 last_clean_interval [53750,76181) 10.166.10.19:6802/18857 10.166.10.19:6803/18857 10.166.10.19:6805/18857 autoout,exists 0fa3f365-f300-4cd6-b22f-d45e187140a3
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
