Hi guys,
I have stumbled over an error that I cannot solve after more than 2 days trying. Now I turn to you in hope to get some help! :)
First of all, I am totally new to Ceph and this is just a test setup for now. I have followed the instructions on the ceph.com homepage as far as I understand.
I am about to set up a Ceph storage cluster on three different nodes (running Ubuntu server 20.04). I am using Ceph Octopus.
On these 3 nodes I have installed 3 monitors, 3 managers and 12 OSDs (I do know that OSDs and Monitors on the same machines are not recommended but for this test setup I do it anyway).
So far is everything working well for me. However, when I try to create an RBD pool it fails with some strange (at least strange for me) error message:
2020-10-08T07:48:36.601+0000 7f8b9b7fe700 -1 librbd::image::GetMetadataRequest: 0x564279b9d670 handle_metadata_list: failed to retrieve image metadata: (1) Operation not permitted
2020-10-08T07:48:36.601+0000 7f8bb6af8380 -1 librbd::PoolMetadata: list: failed listing metadata: (1) Operation not permitted
2020-10-08T07:48:36.601+0000 7f8bb6af8380 -1 librbd::Config: apply_pool_overrides: failed to read pool config overrides: (1) Operation not permitted
2020-10-08T07:48:36.601+0000 7f8b9b7fe700 -1 librbd::image::ValidatePoolRequest: handle_read_rbd_info: failed to read RBD info: (1) Operation not permitted
rbd: pool already registered to a different application.
I run the commands as the Linux root user and as the Ceph user client.admin (I have turned off apparmor and other hardening things as well). The chep user client.admin has the following setup in its keyring:
[client.admin]
key = .....
caps mds = "allow *"
caps mgr = "allow *"
caps mon = "allow *"
For more information regarding permissions in the system see the attached file permissions.txt that contains the output of the ceph auth list command.
I have also attached a file (report.txt) containing the output from the ceph report command.
(NOTE: I have removed some information from the files, i.e. keys, fsid, fingerprints, uuid, and replaced them with "..." instead)
Can anyone please help me understand why I get this error and what I need to do in order to solve it?
Thanks in advanced!
Best regards,
Fredrik
osd.0
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.1
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.10
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.11
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.2
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.3
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.4
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.5
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.6
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.7
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.8
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.9
key: ...
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
client.admin
key: ...
caps: [mds] allow *
caps: [mgr] allow *
caps: [mon] allow *
client.bootstrap-mds
key: ...
caps: [mon] allow profile bootstrap-mds
client.bootstrap-mgr
key: ...
caps: [mon] allow profile bootstrap-mgr
client.bootstrap-osd
key: ...
caps: [mgr] allow r
caps: [mon] profile bootstrap-osd
client.bootstrap-rbd
key: ...
caps: [mon] allow profile bootstrap-rbd
client.bootstrap-rbd-mirror
key: ...
caps: [mon] allow profile bootstrap-rbd-mirror
client.bootstrap-rgw
key: ...
caps: [mon] allow profile bootstrap-rgw
client.fredrik
key: ...
caps: [mgr] profile rbd pool=fredrik-pool-rbd
caps: [mon] profile rbd
caps: [osd] profile rbd pool=fredrik-pool-rbd
mgr.fredrik-storage-1
key: ...
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
mgr.fredrik-storage-2
key: ...
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
mgr.fredrik-storage-3
key: ...
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
{
"cluster_fingerprint": "...",
"version": "15.2.5",
"commit": "2c93eff00150f0cc5f106a559557a58d3d7b6f1f",
"timestamp": "2020-10-08T07:38:39.192589+0000",
"tag": "",
"health": {
"status": "HEALTH_OK",
"checks": {},
"mutes": []
},
"monmap_first_committed": 1,
"monmap_last_committed": 4,
"monmap": {
"epoch": 4,
"fsid": "...",
"modified": "2020-10-07T09:53:29.994283Z",
"created": "2020-10-07T09:32:06.477750Z",
"min_mon_release": 15,
"min_mon_release_name": "octopus",
"features": {
"persistent": [
"kraken",
"luminous",
"mimic",
"osdmap-prune",
"nautilus",
"octopus"
],
"optional": []
},
"mons": [
{
"rank": 0,
"name": "fredrik-storage-1",
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:3300",
"nonce": 0
},
{
"type": "v1",
"addr": "192.168.200.4:6789",
"nonce": 0
}
]
},
"addr": "192.168.200.4:6789/0",
"public_addr": "192.168.200.4:6789/0",
"priority": 0,
"weight": 0
},
{
"rank": 1,
"name": "fredrik-storage-2",
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:3300",
"nonce": 0
},
{
"type": "v1",
"addr": "192.168.200.5:6789",
"nonce": 0
}
]
},
"addr": "192.168.200.5:6789/0",
"public_addr": "192.168.200.5:6789/0",
"priority": 0,
"weight": 0
},
{
"rank": 2,
"name": "fredrik-storage-3",
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:3300",
"nonce": 0
},
{
"type": "v1",
"addr": "192.168.200.6:6789",
"nonce": 0
}
]
},
"addr": "192.168.200.6:6789/0",
"public_addr": "192.168.200.6:6789/0",
"priority": 0,
"weight": 0
}
]
},
"quorum": [
0,
1,
2
],
"osdmap": {
"epoch": 4915,
"fsid": "...",
"created": "2020-10-07T09:35:24.223890+0000",
"modified": "2020-10-08T07:24:19.438484+0000",
"last_up_change": "2020-10-08T07:08:37.618014+0000",
"last_in_change": "2020-10-07T11:18:48.065950+0000",
"flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit",
"flags_num": 5799936,
"flags_set": [
"pglog_hardlimit",
"purged_snapdirs",
"recovery_deletes",
"sortbitwise"
],
"crush_version": 25,
"full_ratio": 0.94999998807907104,
"backfillfull_ratio": 0.89999997615814209,
"nearfull_ratio": 0.85000002384185791,
"cluster_snapshot": "",
"pool_max": 5,
"max_osd": 12,
"require_min_compat_client": "jewel",
"min_compat_client": "jewel",
"require_osd_release": "octopus",
"pools": [
{
"pool": 1,
"pool_name": "device_health_metrics",
"create_time": "2020-10-07T10:55:18.745520+0000",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_rule": 0,
"object_hash": 2,
"pg_autoscale_mode": "on",
"pg_num": 1,
"pg_placement_num": 1,
"pg_placement_num_target": 1,
"pg_num_target": 1,
"pg_num_pending": 1,
"last_pg_merge_meta": {
"source_pgid": "0.0",
"ready_epoch": 0,
"last_epoch_started": 0,
"last_epoch_clean": 0,
"source_version": "0'0",
"target_version": "0'0"
},
"last_change": "9",
"last_force_op_resend": "0",
"last_force_op_resend_prenautilus": "0",
"last_force_op_resend_preluminous": "0",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_dirty_high_ratio_micro": 600000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"use_gmt_hitset": true,
"min_read_recency_for_promote": 0,
"min_write_recency_for_promote": 0,
"hit_set_grade_decay_rate": 0,
"hit_set_search_last_n": 0,
"grade_table": [],
"stripe_width": 0,
"expected_num_objects": 0,
"fast_read": false,
"options": {
"pg_num_min": 1
},
"application_metadata": {
"mgr_devicehealth": {}
}
},
{
"pool": 4,
"pool_name": "fredrik-pool-rbd",
"create_time": "2020-10-07T13:54:18.461179+0000",
"flags": 1,
"flags_names": "hashpspool",
"type": 1,
"size": 3,
"min_size": 2,
"crush_rule": 0,
"object_hash": 2,
"pg_autoscale_mode": "on",
"pg_num": 32,
"pg_placement_num": 32,
"pg_placement_num_target": 32,
"pg_num_target": 32,
"pg_num_pending": 32,
"last_pg_merge_meta": {
"source_pgid": "4.20",
"ready_epoch": 4583,
"last_epoch_started": 4582,
"last_epoch_clean": 4582,
"source_version": "0'0",
"target_version": "0'0"
},
"last_change": "4915",
"last_force_op_resend": "0",
"last_force_op_resend_prenautilus": "4583",
"last_force_op_resend_preluminous": "4581",
"auid": 0,
"snap_mode": "selfmanaged",
"snap_seq": 0,
"snap_epoch": 0,
"pool_snaps": [],
"removed_snaps": "[]",
"quota_max_bytes": 0,
"quota_max_objects": 0,
"tiers": [],
"tier_of": -1,
"read_tier": -1,
"write_tier": -1,
"cache_mode": "none",
"target_max_bytes": 0,
"target_max_objects": 0,
"cache_target_dirty_ratio_micro": 400000,
"cache_target_dirty_high_ratio_micro": 600000,
"cache_target_full_ratio_micro": 800000,
"cache_min_flush_age": 0,
"cache_min_evict_age": 0,
"erasure_code_profile": "",
"hit_set_params": {
"type": "none"
},
"hit_set_period": 0,
"hit_set_count": 0,
"use_gmt_hitset": true,
"min_read_recency_for_promote": 0,
"min_write_recency_for_promote": 0,
"hit_set_grade_decay_rate": 0,
"hit_set_search_last_n": 0,
"grade_table": [],
"stripe_width": 0,
"expected_num_objects": 0,
"fast_read": false,
"options": {},
"application_metadata": {
"rbd": {}
}
}
],
"osds": [
{
"osd": 0,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4905,
"last_clean_end": 4907,
"up_from": 4910,
"up_thru": 4910,
"down_at": 4908,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6800",
"nonce": 1618
},
{
"type": "v1",
"addr": "192.168.200.4:6801",
"nonce": 1618
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6802",
"nonce": 1618
},
{
"type": "v1",
"addr": "192.168.200.4:6804",
"nonce": 1618
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6808",
"nonce": 1618
},
{
"type": "v1",
"addr": "192.168.200.4:6809",
"nonce": 1618
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6805",
"nonce": 1618
},
{
"type": "v1",
"addr": "192.168.200.4:6806",
"nonce": 1618
}
]
},
"public_addr": "192.168.200.4:6801/1618",
"cluster_addr": "192.168.200.4:6804/1618",
"heartbeat_back_addr": "192.168.200.4:6809/1618",
"heartbeat_front_addr": "192.168.200.4:6806/1618",
"state": [
"exists",
"up"
]
},
{
"osd": 1,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4905,
"last_clean_end": 4907,
"up_from": 4910,
"up_thru": 4910,
"down_at": 4908,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6803",
"nonce": 1612
},
{
"type": "v1",
"addr": "192.168.200.4:6807",
"nonce": 1612
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6810",
"nonce": 1612
},
{
"type": "v1",
"addr": "192.168.200.4:6811",
"nonce": 1612
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6814",
"nonce": 1612
},
{
"type": "v1",
"addr": "192.168.200.4:6815",
"nonce": 1612
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6812",
"nonce": 1612
},
{
"type": "v1",
"addr": "192.168.200.4:6813",
"nonce": 1612
}
]
},
"public_addr": "192.168.200.4:6807/1612",
"cluster_addr": "192.168.200.4:6811/1612",
"heartbeat_back_addr": "192.168.200.4:6815/1612",
"heartbeat_front_addr": "192.168.200.4:6813/1612",
"state": [
"exists",
"up"
]
},
{
"osd": 2,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4905,
"last_clean_end": 4907,
"up_from": 4911,
"up_thru": 4884,
"down_at": 4908,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6816",
"nonce": 1617
},
{
"type": "v1",
"addr": "192.168.200.4:6819",
"nonce": 1617
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6821",
"nonce": 1617
},
{
"type": "v1",
"addr": "192.168.200.4:6823",
"nonce": 1617
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6829",
"nonce": 1617
},
{
"type": "v1",
"addr": "192.168.200.4:6831",
"nonce": 1617
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6825",
"nonce": 1617
},
{
"type": "v1",
"addr": "192.168.200.4:6827",
"nonce": 1617
}
]
},
"public_addr": "192.168.200.4:6819/1617",
"cluster_addr": "192.168.200.4:6823/1617",
"heartbeat_back_addr": "192.168.200.4:6831/1617",
"heartbeat_front_addr": "192.168.200.4:6827/1617",
"state": [
"exists",
"up"
]
},
{
"osd": 3,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4904,
"last_clean_end": 4907,
"up_from": 4911,
"up_thru": 4911,
"down_at": 4908,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6817",
"nonce": 1614
},
{
"type": "v1",
"addr": "192.168.200.4:6818",
"nonce": 1614
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6820",
"nonce": 1614
},
{
"type": "v1",
"addr": "192.168.200.4:6822",
"nonce": 1614
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6828",
"nonce": 1614
},
{
"type": "v1",
"addr": "192.168.200.4:6830",
"nonce": 1614
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.4:6824",
"nonce": 1614
},
{
"type": "v1",
"addr": "192.168.200.4:6826",
"nonce": 1614
}
]
},
"public_addr": "192.168.200.4:6818/1614",
"cluster_addr": "192.168.200.4:6822/1614",
"heartbeat_back_addr": "192.168.200.4:6830/1614",
"heartbeat_front_addr": "192.168.200.4:6826/1614",
"state": [
"exists",
"up"
]
},
{
"osd": 4,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4880,
"last_clean_end": 4889,
"up_from": 4893,
"up_thru": 4911,
"down_at": 4892,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6809",
"nonce": 1641
},
{
"type": "v1",
"addr": "192.168.200.5:6812",
"nonce": 1641
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6815",
"nonce": 1641
},
{
"type": "v1",
"addr": "192.168.200.5:6819",
"nonce": 1641
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6822",
"nonce": 1641
},
{
"type": "v1",
"addr": "192.168.200.5:6823",
"nonce": 1641
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6820",
"nonce": 1641
},
{
"type": "v1",
"addr": "192.168.200.5:6821",
"nonce": 1641
}
]
},
"public_addr": "192.168.200.5:6812/1641",
"cluster_addr": "192.168.200.5:6819/1641",
"heartbeat_back_addr": "192.168.200.5:6823/1641",
"heartbeat_front_addr": "192.168.200.5:6821/1641",
"state": [
"exists",
"up"
]
},
{
"osd": 5,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4880,
"last_clean_end": 4889,
"up_from": 4893,
"up_thru": 4884,
"down_at": 4892,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6800",
"nonce": 1647
},
{
"type": "v1",
"addr": "192.168.200.5:6801",
"nonce": 1647
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6802",
"nonce": 1647
},
{
"type": "v1",
"addr": "192.168.200.5:6803",
"nonce": 1647
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6806",
"nonce": 1647
},
{
"type": "v1",
"addr": "192.168.200.5:6807",
"nonce": 1647
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6804",
"nonce": 1647
},
{
"type": "v1",
"addr": "192.168.200.5:6805",
"nonce": 1647
}
]
},
"public_addr": "192.168.200.5:6801/1647",
"cluster_addr": "192.168.200.5:6803/1647",
"heartbeat_back_addr": "192.168.200.5:6807/1647",
"heartbeat_front_addr": "192.168.200.5:6805/1647",
"state": [
"exists",
"up"
]
},
{
"osd": 6,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4880,
"last_clean_end": 4889,
"up_from": 4893,
"up_thru": 4910,
"down_at": 4892,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6824",
"nonce": 1643
},
{
"type": "v1",
"addr": "192.168.200.5:6825",
"nonce": 1643
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6826",
"nonce": 1643
},
{
"type": "v1",
"addr": "192.168.200.5:6827",
"nonce": 1643
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6830",
"nonce": 1643
},
{
"type": "v1",
"addr": "192.168.200.5:6831",
"nonce": 1643
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6828",
"nonce": 1643
},
{
"type": "v1",
"addr": "192.168.200.5:6829",
"nonce": 1643
}
]
},
"public_addr": "192.168.200.5:6825/1643",
"cluster_addr": "192.168.200.5:6827/1643",
"heartbeat_back_addr": "192.168.200.5:6831/1643",
"heartbeat_front_addr": "192.168.200.5:6829/1643",
"state": [
"exists",
"up"
]
},
{
"osd": 7,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4878,
"last_clean_end": 4889,
"up_from": 4894,
"up_thru": 4911,
"down_at": 4893,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6808",
"nonce": 1645
},
{
"type": "v1",
"addr": "192.168.200.5:6810",
"nonce": 1645
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6811",
"nonce": 1645
},
{
"type": "v1",
"addr": "192.168.200.5:6813",
"nonce": 1645
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6817",
"nonce": 1645
},
{
"type": "v1",
"addr": "192.168.200.5:6818",
"nonce": 1645
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.5:6814",
"nonce": 1645
},
{
"type": "v1",
"addr": "192.168.200.5:6816",
"nonce": 1645
}
]
},
"public_addr": "192.168.200.5:6810/1645",
"cluster_addr": "192.168.200.5:6813/1645",
"heartbeat_back_addr": "192.168.200.5:6818/1645",
"heartbeat_front_addr": "192.168.200.5:6816/1645",
"state": [
"exists",
"up"
]
},
{
"osd": 8,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4587,
"last_clean_end": 4889,
"up_from": 4891,
"up_thru": 4911,
"down_at": 4890,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6808",
"nonce": 1694
},
{
"type": "v1",
"addr": "192.168.200.6:6809",
"nonce": 1694
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6810",
"nonce": 1694
},
{
"type": "v1",
"addr": "192.168.200.6:6811",
"nonce": 1694
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6814",
"nonce": 1694
},
{
"type": "v1",
"addr": "192.168.200.6:6815",
"nonce": 1694
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6812",
"nonce": 1694
},
{
"type": "v1",
"addr": "192.168.200.6:6813",
"nonce": 1694
}
]
},
"public_addr": "192.168.200.6:6809/1694",
"cluster_addr": "192.168.200.6:6811/1694",
"heartbeat_back_addr": "192.168.200.6:6815/1694",
"heartbeat_front_addr": "192.168.200.6:6813/1694",
"state": [
"exists",
"up"
]
},
{
"osd": 9,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4587,
"last_clean_end": 4889,
"up_from": 4892,
"up_thru": 4911,
"down_at": 4891,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6816",
"nonce": 1689
},
{
"type": "v1",
"addr": "192.168.200.6:6817",
"nonce": 1689
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6818",
"nonce": 1689
},
{
"type": "v1",
"addr": "192.168.200.6:6819",
"nonce": 1689
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6826",
"nonce": 1689
},
{
"type": "v1",
"addr": "192.168.200.6:6828",
"nonce": 1689
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6822",
"nonce": 1689
},
{
"type": "v1",
"addr": "192.168.200.6:6824",
"nonce": 1689
}
]
},
"public_addr": "192.168.200.6:6817/1689",
"cluster_addr": "192.168.200.6:6819/1689",
"heartbeat_back_addr": "192.168.200.6:6828/1689",
"heartbeat_front_addr": "192.168.200.6:6824/1689",
"state": [
"exists",
"up"
]
},
{
"osd": 10,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4587,
"last_clean_end": 4889,
"up_from": 4891,
"up_thru": 4911,
"down_at": 4890,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6800",
"nonce": 1675
},
{
"type": "v1",
"addr": "192.168.200.6:6801",
"nonce": 1675
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6802",
"nonce": 1675
},
{
"type": "v1",
"addr": "192.168.200.6:6803",
"nonce": 1675
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6806",
"nonce": 1675
},
{
"type": "v1",
"addr": "192.168.200.6:6807",
"nonce": 1675
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6804",
"nonce": 1675
},
{
"type": "v1",
"addr": "192.168.200.6:6805",
"nonce": 1675
}
]
},
"public_addr": "192.168.200.6:6801/1675",
"cluster_addr": "192.168.200.6:6803/1675",
"heartbeat_back_addr": "192.168.200.6:6807/1675",
"heartbeat_front_addr": "192.168.200.6:6805/1675",
"state": [
"exists",
"up"
]
},
{
"osd": 11,
"uuid": "...",
"up": 1,
"in": 1,
"weight": 1,
"primary_affinity": 1,
"last_clean_begin": 4587,
"last_clean_end": 4889,
"up_from": 4892,
"up_thru": 4911,
"down_at": 4891,
"lost_at": 0,
"public_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6820",
"nonce": 1695
},
{
"type": "v1",
"addr": "192.168.200.6:6821",
"nonce": 1695
}
]
},
"cluster_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6823",
"nonce": 1695
},
{
"type": "v1",
"addr": "192.168.200.6:6825",
"nonce": 1695
}
]
},
"heartbeat_back_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6830",
"nonce": 1695
},
{
"type": "v1",
"addr": "192.168.200.6:6831",
"nonce": 1695
}
]
},
"heartbeat_front_addrs": {
"addrvec": [
{
"type": "v2",
"addr": "192.168.200.6:6827",
"nonce": 1695
},
{
"type": "v1",
"addr": "192.168.200.6:6829",
"nonce": 1695
}
]
},
"public_addr": "192.168.200.6:6821/1695",
"cluster_addr": "192.168.200.6:6825/1695",
"heartbeat_back_addr": "192.168.200.6:6831/1695",
"heartbeat_front_addr": "192.168.200.6:6829/1695",
"state": [
"exists",
"up"
]
}
],
"osd_xinfo": [
{
"osd": 0,
"down_stamp": "2020-10-07T14:41:12.136686+0000",
"laggy_probability": 0.035294700413942337,
"laggy_interval": 4,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:10:16.713131+0000",
"dead_epoch": 4908
},
{
"osd": 1,
"down_stamp": "2020-10-07T14:41:12.136686+0000",
"laggy_probability": 0.035294700413942337,
"laggy_interval": 4,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:10:27.307048+0000",
"dead_epoch": 4908
},
{
"osd": 2,
"down_stamp": "2020-10-07T14:41:11.132961+0000",
"laggy_probability": 0.035294700413942337,
"laggy_interval": 4,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:10:33.728095+0000",
"dead_epoch": 4908
},
{
"osd": 3,
"down_stamp": "2020-10-07T14:41:12.136686+0000",
"laggy_probability": 0.035294700413942337,
"laggy_interval": 4,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:10:40.990746+0000",
"dead_epoch": 4908
},
{
"osd": 4,
"down_stamp": "2020-10-07T15:49:55.988766+0000",
"laggy_probability": 0,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:16:09.985081+0000",
"dead_epoch": 4892
},
{
"osd": 5,
"down_stamp": "2020-10-07T15:49:55.988766+0000",
"laggy_probability": 0,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:16:17.196160+0000",
"dead_epoch": 4892
},
{
"osd": 6,
"down_stamp": "2020-10-07T15:49:55.988766+0000",
"laggy_probability": 0,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:16:24.468766+0000",
"dead_epoch": 4892
},
{
"osd": 7,
"down_stamp": "2020-10-08T06:48:02.848469+0000",
"laggy_probability": 0,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:16:34.135464+0000",
"dead_epoch": 0
},
{
"osd": 8,
"down_stamp": "2020-10-08T06:47:56.333080+0000",
"laggy_probability": 0.21000000834465027,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:18:27.975630+0000",
"dead_epoch": 4884
},
{
"osd": 9,
"down_stamp": "2020-10-08T06:47:57.337681+0000",
"laggy_probability": 0.21000000834465027,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:18:34.013912+0000",
"dead_epoch": 4887
},
{
"osd": 10,
"down_stamp": "2020-10-08T06:47:56.333080+0000",
"laggy_probability": 0.21000000834465027,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:18:40.708812+0000",
"dead_epoch": 4884
},
{
"osd": 11,
"down_stamp": "2020-10-08T06:47:57.337681+0000",
"laggy_probability": 0.21000000834465027,
"laggy_interval": 0,
"features": 4540138292836696063,
"old_weight": 0,
"last_purged_snaps_scrub": "2020-10-07T11:18:46.803204+0000",
"dead_epoch": 4886
}
],
"pg_upmap": [],
"pg_upmap_items": [],
"pg_temp": [],
"primary_temp": [],
"blacklist": {
"192.168.200.4:6833/3065": "2020-10-08T15:50:06.344949+0000",
"192.168.200.4:0/1073465076": "2020-10-08T14:41:46.230293+0000",
"192.168.200.4:6833/3052": "2020-10-08T14:41:46.230293+0000",
"192.168.200.4:6832/3052": "2020-10-08T14:41:46.230293+0000",
"192.168.200.4:6833/4578": "2020-10-09T07:05:07.263603+0000",
"192.168.200.4:6832/4578": "2020-10-09T07:05:07.263603+0000",
"192.168.200.4:0/1792506827": "2020-10-08T11:36:44.027070+0000",
"192.168.200.4:0/4209226949": "2020-10-08T14:41:46.230293+0000",
"192.168.200.4:6832/2644": "2020-10-09T06:48:19.773296+0000",
"192.168.200.4:6833/2644": "2020-10-09T06:48:19.773296+0000",
"192.168.200.5:6833/46194": "2020-10-08T11:47:54.181066+0000",
"192.168.200.4:0/3306137174": "2020-10-08T11:36:44.027070+0000",
"192.168.200.4:6832/3065": "2020-10-08T15:50:06.344949+0000",
"192.168.200.5:0/1593246043": "2020-10-08T11:47:54.181066+0000",
"192.168.200.4:0/677449457": "2020-10-09T06:48:19.773296+0000",
"192.168.200.5:0/2306087537": "2020-10-08T11:47:54.181066+0000",
"192.168.200.4:0/1143918189": "2020-10-08T15:50:06.344949+0000",
"192.168.200.5:6832/46194": "2020-10-08T11:47:54.181066+0000",
"192.168.200.4:0/526602322": "2020-10-09T07:05:07.263603+0000",
"192.168.200.4:0/3847305079": "2020-10-09T07:05:07.263603+0000",
"192.168.200.4:6801/49129": "2020-10-08T11:36:44.027070+0000",
"192.168.200.4:6800/49129": "2020-10-08T11:36:44.027070+0000",
"192.168.200.4:0/2656952789": "2020-10-09T07:05:07.263603+0000",
"192.168.200.4:0/3910996860": "2020-10-08T15:50:06.344949+0000",
"192.168.200.4:0/430149818": "2020-10-08T11:36:44.027070+0000"
},
"erasure_code_profiles": {
"default": {
"k": "2",
"m": "2",
"plugin": "jerasure",
"technique": "reed_sol_van"
}
},
"removed_snaps_queue": [],
"new_removed_snaps": [],
"new_purged_snaps": [],
"crush_node_flags": {},
"device_class_flags": {}
},
"osd_metadata": [
{
"id": 0,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.4:6802/1618,v1:192.168.200.4:6804/1618]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-0",
"bluestore_bdev_devices": "sdc",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-0",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdc=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:2:0",
"devices": "sdc",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.4:6800/1618,v1:192.168.200.4:6801/1618]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.4:6808/1618,v1:192.168.200.4:6809/1618]",
"hb_front_addr": "[v2:192.168.200.4:6805/1618,v1:192.168.200.4:6806/1618]",
"hostname": "fredrik-storage-1",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088476",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdc",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-0",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 1,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.4:6810/1612,v1:192.168.200.4:6811/1612]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-3",
"bluestore_bdev_devices": "sdd",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-3",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdd=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:3:0",
"devices": "sdd",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.4:6803/1612,v1:192.168.200.4:6807/1612]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.4:6814/1612,v1:192.168.200.4:6815/1612]",
"hb_front_addr": "[v2:192.168.200.4:6812/1612,v1:192.168.200.4:6813/1612]",
"hostname": "fredrik-storage-1",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088476",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdd",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-1",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 2,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.4:6821/1617,v1:192.168.200.4:6823/1617]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-2",
"bluestore_bdev_devices": "sde",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-2",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sde=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:4:0",
"devices": "sde",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.4:6816/1617,v1:192.168.200.4:6819/1617]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.4:6829/1617,v1:192.168.200.4:6831/1617]",
"hb_front_addr": "[v2:192.168.200.4:6825/1617,v1:192.168.200.4:6827/1617]",
"hostname": "fredrik-storage-1",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088476",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sde",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-2",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 3,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.4:6820/1614,v1:192.168.200.4:6822/1614]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-1",
"bluestore_bdev_devices": "sdf",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-1",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdf=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:5:0",
"devices": "sdf",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.4:6817/1614,v1:192.168.200.4:6818/1614]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.4:6828/1614,v1:192.168.200.4:6830/1614]",
"hb_front_addr": "[v2:192.168.200.4:6824/1614,v1:192.168.200.4:6826/1614]",
"hostname": "fredrik-storage-1",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088476",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdf",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-3",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 4,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.5:6815/1641,v1:192.168.200.5:6819/1641]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-0",
"bluestore_bdev_devices": "sdc",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-0",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdc=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:2:0",
"devices": "sdc",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.5:6809/1641,v1:192.168.200.5:6812/1641]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.5:6822/1641,v1:192.168.200.5:6823/1641]",
"hb_front_addr": "[v2:192.168.200.5:6820/1641,v1:192.168.200.5:6821/1641]",
"hostname": "fredrik-storage-2",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdc",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-4",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 5,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.5:6802/1647,v1:192.168.200.5:6803/1647]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-1",
"bluestore_bdev_devices": "sdd",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-1",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdd=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:3:0",
"devices": "sdd",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.5:6800/1647,v1:192.168.200.5:6801/1647]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.5:6806/1647,v1:192.168.200.5:6807/1647]",
"hb_front_addr": "[v2:192.168.200.5:6804/1647,v1:192.168.200.5:6805/1647]",
"hostname": "fredrik-storage-2",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdd",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-5",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 6,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.5:6826/1643,v1:192.168.200.5:6827/1643]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-2",
"bluestore_bdev_devices": "sde",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-2",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sde=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:4:0",
"devices": "sde",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.5:6824/1643,v1:192.168.200.5:6825/1643]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.5:6830/1643,v1:192.168.200.5:6831/1643]",
"hb_front_addr": "[v2:192.168.200.5:6828/1643,v1:192.168.200.5:6829/1643]",
"hostname": "fredrik-storage-2",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sde",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-6",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 7,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.5:6811/1645,v1:192.168.200.5:6813/1645]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-3",
"bluestore_bdev_devices": "sdf",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-3",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdf=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:5:0",
"devices": "sdf",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.5:6808/1645,v1:192.168.200.5:6810/1645]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.5:6817/1645,v1:192.168.200.5:6818/1645]",
"hb_front_addr": "[v2:192.168.200.5:6814/1645,v1:192.168.200.5:6816/1645]",
"hostname": "fredrik-storage-2",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdf",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-7",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 8,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.6:6810/1694,v1:192.168.200.6:6811/1694]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-1",
"bluestore_bdev_devices": "sdc",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-1",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdc=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:2:0",
"devices": "sdc",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.6:6808/1694,v1:192.168.200.6:6809/1694]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.6:6814/1694,v1:192.168.200.6:6815/1694]",
"hb_front_addr": "[v2:192.168.200.6:6812/1694,v1:192.168.200.6:6813/1694]",
"hostname": "fredrik-storage-3",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdc",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-8",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 9,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.6:6818/1689,v1:192.168.200.6:6819/1689]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-0",
"bluestore_bdev_devices": "sdd",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-0",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdd=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:3:0",
"devices": "sdd",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.6:6816/1689,v1:192.168.200.6:6817/1689]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.6:6826/1689,v1:192.168.200.6:6828/1689]",
"hb_front_addr": "[v2:192.168.200.6:6822/1689,v1:192.168.200.6:6824/1689]",
"hostname": "fredrik-storage-3",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdd",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-9",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 10,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.6:6802/1675,v1:192.168.200.6:6803/1675]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-3",
"bluestore_bdev_devices": "sde",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-3",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sde=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:4:0",
"devices": "sde",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.6:6800/1675,v1:192.168.200.6:6801/1675]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.6:6806/1675,v1:192.168.200.6:6807/1675]",
"hb_front_addr": "[v2:192.168.200.6:6804/1675,v1:192.168.200.6:6805/1675]",
"hostname": "fredrik-storage-3",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sde",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-10",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
},
{
"id": 11,
"arch": "x86_64",
"back_addr": "[v2:192.168.200.6:6823/1695,v1:192.168.200.6:6825/1695]",
"back_iface": "enp0s8",
"bluefs": "1",
"bluefs_dedicated_db": "0",
"bluefs_dedicated_wal": "0",
"bluefs_single_shared_device": "1",
"bluestore_bdev_access_mode": "blk",
"bluestore_bdev_block_size": "4096",
"bluestore_bdev_dev_node": "/dev/dm-2",
"bluestore_bdev_devices": "sdf",
"bluestore_bdev_driver": "KernelDevice",
"bluestore_bdev_partition_path": "/dev/dm-2",
"bluestore_bdev_rotational": "1",
"bluestore_bdev_size": "16101933056",
"bluestore_bdev_support_discard": "0",
"bluestore_bdev_type": "hdd",
"ceph_release": "octopus",
"ceph_version": "ceph version 15.2.5 (2c93eff00150f0cc5f106a559557a58d3d7b6f1f) octopus (stable)",
"ceph_version_short": "15.2.5",
"cpu": "Intel(R) Core(TM) i5-10500 CPU @ 3.10GHz",
"default_device_class": "hdd",
"device_ids": "",
"device_paths": "sdf=/dev/disk/by-path/pci-0000:00:0f.0-scsi-0:0:5:0",
"devices": "sdf",
"distro": "ubuntu",
"distro_description": "Ubuntu 20.04.1 LTS",
"distro_version": "20.04",
"front_addr": "[v2:192.168.200.6:6820/1695,v1:192.168.200.6:6821/1695]",
"front_iface": "enp0s8",
"hb_back_addr": "[v2:192.168.200.6:6830/1695,v1:192.168.200.6:6831/1695]",
"hb_front_addr": "[v2:192.168.200.6:6827/1695,v1:192.168.200.6:6829/1695]",
"hostname": "fredrik-storage-3",
"journal_rotational": "1",
"kernel_description": "#52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020",
"kernel_version": "5.4.0-48-generic",
"mem_swap_kb": "4194300",
"mem_total_kb": "6088468",
"network_numa_nodes": "-1",
"objectstore_numa_unknown_devices": "sdf",
"os": "Linux",
"osd_data": "/var/lib/ceph/osd/ceph-11",
"osd_objectstore": "bluestore",
"osdspec_affinity": "",
"rotational": "1"
}
],
"osdmap_clean_epochs": {
"min_last_epoch_clean": 4914,
"last_epoch_clean": {
"per_pool": [
{
"poolid": 1,
"floor": 4914
},
{
"poolid": 4,
"floor": 4914
}
]
},
"osd_epochs": [
{
"id": 0,
"epoch": 4915
},
{
"id": 1,
"epoch": 4915
},
{
"id": 2,
"epoch": 4915
},
{
"id": 3,
"epoch": 4915
},
{
"id": 4,
"epoch": 4915
},
{
"id": 5,
"epoch": 4915
},
{
"id": 6,
"epoch": 4915
},
{
"id": 7,
"epoch": 4915
},
{
"id": 8,
"epoch": 4915
},
{
"id": 9,
"epoch": 4915
},
{
"id": 10,
"epoch": 4915
},
{
"id": 11,
"epoch": 4915
}
]
},
"osdmap_first_committed": 4363,
"osdmap_last_committed": 4915,
"crushmap": {
"devices": [
{
"id": 0,
"name": "osd.0",
"class": "hdd"
},
{
"id": 1,
"name": "osd.1",
"class": "hdd"
},
{
"id": 2,
"name": "osd.2",
"class": "hdd"
},
{
"id": 3,
"name": "osd.3",
"class": "hdd"
},
{
"id": 4,
"name": "osd.4",
"class": "hdd"
},
{
"id": 5,
"name": "osd.5",
"class": "hdd"
},
{
"id": 6,
"name": "osd.6",
"class": "hdd"
},
{
"id": 7,
"name": "osd.7",
"class": "hdd"
},
{
"id": 8,
"name": "osd.8",
"class": "hdd"
},
{
"id": 9,
"name": "osd.9",
"class": "hdd"
},
{
"id": 10,
"name": "osd.10",
"class": "hdd"
},
{
"id": 11,
"name": "osd.11",
"class": "hdd"
}
],
"types": [
{
"type_id": 0,
"name": "osd"
},
{
"type_id": 1,
"name": "host"
},
{
"type_id": 2,
"name": "chassis"
},
{
"type_id": 3,
"name": "rack"
},
{
"type_id": 4,
"name": "row"
},
{
"type_id": 5,
"name": "pdu"
},
{
"type_id": 6,
"name": "pod"
},
{
"type_id": 7,
"name": "room"
},
{
"type_id": 8,
"name": "datacenter"
},
{
"type_id": 9,
"name": "zone"
},
{
"type_id": 10,
"name": "region"
},
{
"type_id": 11,
"name": "root"
}
],
"buckets": [
{
"id": -1,
"name": "default",
"type_id": 11,
"type_name": "root",
"weight": 11472,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": -3,
"weight": 3824,
"pos": 0
},
{
"id": -5,
"weight": 3824,
"pos": 1
},
{
"id": -7,
"weight": 3824,
"pos": 2
}
]
},
{
"id": -2,
"name": "default~hdd",
"type_id": 11,
"type_name": "root",
"weight": 11472,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": -4,
"weight": 3824,
"pos": 0
},
{
"id": -6,
"weight": 3824,
"pos": 1
},
{
"id": -8,
"weight": 3824,
"pos": 2
}
]
},
{
"id": -3,
"name": "fredrik-storage-1",
"type_id": 1,
"type_name": "host",
"weight": 3824,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": 0,
"weight": 956,
"pos": 0
},
{
"id": 1,
"weight": 956,
"pos": 1
},
{
"id": 2,
"weight": 956,
"pos": 2
},
{
"id": 3,
"weight": 956,
"pos": 3
}
]
},
{
"id": -4,
"name": "fredrik-storage-1~hdd",
"type_id": 1,
"type_name": "host",
"weight": 3824,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": 0,
"weight": 956,
"pos": 0
},
{
"id": 1,
"weight": 956,
"pos": 1
},
{
"id": 2,
"weight": 956,
"pos": 2
},
{
"id": 3,
"weight": 956,
"pos": 3
}
]
},
{
"id": -5,
"name": "fredrik-storage-2",
"type_id": 1,
"type_name": "host",
"weight": 3824,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": 4,
"weight": 956,
"pos": 0
},
{
"id": 5,
"weight": 956,
"pos": 1
},
{
"id": 6,
"weight": 956,
"pos": 2
},
{
"id": 7,
"weight": 956,
"pos": 3
}
]
},
{
"id": -6,
"name": "fredrik-storage-2~hdd",
"type_id": 1,
"type_name": "host",
"weight": 3824,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": 4,
"weight": 956,
"pos": 0
},
{
"id": 5,
"weight": 956,
"pos": 1
},
{
"id": 6,
"weight": 956,
"pos": 2
},
{
"id": 7,
"weight": 956,
"pos": 3
}
]
},
{
"id": -7,
"name": "fredrik-storage-3",
"type_id": 1,
"type_name": "host",
"weight": 3824,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": 8,
"weight": 956,
"pos": 0
},
{
"id": 9,
"weight": 956,
"pos": 1
},
{
"id": 10,
"weight": 956,
"pos": 2
},
{
"id": 11,
"weight": 956,
"pos": 3
}
]
},
{
"id": -8,
"name": "fredrik-storage-3~hdd",
"type_id": 1,
"type_name": "host",
"weight": 3824,
"alg": "straw2",
"hash": "rjenkins1",
"items": [
{
"id": 8,
"weight": 956,
"pos": 0
},
{
"id": 9,
"weight": 956,
"pos": 1
},
{
"id": 10,
"weight": 956,
"pos": 2
},
{
"id": 11,
"weight": 956,
"pos": 3
}
]
}
],
"rules": [
{
"rule_id": 0,
"rule_name": "replicated_rule",
"ruleset": 0,
"type": 1,
"min_size": 1,
"max_size": 10,
"steps": [
{
"op": "take",
"item": -1,
"item_name": "default"
},
{
"op": "chooseleaf_firstn",
"num": 0,
"type": "host"
},
{
"op": "emit"
}
]
}
],
"tunables": {
"choose_local_tries": 0,
"choose_local_fallback_tries": 0,
"choose_total_tries": 50,
"chooseleaf_descend_once": 1,
"chooseleaf_vary_r": 1,
"chooseleaf_stable": 1,
"straw_calc_version": 1,
"allowed_bucket_algs": 54,
"profile": "jewel",
"optimal_tunables": 1,
"legacy_tunables": 0,
"minimum_required_version": "jewel",
"require_feature_tunables": 1,
"require_feature_tunables2": 1,
"has_v2_rules": 0,
"require_feature_tunables3": 1,
"has_v3_rules": 0,
"has_v4_buckets": 1,
"require_feature_tunables5": 1,
"has_v5_rules": 0
},
"choose_args": {}
},
"fsmap": {
"epoch": 1,
"default_fscid": -1,
"compat": {
"compat": {},
"ro_compat": {},
"incompat": {
"feature_1": "base v0.20",
"feature_2": "client writeable ranges",
"feature_3": "default file layouts on dirs",
"feature_4": "dir inode in separate object",
"feature_5": "mds uses versioned encoding",
"feature_6": "dirfrag is stored in omap",
"feature_8": "no anchor table",
"feature_9": "file layout v2",
"feature_10": "snaprealm v2"
}
},
"feature_flags": {
"enable_multiple": false,
"ever_enabled_multiple": false
},
"standbys": [],
"filesystems": []
},
"mdsmap_first_committed": 1,
"mdsmap_last_committed": 1,
"auth": {
"first_committed": 1,
"last_committed": 47,
"num_secrets": 23
},
"num_pg": 33,
"num_pg_active": 33,
"num_pg_unknown": 0,
"num_osd": 12,
"pool_sum": {
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0,
"num_large_omap_objects": 0,
"num_objects_manifest": 0,
"num_omap_bytes": 0,
"num_omap_keys": 0,
"num_objects_repaired": 0
},
"store_stats": {
"total": 0,
"available": 0,
"internally_reserved": 0,
"allocated": 0,
"data_stored": 0,
"data_compressed": 0,
"data_compressed_allocated": 0,
"data_compressed_original": 0,
"omap_allocated": 0,
"internal_metadata": 0
},
"log_size": 0,
"ondisk_log_size": 0,
"up": 99,
"acting": 99,
"num_store_stats": 0
},
"osd_sum": {
"up_from": 0,
"seq": 0,
"num_pgs": 99,
"num_osds": 12,
"num_per_pool_osds": 12,
"num_per_pool_omap_osds": 12,
"kb": 188694528,
"kb_used": 13436928,
"kb_used_data": 853248,
"kb_used_omap": 174,
"kb_used_meta": 12582737,
"kb_avail": 175257600,
"statfs": {
"total": 193223196672,
"available": 179463782400,
"internally_reserved": 12884901888,
"allocated": 873725952,
"data_stored": 54423240,
"data_compressed": 0,
"data_compressed_allocated": 0,
"data_compressed_original": 0,
"omap_allocated": 178311,
"internal_metadata": 12884723577
},
"hb_peers": [],
"snap_trim_queue_len": 0,
"num_snap_trimming": 0,
"num_shards_repaired": 0,
"op_queue_age_hist": {
"histogram": [],
"upper_bound": 1
},
"perf_stat": {
"commit_latency_ms": 0,
"apply_latency_ms": 0,
"commit_latency_ns": 0,
"apply_latency_ns": 0
},
"alerts": [],
"network_ping_times": []
},
"osd_sum_by_class": {
"hdd": {
"up_from": 0,
"seq": 0,
"num_pgs": 99,
"num_osds": 12,
"num_per_pool_osds": 12,
"num_per_pool_omap_osds": 12,
"kb": 188694528,
"kb_used": 13436928,
"kb_used_data": 853248,
"kb_used_omap": 174,
"kb_used_meta": 12582737,
"kb_avail": 175257600,
"statfs": {
"total": 193223196672,
"available": 179463782400,
"internally_reserved": 12884901888,
"allocated": 873725952,
"data_stored": 54423240,
"data_compressed": 0,
"data_compressed_allocated": 0,
"data_compressed_original": 0,
"omap_allocated": 178311,
"internal_metadata": 12884723577
},
"hb_peers": [],
"snap_trim_queue_len": 0,
"num_snap_trimming": 0,
"num_shards_repaired": 0,
"op_queue_age_hist": {
"histogram": [],
"upper_bound": 1
},
"perf_stat": {
"commit_latency_ms": 0,
"apply_latency_ms": 0,
"commit_latency_ns": 0,
"apply_latency_ns": 0
},
"alerts": [],
"network_ping_times": []
}
},
"pool_stats": [
{
"poolid": 4,
"num_pg": 32,
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0,
"num_large_omap_objects": 0,
"num_objects_manifest": 0,
"num_omap_bytes": 0,
"num_omap_keys": 0,
"num_objects_repaired": 0
},
"store_stats": {
"total": 0,
"available": 0,
"internally_reserved": 0,
"allocated": 0,
"data_stored": 0,
"data_compressed": 0,
"data_compressed_allocated": 0,
"data_compressed_original": 0,
"omap_allocated": 0,
"internal_metadata": 0
},
"log_size": 0,
"ondisk_log_size": 0,
"up": 96,
"acting": 96,
"num_store_stats": 12
},
{
"poolid": 1,
"num_pg": 1,
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0,
"num_large_omap_objects": 0,
"num_objects_manifest": 0,
"num_omap_bytes": 0,
"num_omap_keys": 0,
"num_objects_repaired": 0
},
"store_stats": {
"total": 0,
"available": 0,
"internally_reserved": 0,
"allocated": 0,
"data_stored": 0,
"data_compressed": 0,
"data_compressed_allocated": 0,
"data_compressed_original": 0,
"omap_allocated": 0,
"internal_metadata": 0
},
"log_size": 0,
"ondisk_log_size": 0,
"up": 3,
"acting": 3,
"num_store_stats": 3
}
],
"osd_stats": [
{
"osd": 0,
"seq": 21088289423721
},
{
"osd": 1,
"seq": 21088289423721
},
{
"osd": 2,
"seq": 21092584391017
},
{
"osd": 3,
"seq": 21092584391017
},
{
"osd": 4,
"seq": 21015274979922
},
{
"osd": 5,
"seq": 21015274979922
},
{
"osd": 6,
"seq": 21015274979922
},
{
"osd": 7,
"seq": 21019569947218
},
{
"osd": 8,
"seq": 21006685045330
},
{
"osd": 9,
"seq": 21010980012626
},
{
"osd": 10,
"seq": 21006685045330
},
{
"osd": 11,
"seq": 21010980012626
}
],
"num_pg_by_state": [
{
"state": "active+clean",
"num": 33
}
],
"num_pg_by_osd": [
{
"osd": 1,
"num_primary_pg": 4,
"num_acting_pg": 15,
"num_up_not_acting_pg": 0
},
{
"osd": 2,
"num_primary_pg": 0,
"num_acting_pg": 2,
"num_up_not_acting_pg": 0
},
{
"osd": 0,
"num_primary_pg": 3,
"num_acting_pg": 5,
"num_up_not_acting_pg": 0
},
{
"osd": 3,
"num_primary_pg": 4,
"num_acting_pg": 11,
"num_up_not_acting_pg": 0
},
{
"osd": 4,
"num_primary_pg": 7,
"num_acting_pg": 10,
"num_up_not_acting_pg": 0
},
{
"osd": 6,
"num_primary_pg": 2,
"num_acting_pg": 5,
"num_up_not_acting_pg": 0
},
{
"osd": 10,
"num_primary_pg": 1,
"num_acting_pg": 8,
"num_up_not_acting_pg": 0
},
{
"osd": 9,
"num_primary_pg": 4,
"num_acting_pg": 11,
"num_up_not_acting_pg": 0
},
{
"osd": 5,
"num_primary_pg": 0,
"num_acting_pg": 5,
"num_up_not_acting_pg": 0
},
{
"osd": 8,
"num_primary_pg": 2,
"num_acting_pg": 6,
"num_up_not_acting_pg": 0
},
{
"osd": 11,
"num_primary_pg": 5,
"num_acting_pg": 8,
"num_up_not_acting_pg": 0
},
{
"osd": 7,
"num_primary_pg": 1,
"num_acting_pg": 13,
"num_up_not_acting_pg": 0
}
],
"purged_snaps": [
{
"pool": 1,
"purged_snaps": {}
},
{
"pool": 4,
"purged_snaps": {}
}
],
"servicemap": {
"epoch": 1,
"modified": "2020-10-07T09:35:24.224276+0000",
"services": {}
},
"paxos": {
"first_committed": 31627,
"last_committed": 32271,
"last_pn": 3000,
"accepted_pn": 3000
}
}
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
