I have had the same problem before with unfound objects that happened while backfilling after losing a drive. We didn't lose drives outside of the failure domains and ultimately didn't lose any data, but we did have to wait until after all of the PGs in recovery_wait state were caught up. So if the 2 disks you lost were in the same host and your CRUSH rules are set so that you can lose a host without losing data, then the cluster will likely find all of the objects by the time it's done backfilling. With only losing 2 disks, I wouldn't worry about the missing objects not becoming found unless you're pool size=2.
On Mon, Jan 22, 2018 at 11:47 AM Nico Schottelius <nico.schottelius@xxxxxxxxxxx> wrote:
Hello,
we added about 7 new disks yesterday/today and our cluster became very
slow. While the rebalancing took place, 2 of the 7 new added disks
died.
Our cluster is still recovering, however we spotted that there are a lot
of unfound objects.
We lost osd.63 and osd.64, which seem not to be involved into the sample
pg that has unfound objects.
We were wondering why there are unfound objects, where they are coming
from and if there is a way to recover them?
Any help appreciated,
Best,
Nico
Our status is:
cluster:
id: 26c0c5a8-d7ce-49ac-b5a7-bfd9d0ba81ab
health: HEALTH_WARN
261953/3006663 objects misplaced (8.712%)
9377/1002221 objects unfound (0.936%)
Reduced data availability: 176 pgs inactive
Degraded data redundancy: 609338/3006663 objects degraded (20.266%), 243 pgs unclea
n, 222 pgs degraded, 213 pgs undersized
mon server2 is low on available space
services:
mon: 3 daemons, quorum server5,server3,server2
mgr: server5(active), standbys: 2, server2, 0, server3
osd: 54 osds: 54 up, 54 in; 234 remapped pgs
data:
pools: 3 pools, 1344 pgs
objects: 978k objects, 3823 GB
usage: 9350 GB used, 40298 GB / 49648 GB avail
pgs: 13.095% pgs not active
609338/3006663 objects degraded (20.266%)
261953/3006663 objects misplaced (8.712%)
9377/1002221 objects unfound (0.936%)
1101 active+clean
84 recovery_wait+undersized+degraded+remapped+peered
82 undersized+degraded+remapped+backfill_wait+peered
23 active+undersized+degraded+remapped+backfill_wait
18 active+remapped+backfill_wait
14 active+undersized+degraded+remapped+backfilling
10 undersized+degraded+remapped+backfilling+peered
9 active+recovery_wait+degraded
3 active+remapped+backfilling
io:
client: 624 kB/s rd, 3255 kB/s wr, 22 op/s rd, 66 op/s wr
recovery: 90148 kB/s, 22 objects/s
Looking at the unfound objects:
[17:32:17] server1:~# ceph health detail
HEALTH_WARN 263745/3006663 objects misplaced (8.772%); 9377/1002221 objects unfound (0.936%); Reduced data availability: 176 pgs inactive; Degraded data redundancy: 612398/3006663 objects degraded (20.368%), 244 pgs unclean, 223 pgs degraded, 214 pgs undersized; mon server2 is low on available space
OBJECT_MISPLACED 263745/3006663 objects misplaced (8.772%)
OBJECT_UNFOUND 9377/1002221 objects unfound (0.936%)
pg 4.fa has 117 unfound objects
pg 4.ff has 107 unfound objects
pg 4.fd has 113 unfound objects
pg 4.f0 has 120 unfound objects
....
Output from ceph pg 4.fa query:
{
"state": "recovery_wait+undersized+degraded+remapped+peered",
"snap_trimq": "[]",
"epoch": 17561,
"up": [
8,
17,
25
],
"acting": [
61
],
"backfill_targets": [
"8",
"17",
"25"
],
"actingbackfill": [
"8",
"17",
"25",
"61"
],
"info": {
"pgid": "4.fa",
"last_update": "17529'85051",
"last_complete": "17217'77468",
"log_tail": "17091'75034",
"last_user_version": 85051,
"last_backfill": "MAX",
"last_backfill_bitwise": 0,
"purged_snaps": [
{
"start": "1",
"length": "3"
},
{
"start": "6",
"length": "8"
},
{
"start": "10",
"length": "2"
}
],
"history": {
"epoch_created": 9134,
"epoch_pool_created": 9134,
"last_epoch_started": 17528,
"last_interval_started": 17527,
"last_epoch_clean": 17079,
"last_interval_clean": 17078,
"last_epoch_split": 0,
"last_epoch_marked_full": 0,
"same_up_since": 17143,
"same_interval_since": 17530,
"same_primary_since": 17515,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142"
},
"stats": {
"version": "17529'85051",
"reported_seq": "218453",
"reported_epoch": "17561",
"state": "recovery_wait+undersized+degraded+remapped+peered",
"last_fresh": "2018-01-22 17:42:28.196701",
"last_change": "2018-01-22 15:00:46.507189",
"last_active": "2018-01-22 15:00:44.635399",
"last_peered": "2018-01-22 17:42:28.196701",
"last_clean": "2018-01-21 20:15:48.267209",
"last_became_active": "2018-01-22 14:53:07.918893",
"last_became_peered": "2018-01-22 15:00:46.437493",
"last_unstale": "2018-01-22 17:42:28.196701",
"last_undegraded": "2018-01-22 15:00:45.664468",
"last_fullsized": "2018-01-22 15:00:45.664468",
"mapping_epoch": 17530,
"log_start": "17091'75034",
"ondisk_log_start": "17091'75034",
"created": 9134,
"last_epoch_clean": 17079,
"parent": "0.0",
"parent_split_bits": 0,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142",
"log_size": 10017,
"ondisk_log_size": 10017,
"stats_invalid": false,
"dirty_stats_invalid": false,
"omap_stats_invalid": false,
"hitset_stats_invalid": false,
"hitset_bytes_stats_invalid": false,
"pin_stats_invalid": false,
"stat_sum": {
"num_bytes": 6112366592,
"num_objects": 1503,
"num_object_clones": 0,
"num_object_copies": 4509,
"num_objects_missing_on_primary": 117,
"num_objects_missing": 0,
"num_objects_degraded": 3123,
"num_objects_misplaced": 1386,
"num_objects_unfound": 117,
"num_objects_dirty": 1503,
"num_whiteouts": 0,
"num_read": 165528,
"num_read_kb": 3252584,
"num_write": 87725,
"num_write_kb": 14125516,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 244,
"num_bytes_recovered": 963108864,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0
},
"up": [
8,
17,
25
],
"acting": [
61
],
"blocked_by": [],
"up_primary": 8,
"acting_primary": 61
},
"empty": 0,
"dne": 0,
"incomplete": 0,
"last_epoch_started": 17528,
"hit_set_history": {
"current_last_update": "0'0",
"history": []
}
},
"peer_info": [
{
"peer": "8",
"pgid": "4.fa",
"last_update": "17529'85051",
"last_complete": "17529'85051",
"log_tail": "17091'75034",
"last_user_version": 85051,
"last_backfill": "MIN",
"last_backfill_bitwise": 1,
"purged_snaps": [
{
"start": "1",
"length": "3"
},
{
"start": "6",
"length": "8"
},
{
"start": "10",
"length": "2"
}
],
"history": {
"epoch_created": 9134,
"epoch_pool_created": 9134,
"last_epoch_started": 17528,
"last_interval_started": 17527,
"last_epoch_clean": 17079,
"last_interval_clean": 17078,
"last_epoch_split": 0,
"last_epoch_marked_full": 0,
"same_up_since": 17143,
"same_interval_since": 17530,
"same_primary_since": 17515,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142"
},
"stats": {
"version": "0'0",
"reported_seq": "3",
"reported_epoch": "17097",
"state": "peering",
"last_fresh": "2018-01-21 20:15:49.798018",
"last_change": "2018-01-21 20:15:49.786423",
"last_active": "0.000000",
"last_peered": "0.000000",
"last_clean": "0.000000",
"last_became_active": "0.000000",
"last_became_peered": "0.000000",
"last_unstale": "2018-01-21 20:15:49.798018",
"last_undegraded": "2018-01-21 20:15:49.798018",
"last_fullsized": "2018-01-21 20:15:49.798018",
"mapping_epoch": 17530,
"log_start": "0'0",
"ondisk_log_start": "0'0",
"created": 9134,
"last_epoch_clean": 17079,
"parent": "0.0",
"parent_split_bits": 0,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142",
"log_size": 0,
"ondisk_log_size": 0,
"stats_invalid": false,
"dirty_stats_invalid": false,
"omap_stats_invalid": false,
"hitset_stats_invalid": false,
"hitset_bytes_stats_invalid": false,
"pin_stats_invalid": false,
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0
},
"up": [
8,
17,
25
],
"acting": [
61
],
"blocked_by": [],
"up_primary": 8,
"acting_primary": 61
},
"empty": 0,
"dne": 0,
"incomplete": 1,
"last_epoch_started": 17528,
"hit_set_history": {
"current_last_update": "0'0",
"history": []
}
},
{
"peer": "17",
"pgid": "4.fa",
"last_update": "17529'85051",
"last_complete": "17529'85051",
"log_tail": "17091'75034",
"last_user_version": 85051,
"last_backfill": "MIN",
"last_backfill_bitwise": 1,
"purged_snaps": [
{
"start": "1",
"length": "3"
},
{
"start": "6",
"length": "8"
},
{
"start": "10",
"length": "2"
}
],
"history": {
"epoch_created": 9134,
"epoch_pool_created": 9134,
"last_epoch_started": 17528,
"last_interval_started": 17527,
"last_epoch_clean": 17079,
"last_interval_clean": 17078,
"last_epoch_split": 0,
"last_epoch_marked_full": 0,
"same_up_since": 17143,
"same_interval_since": 17530,
"same_primary_since": 17515,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142"
},
"stats": {
"version": "0'0",
"reported_seq": "0",
"reported_epoch": "0",
"state": "unknown",
"last_fresh": "0.000000",
"last_change": "0.000000",
"last_active": "0.000000",
"last_peered": "0.000000",
"last_clean": "0.000000",
"last_became_active": "0.000000",
"last_became_peered": "0.000000",
"last_unstale": "0.000000",
"last_undegraded": "0.000000",
"last_fullsized": "0.000000",
"mapping_epoch": 17530,
"log_start": "0'0",
"ondisk_log_start": "0'0",
"created": 0,
"last_epoch_clean": 0,
"parent": "0.0",
"parent_split_bits": 0,
"last_scrub": "0'0",
"last_scrub_stamp": "0.000000",
"last_deep_scrub": "0'0",
"last_deep_scrub_stamp": "0.000000",
"last_clean_scrub_stamp": "0.000000",
"log_size": 0,
"ondisk_log_size": 0,
"stats_invalid": false,
"dirty_stats_invalid": false,
"omap_stats_invalid": false,
"hitset_stats_invalid": false,
"hitset_bytes_stats_invalid": false,
"pin_stats_invalid": false,
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0
},
"up": [
8,
17,
25
],
"acting": [
61
],
"blocked_by": [],
"up_primary": 8,
"acting_primary": 61
},
"empty": 0,
"dne": 0,
"incomplete": 1,
"last_epoch_started": 17528,
"hit_set_history": {
"current_last_update": "0'0",
"history": []
}
},
{
"peer": "20",
"pgid": "4.fa",
"last_update": "17138'75260",
"last_complete": "17138'75260",
"log_tail": "17090'73735",
"last_user_version": 75260,
"last_backfill": "MIN",
"last_backfill_bitwise": 1,
"purged_snaps": [
{
"start": "1",
"length": "3"
},
{
"start": "6",
"length": "8"
},
{
"start": "10",
"length": "2"
}
],
"history": {
"epoch_created": 9134,
"epoch_pool_created": 9134,
"last_epoch_started": 17121,
"last_interval_started": 17120,
"last_epoch_clean": 17079,
"last_interval_clean": 17078,
"last_epoch_split": 0,
"last_epoch_marked_full": 0,
"same_up_since": 17143,
"same_interval_since": 17530,
"same_primary_since": 17515,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142"
},
"stats": {
"version": "0'0",
"reported_seq": "0",
"reported_epoch": "0",
"state": "unknown",
"last_fresh": "0.000000",
"last_change": "0.000000",
"last_active": "0.000000",
"last_peered": "0.000000",
"last_clean": "0.000000",
"last_became_active": "0.000000",
"last_became_peered": "0.000000",
"last_unstale": "0.000000",
"last_undegraded": "0.000000",
"last_fullsized": "0.000000",
"mapping_epoch": 17530,
"log_start": "0'0",
"ondisk_log_start": "0'0",
"created": 0,
"last_epoch_clean": 0,
"parent": "0.0",
"parent_split_bits": 0,
"last_scrub": "0'0",
"last_scrub_stamp": "0.000000",
"last_deep_scrub": "0'0",
"last_deep_scrub_stamp": "0.000000",
"last_clean_scrub_stamp": "0.000000",
"log_size": 0,
"ondisk_log_size": 0,
"stats_invalid": false,
"dirty_stats_invalid": false,
"omap_stats_invalid": false,
"hitset_stats_invalid": false,
"hitset_bytes_stats_invalid": false,
"pin_stats_invalid": false,
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0
},
"up": [
8,
17,
25
],
"acting": [
61
],
"blocked_by": [],
"up_primary": 8,
"acting_primary": 61
},
"empty": 0,
"dne": 0,
"incomplete": 1,
"last_epoch_started": 17121,
"hit_set_history": {
"current_last_update": "0'0",
"history": []
}
},
{
"peer": "25",
"pgid": "4.fa",
"last_update": "17529'85051",
"last_complete": "17529'85051",
"log_tail": "17091'75034",
"last_user_version": 85051,
"last_backfill": "MIN",
"last_backfill_bitwise": 1,
"purged_snaps": [
{
"start": "1",
"length": "3"
},
{
"start": "6",
"length": "8"
},
{
"start": "10",
"length": "2"
}
],
"history": {
"epoch_created": 9134,
"epoch_pool_created": 9134,
"last_epoch_started": 17528,
"last_interval_started": 17527,
"last_epoch_clean": 17079,
"last_interval_clean": 17078,
"last_epoch_split": 0,
"last_epoch_marked_full": 0,
"same_up_since": 17143,
"same_interval_since": 17530,
"same_primary_since": 17515,
"last_scrub": "17090'57357",
"last_scrub_stamp": "2018-01-20 20:45:32.616142",
"last_deep_scrub": "17082'54734",
"last_deep_scrub_stamp": "2018-01-15 21:09:34.121488",
"last_clean_scrub_stamp": "2018-01-20 20:45:32.616142"
},
"stats": {
"version": "0'0",
"reported_seq": "0",
"reported_epoch": "0",
"state": "unknown",
"last_fresh": "0.000000",
"last_change": "0.000000",
"last_active": "0.000000",
"last_peered": "0.000000",
"last_clean": "0.000000",
"last_became_active": "0.000000",
"last_became_peered": "0.000000",
"last_unstale": "0.000000",
"last_undegraded": "0.000000",
"last_fullsized": "0.000000",
"mapping_epoch": 17530,
"log_start": "0'0",
"ondisk_log_start": "0'0",
"created": 0,
"last_epoch_clean": 0,
"parent": "0.0",
"parent_split_bits": 0,
"last_scrub": "0'0",
"last_scrub_stamp": "0.000000",
"last_deep_scrub": "0'0",
"last_deep_scrub_stamp": "0.000000",
"last_clean_scrub_stamp": "0.000000",
"log_size": 0,
"ondisk_log_size": 0,
"stats_invalid": false,
"dirty_stats_invalid": false,
"omap_stats_invalid": false,
"hitset_stats_invalid": false,
"hitset_bytes_stats_invalid": false,
"pin_stats_invalid": false,
"stat_sum": {
"num_bytes": 0,
"num_objects": 0,
"num_object_clones": 0,
"num_object_copies": 0,
"num_objects_missing_on_primary": 0,
"num_objects_missing": 0,
"num_objects_degraded": 0,
"num_objects_misplaced": 0,
"num_objects_unfound": 0,
"num_objects_dirty": 0,
"num_whiteouts": 0,
"num_read": 0,
"num_read_kb": 0,
"num_write": 0,
"num_write_kb": 0,
"num_scrub_errors": 0,
"num_shallow_scrub_errors": 0,
"num_deep_scrub_errors": 0,
"num_objects_recovered": 0,
"num_bytes_recovered": 0,
"num_keys_recovered": 0,
"num_objects_omap": 0,
"num_objects_hit_set_archive": 0,
"num_bytes_hit_set_archive": 0,
"num_flush": 0,
"num_flush_kb": 0,
"num_evict": 0,
"num_evict_kb": 0,
"num_promote": 0,
"num_flush_mode_high": 0,
"num_flush_mode_low": 0,
"num_evict_mode_some": 0,
"num_evict_mode_full": 0,
"num_objects_pinned": 0,
"num_legacy_snapsets": 0
},
"up": [
8,
17,
25
],
"acting": [
61
],
"blocked_by": [],
"up_primary": 8,
"acting_primary": 61
},
"empty": 0,
"dne": 0,
"incomplete": 1,
"last_epoch_started": 17528,
"hit_set_history": {
"current_last_update": "0'0",
"history": []
}
}
],
"recovery_state": [
{
"name": "Started/Primary/Active",
"enter_time": "2018-01-22 15:00:45.664496",
"might_have_unfound": [
{
"osd": "8",
"status": "already probed"
},
{
"osd": "17",
"status": "already probed"
},
{
"osd": "20",
"status": "already probed"
},
{
"osd": "25",
"status": "already probed"
},
{
"osd": "63",
"status": "osd is down"
},
{
"osd": "64",
"status": "osd is down"
}
],
"recovery_progress": {
"backfill_targets": [
"8",
"17",
"25"
],
"waiting_on_backfill": [],
"last_backfill_started": "MIN",
"backfill_info": {
"begin": "MIN",
"end": "MIN",
"objects": []
},
"peer_backfill_info": [],
"backfills_in_flight": [],
"recovering": [],
"pg_backend": {
"pull_from_peer": [],
"pushing": []
}
},
"scrub": {
"scrubber.epoch_start": "17078",
"scrubber.active": false,
"scrubber.state": "INACTIVE",
"scrubber.start": "MIN",
"scrubber.end": "MIN",
"scrubber.subset_last_update": "0'0",
"scrubber.deep": false,
"scrubber.seed": 0,
"scrubber.waiting_on": 0,
"scrubber.waiting_on_whom": []
}
},
{
"name": "Started",
"enter_time": "2018-01-22 15:00:45.613486"
}
],
"agent_state": {}
}
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 48.94531 root default
-6 11.88382 host server1
25 hdd 4.59999 osd.25 up 1.00000 1.00000
61 hdd 4.59999 osd.61 up 1.00000 1.00000
48 ssd 0.22198 osd.48 up 1.00000 1.00000
49 ssd 0.22198 osd.49 up 1.00000 1.00000
50 ssd 0.22198 osd.50 up 1.00000 1.00000
51 ssd 0.22699 osd.51 up 1.00000 1.00000
52 ssd 0.22198 osd.52 up 1.00000 1.00000
53 ssd 0.22198 osd.53 up 1.00000 1.00000
54 ssd 0.22198 osd.54 up 1.00000 1.00000
55 ssd 0.22699 osd.55 up 1.00000 1.00000
56 ssd 0.22198 osd.56 up 1.00000 1.00000
57 ssd 0.22198 osd.57 up 1.00000 1.00000
58 ssd 0.22699 osd.58 up 1.00000 1.00000
59 ssd 0.22699 osd.59 up 1.00000 1.00000
-2 11.95193 host server2
21 hdd 4.59999 osd.21 up 1.00000 1.00000
24 hdd 4.59999 osd.24 up 1.00000 1.00000
0 ssd 0.68799 osd.0 up 1.00000 1.00000
4 ssd 0.68799 osd.4 up 1.00000 1.00000
6 ssd 0.68799 osd.6 up 1.00000 1.00000
10 ssd 0.68799 osd.10 up 1.00000 1.00000
-3 6.61287 host server3
20 hdd 4.59999 osd.20 up 1.00000 1.00000
1 ssd 0.22198 osd.1 up 1.00000 1.00000
7 ssd 0.22198 osd.7 up 1.00000 1.00000
12 ssd 0.22198 osd.12 up 1.00000 1.00000
15 ssd 0.22699 osd.15 up 1.00000 1.00000
23 ssd 0.22198 osd.23 up 1.00000 1.00000
27 ssd 0.22198 osd.27 up 1.00000 1.00000
29 ssd 0.22699 osd.29 up 1.00000 1.00000
33 ssd 0.22198 osd.33 up 1.00000 1.00000
42 ssd 0.22699 osd.42 up 1.00000 1.00000
-5 6.61287 host server4
17 hdd 4.59999 osd.17 up 1.00000 1.00000
3 ssd 0.22198 osd.3 up 1.00000 1.00000
11 ssd 0.22198 osd.11 up 1.00000 1.00000
16 ssd 0.22699 osd.16 up 1.00000 1.00000
19 ssd 0.22198 osd.19 up 1.00000 1.00000
28 ssd 0.22198 osd.28 up 1.00000 1.00000
37 ssd 0.22198 osd.37 up 1.00000 1.00000
41 ssd 0.22198 osd.41 up 1.00000 1.00000
43 ssd 0.22699 osd.43 up 1.00000 1.00000
46 ssd 0.22699 osd.46 up 1.00000 1.00000
-4 11.88382 host server5
8 hdd 4.59999 osd.8 up 1.00000 1.00000
13 hdd 4.59999 osd.13 up 1.00000 1.00000
2 ssd 0.22198 osd.2 up 1.00000 1.00000
5 ssd 0.22198 osd.5 up 1.00000 1.00000
9 ssd 0.22198 osd.9 up 1.00000 1.00000
14 ssd 0.22699 osd.14 up 1.00000 1.00000
18 ssd 0.22198 osd.18 up 1.00000 1.00000
22 ssd 0.22198 osd.22 up 1.00000 1.00000
26 ssd 0.22198 osd.26 up 1.00000 1.00000
30 ssd 0.22699 osd.30 up 1.00000 1.00000
36 ssd 0.22198 osd.36 up 1.00000 1.00000
40 ssd 0.22198 osd.40 up 1.00000 1.00000
45 ssd 0.22699 osd.45 up 1.00000 1.00000
47 ssd 0.22699 osd.47 up 1.00000 1.00000
--
Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
