Re: ceph -w question

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



OSD tree and OSD map inline. Thanks for the help. OSD 32 was removed due to a bad drive.

# id    weight  type name       up/down reweight
-1      96      root default
-3      48              rack rack_aa12
-2      4                       host osdhost001
0       1                               osd.0   up      1
1       1                               osd.1   up      1
2       1                               osd.2   up      1
3       1                               osd.3   up      1
-4      4                       host osdhost003
10      1                               osd.10  up      1
11      1                               osd.11  up      1
8       1                               osd.8   up      1
9       1                               osd.9   up      1
-5      4                       host osdhost004
12      1                               osd.12  up      1
13      1                               osd.13  up      1
14      1                               osd.14  up      1
15      1                               osd.15  up      1
-6      4                       host osdhost005
16      1                               osd.16  up      1
17      1                               osd.17  up      1
18      1                               osd.18  up      1
19      1                               osd.19  up      1
-7      4                       host osdhost006
20      1                               osd.20  up      1
21      1                               osd.21  up      1
22      1                               osd.22  up      1
23      1                               osd.23  up      1
-8      4                       host osdhost007
24      1                               osd.24  up      1
25      1                               osd.25  up      1
26      1                               osd.26  up      1
27      1                               osd.27  up      1
-9      4                       host osdhost008
28      1                               osd.28  up      1
29      1                               osd.29  up      1
30      1                               osd.30  up      1
31      1                               osd.31  up      1
-10     4                       host osdhost009
33      1                               osd.33  up      1
34      1                               osd.34  up      1
35      1                               osd.35  up      1
-11     4                       host osdhost010
36      1                               osd.36  up      1
37      1                               osd.37  up      1
38      1                               osd.38  up      1
39      1                               osd.39  up      1
-12     4                       host osdhost002
4       1                               osd.4   up      1
5       1                               osd.5   up      1
6       1                               osd.6   up      1
7       1                               osd.7   up      1
-13     4                       host osdhost011
40      1                               osd.40  up      1
41      1                               osd.41  up      1
42      1                               osd.42  up      1
43      1                               osd.43  up      1
-14     4                       host osdhost012
44      1                               osd.44  up      1
45      1                               osd.45  up      1
46      1                               osd.46  up      1
47      1                               osd.47  up      1
-27     48              rack rack_aa20
-15     4                       host osdhost013
48      1                               osd.48  up      1
49      1                               osd.49  up      1
50      1                               osd.50  up      1
51      1                               osd.51  up      1
-16     4                       host osdhost014
52      1                               osd.52  up      1
53      1                               osd.53  up      1
54      1                               osd.54  up      1
55      1                               osd.55  up      1
-17     4                       host osdhost015
56      1                               osd.56  up      1
57      1                               osd.57  up      1
58      1                               osd.58  up      1
59      1                               osd.59  up      1
-18     4                       host osdhost016
60      1                               osd.60  up      1
61      1                               osd.61  up      1
62      1                               osd.62  up      1
63      1                               osd.63  up      1
-19     4                       host osdhost017
64      1                               osd.64  up      1
65      1                               osd.65  up      1
66      1                               osd.66  up      1
67      1                               osd.67  up      1
-20     4                       host osdhost018
68      1                               osd.68  up      1
69      1                               osd.69  up      1
70      1                               osd.70  up      1
71      1                               osd.71  up      1
-21     4                       host osdhost019
72      1                               osd.72  up      1
73      1                               osd.73  up      1
74      1                               osd.74  up      1
75      1                               osd.75  up      1
-22     4                       host osdhost020
76      1                               osd.76  up      1
77      1                               osd.77  up      1
78      1                               osd.78  up      1
79      1                               osd.79  up      1
-23     4                       host osdhost021
80      1                               osd.80  up      1
81      1                               osd.81  up      1
82      1                               osd.82  up      1
83      1                               osd.83  up      1
-24     4                       host osdhost022
84      1                               osd.84  up      1
85      1                               osd.85  up      1
86      1                               osd.86  up      1
87      1                               osd.87  up      1
-25     4                       host osdhost023
88      1                               osd.88  up      1
89      1                               osd.89  up      1
90      1                               osd.90  up      1
91      1                               osd.91  up      1
-26     4                       host osdhost024
92      1                               osd.92  up      1
93      1                               osd.93  up      1
94      1                               osd.94  up      1
95      1                               osd.95  up      1


root@osdhost001:~# osdmaptool /tmp/map  --print
osdmaptool: osdmap file '/tmp/map'
epoch 2028
fsid 969f28c3-5ee1-4451-9b5b-97c52b724a06
created 2013-01-26 15:31:21.670594
modifed 2013-04-15 10:13:03.045973
flags

pool 13 '.rgw.buckets' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 4800 pgp_num 4800 last_change 1198 owner 0
pool 14 '.rgw' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 242 owner 18446744073709551615
pool 15 '.rgw.gc' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 243 owner 18446744073709551615
pool 16 '.rgw.control' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 244 owner 18446744073709551615
pool 17 '.users.uid' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 246 owner 0
pool 18 '.users.email' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 248 owner 0
pool 19 '.users' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 250 owner 0
pool 20 '.usage' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 256 owner 18446744073709551615
pool 21 '.users.swift' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 1138 owner 0

max_osd 96
osd.0 up   in  weight 1 up_from 1770 up_thru 1994 down_at 1766 last_clean_interval [940,1765) 10.198.141.21:6800/1689 10.198.142.21:6800/1689 10.198.142.21:6801/1689 exists,up 60fdecb6-fb58-4742-91f9-bc21265bcb0b
osd.1 up   in  weight 1 up_from 1771 up_thru 1994 down_at 1766 last_clean_interval [941,1765) 10.198.141.21:6801/1786 10.198.142.21:6802/1786 10.198.142.21:6803/1786 exists,up 2bb3ac97-ddae-4782-89b2-51e216e39923
osd.2 up   in  weight 1 up_from 1772 up_thru 1992 down_at 1768 last_clean_interval [941,1765) 10.198.141.21:6802/2015 10.198.142.21:6804/2015 10.198.142.21:6805/2015 exists,up 6f5671f3-503d-41fe-8b9e-895cf8ddb287
osd.3 up   in  weight 1 up_from 1772 up_thru 1994 down_at 1768 last_clean_interval [941,1765) 10.198.141.21:6803/2254 10.198.142.21:6806/2254 10.198.142.21:6807/2254 exists,up 123d88f1-901d-40ed-92cc-1cdb7e7ac530
osd.4 up   in  weight 1 up_from 1780 up_thru 1994 down_at 1774 last_clean_interval [1589,1773) 10.198.141.22:6800/2063 10.198.142.22:6800/2063 10.198.142.22:6801/2063 exists,up ffb8ba24-80a8-4716-be06-5a3dc7274bd3
osd.5 up   in  weight 1 up_from 1781 up_thru 1994 down_at 1774 last_clean_interval [1588,1773) 10.198.141.22:6801/2277 10.198.142.22:6802/2277 10.198.142.22:6803/2277 exists,up 00b83261-1262-4c56-af1c-13393459ca9a
osd.6 up   in  weight 1 up_from 1781 up_thru 1994 down_at 1775 last_clean_interval [1589,1773) 10.198.141.22:6802/2516 10.198.142.22:6804/2516 10.198.142.22:6805/2516 exists,up cf4f3f79-96b4-40fc-b724-d04ffb03a737
osd.7 up   in  weight 1 up_from 1783 up_thru 1994 down_at 1777 last_clean_interval [1589,1773) 10.198.141.22:6803/2746 10.198.142.22:6806/2746 10.198.142.22:6807/2746 exists,up 30ae4d62-516e-4548-8c9f-9aee632e8ef5
osd.8 up   in  weight 1 up_from 1793 up_thru 1994 down_at 1789 last_clean_interval [1596,1785) 10.198.141.23:6802/2765 10.198.142.23:6804/2765 10.198.142.23:6805/2765 exists,up be1553cf-ee60-4bc2-8853-cf30920521b0
osd.9 up   in  weight 1 up_from 1792 up_thru 1992 down_at 1789 last_clean_interval [1596,1785) 10.198.141.23:6803/2977 10.198.142.23:6806/2977 10.198.142.23:6807/2977 exists,up bb15ac76-7b01-45a6-8ca4-20a238d1449d
osd.10 up   in  weight 1 up_from 1791 up_thru 1994 down_at 1786 last_clean_interval [1594,1785) 10.198.141.23:6800/1637 10.198.142.23:6800/1637 10.198.142.23:6801/1637 exists,up 3e4ecf43-9de3-4c90-86d9-50d18e71d8de
osd.11 up   in  weight 1 up_from 1792 up_thru 1992 down_at 1786 last_clean_interval [1595,1785) 10.198.141.23:6801/1731 10.198.142.23:6802/1731 10.198.142.23:6803/1731 exists,up 00ec7fdd-cf91-44be-862c-0e1054e3f5b5
osd.12 up   in  weight 1 up_from 1798 up_thru 1994 down_at 1796 last_clean_interval [1601,1795) 10.198.141.24:6800/1656 10.198.142.24:6800/1656 10.198.142.24:6801/1656 exists,up cc39f155-a3c6-4499-ae4d-03eb5bafb338
osd.13 up   in  weight 1 up_from 1800 up_thru 1992 down_at 1796 last_clean_interval [1602,1795) 10.198.141.24:6801/1750 10.198.142.24:6802/1750 10.198.142.24:6803/1750 exists,up dbf0a788-bfba-417c-aec9-a14791ad5096
osd.14 up   in  weight 1 up_from 1798 up_thru 1992 down_at 1796 last_clean_interval [1601,1795) 10.198.141.24:6802/1843 10.198.142.24:6804/1843 10.198.142.24:6805/1843 exists,up ff4aa630-e637-43e6-97b8-de7033f6a271
osd.15 up   in  weight 1 up_from 1800 up_thru 1994 down_at 1796 last_clean_interval [1602,1795) 10.198.141.24:6803/1953 10.198.142.24:6806/1953 10.198.142.24:6807/1953 exists,up 04a2cfb7-a9ec-4701-8ba1-e97e3d78a321
osd.16 up   in  weight 1 up_from 1808 up_thru 1992 down_at 1803 last_clean_interval [1609,1802) 10.198.141.25:6800/1618 10.198.142.25:6800/1618 10.198.142.25:6801/1618 exists,up 6099188a-073d-4180-ba64-b12bf975986a
osd.17 up   in  weight 1 up_from 1807 up_thru 1992 down_at 1803 last_clean_interval [1610,1802) 10.198.141.25:6801/1712 10.198.142.25:6802/1712 10.198.142.25:6803/1712 exists,up 838f20dc-147a-47e7-ac63-4016d4801bdb
osd.18 up   in  weight 1 up_from 1808 up_thru 1982 down_at 1805 last_clean_interval [1610,1802) 10.198.141.25:6802/1804 10.198.142.25:6804/1804 10.198.142.25:6805/1804 exists,up dcff4b5c-ff48-4ff4-9e46-953fa185d6d8
osd.19 up   in  weight 1 up_from 1808 up_thru 1994 down_at 1805 last_clean_interval [1611,1802) 10.198.141.25:6803/1913 10.198.142.25:6806/1913 10.198.142.25:6807/1913 exists,up c29b0f83-fe81-4eac-afa5-e3b60bf910fe
osd.20 up   in  weight 1 up_from 1815 up_thru 1992 down_at 1811 last_clean_interval [1616,1810) 10.198.141.26:6800/1683 10.198.142.26:6800/1683 10.198.142.26:6801/1683 exists,up 9550a366-d28e-45d6-ab83-20825d0531a9
osd.21 up   in  weight 1 up_from 1816 up_thru 1992 down_at 1811 last_clean_interval [1615,1810) 10.198.141.26:6801/1777 10.198.142.26:6802/1777 10.198.142.26:6803/1777 exists,up aabc75c5-2d98-4ff5-a437-fdfa92e93ade
osd.22 up   in  weight 1 up_from 1817 up_thru 1994 down_at 1813 last_clean_interval [1617,1810) 10.198.141.26:6802/1869 10.198.142.26:6804/1869 10.198.142.26:6805/1869 exists,up 0a88db42-9427-4980-8d63-ed5f7adc9663
osd.23 up   in  weight 1 up_from 1816 up_thru 1994 down_at 1813 last_clean_interval [1616,1810) 10.198.141.26:6803/1990 10.198.142.26:6806/1990 10.198.142.26:6807/1990 exists,up f7aac20c-7153-498e-b076-87f69ad93c2e
osd.24 up   in  weight 1 up_from 1825 up_thru 1994 down_at 1820 last_clean_interval [1625,1819) 10.198.141.27:6800/1763 10.198.142.27:6800/1763 10.198.142.27:6801/1763 exists,up e971ae57-687b-465f-9b08-4adb12a7cf42
osd.25 up   in  weight 1 up_from 1825 up_thru 1992 down_at 1820 last_clean_interval [1624,1819) 10.198.141.27:6801/1858 10.198.142.27:6802/1858 10.198.142.27:6803/1858 exists,up 3dd19146-1fb1-4626-bfde-c50c7923e7b2
osd.26 up   in  weight 1 up_from 1826 up_thru 1994 down_at 1821 last_clean_interval [1625,1819) 10.198.141.27:6802/1950 10.198.142.27:6804/1950 10.198.142.27:6805/1950 exists,up 3adc9c6c-1d61-43e2-9c20-b277703965f7
osd.27 up   in  weight 1 up_from 1828 up_thru 1992 down_at 1823 last_clean_interval [1625,1819) 10.198.141.27:6803/2059 10.198.142.27:6806/2059 10.198.142.27:6807/2059 exists,up 2dc425b4-6d19-4624-a677-68f2be87280a
osd.28 up   in  weight 1 up_from 1835 up_thru 1992 down_at 1831 last_clean_interval [1632,1830) 10.198.141.28:6800/1786 10.198.142.28:6800/1786 10.198.142.28:6801/1786 exists,up b0c33e64-8b24-4227-b752-aec1a1cbdaa5
osd.29 up   in  weight 1 up_from 1836 up_thru 1994 down_at 1831 last_clean_interval [1633,1830) 10.198.141.28:6801/1881 10.198.142.28:6802/1881 10.198.142.28:6803/1881 exists,up 44b3a173-3749-416c-9b54-dd1020cddce0
osd.30 up   in  weight 1 up_from 1836 up_thru 1994 down_at 1831 last_clean_interval [1631,1830) 10.198.141.28:6802/1985 10.198.142.28:6804/1985 10.198.142.28:6805/1985 exists,up 2fdd18c3-370b-48a2-8567-e532a0601421
osd.31 up   in  weight 1 up_from 1836 up_thru 1992 down_at 1833 last_clean_interval [1633,1830) 10.198.141.28:6803/2095 10.198.142.28:6806/2095 10.198.142.28:6807/2095 exists,up a50985ac-2f05-488d-b119-079b1a90a38f
osd.33 up   in  weight 1 up_from 1872 up_thru 2019 down_at 1860 last_clean_interval [1845,1859) 10.198.141.29:6801/1960 10.198.142.29:6802/1960 10.198.142.29:6803/1960 exists,up 4dd53408-3b0b-4ca8-a834-eca30a152e82
osd.34 up   in  weight 1 up_from 1872 up_thru 2025 down_at 1860 last_clean_interval [1846,1859) 10.198.141.29:6802/2082 10.198.142.29:6804/2082 10.198.142.29:6805/2082 exists,up 365a4f9f-adf9-44e4-8132-71bccb313b65
osd.35 up   in  weight 1 up_from 1875 up_thru 2005 down_at 1860 last_clean_interval [1848,1859) 10.198.141.29:6803/2175 10.198.142.29:6806/2175 10.198.142.29:6807/2175 exists,up c507dd1a-8de3-4fe5-804b-f1271eab3959
osd.36 up   in  weight 1 up_from 1856 up_thru 1994 down_at 1851 last_clean_interval [1649,1850) 10.198.141.30:6800/1914 10.198.142.30:6800/1914 10.198.142.30:6801/1914 exists,up 5e536ea7-9478-4022-8f8a-954cbe299fd2
osd.37 up   in  weight 1 up_from 1857 up_thru 1991 down_at 1851 last_clean_interval [1648,1850) 10.198.141.30:6801/2008 10.198.142.30:6802/2008 10.198.142.30:6803/2008 exists,up 89f03a60-7a00-4e4c-8302-8a9defbad9b4
osd.38 up   in  weight 1 up_from 1857 up_thru 1994 down_at 1853 last_clean_interval [1649,1850) 10.198.141.30:6802/2101 10.198.142.30:6804/2101 10.198.142.30:6805/2101 exists,up ffce38bd-4b60-4620-a9db-00613a0fbd48
osd.39 up   in  weight 1 up_from 1857 up_thru 1994 down_at 1853 last_clean_interval [1650,1850) 10.198.141.30:6803/2227 10.198.142.30:6806/2227 10.198.142.30:6807/2227 exists,up 525f1028-9b4c-4751-95a3-8e4e41c0489c
osd.40 up   in  weight 1 up_from 1884 up_thru 1992 down_at 1866 last_clean_interval [1656,1865) 10.198.141.31:6800/1971 10.198.142.31:6800/1971 10.198.142.31:6801/1971 exists,up f7882b62-ae97-45e9-83de-170df085a82c
osd.41 up   in  weight 1 up_from 1884 up_thru 1994 down_at 1866 last_clean_interval [1657,1865) 10.198.141.31:6801/2066 10.198.142.31:6802/2066 10.198.142.31:6803/2066 exists,up 4a4a41bc-1be9-4e21-992f-2c9ed7c55c15
osd.42 up   in  weight 1 up_from 1887 up_thru 1992 down_at 1868 last_clean_interval [1658,1865) 10.198.141.31:6802/2175 10.198.142.31:6804/2175 10.198.142.31:6805/2175 exists,up 0cfde622-b84b-4d28-a6af-899dbc9f1a67
osd.43 up   in  weight 1 up_from 1884 up_thru 1992 down_at 1868 last_clean_interval [1657,1865) 10.198.141.31:6803/2277 10.198.142.31:6806/2277 10.198.142.31:6807/2277 exists,up dbeb8a3e-b359-4e28-bc7f-fa7677fb8e46
osd.44 up   in  weight 1 up_from 1901 up_thru 1994 down_at 1895 last_clean_interval [1667,1894) 10.198.141.32:6800/2005 10.198.142.32:6800/2005 10.198.142.32:6801/2005 exists,up 96b2f069-3628-4aed-af4e-63ef42442620
osd.45 up   in  weight 1 up_from 1900 up_thru 1994 down_at 1895 last_clean_interval [1666,1894) 10.198.141.32:6801/2099 10.198.142.32:6802/2099 10.198.142.32:6803/2099 exists,up c7e68723-df40-42fe-85a4-649ceeed8c4e
osd.46 up   in  weight 1 up_from 1901 up_thru 1994 down_at 1896 last_clean_interval [1668,1894) 10.198.141.32:6802/2192 10.198.142.32:6804/2192 10.198.142.32:6805/2192 exists,up 787ccacb-9b86-4e5f-9f5f-e381b5229647
osd.47 up   in  weight 1 up_from 1901 up_thru 1992 down_at 1898 last_clean_interval [1668,1894) 10.198.141.32:6803/2328 10.198.142.32:6806/2328 10.198.142.32:6807/2328 exists,up 2b76b5ef-61fe-45d9-b48b-e165bebe9d83
osd.48 up   in  weight 1 up_from 1910 up_thru 1927 down_at 1904 last_clean_interval [1676,1903) 10.198.141.33:6800/2101 10.198.142.33:6800/2101 10.198.142.33:6801/2101 exists,up 09bff29e-f07f-4239-942a-1ce4531a00a9
osd.49 up   in  weight 1 up_from 1908 up_thru 1927 down_at 1904 last_clean_interval [1677,1903) 10.198.141.33:6801/2195 10.198.142.33:6802/2195 10.198.142.33:6803/2195 exists,up 4fc6d620-5801-4000-a37a-debcbbafb55b
osd.50 up   in  weight 1 up_from 1911 up_thru 2012 down_at 1906 last_clean_interval [1679,1903) 10.198.141.33:6802/2299 10.198.142.33:6804/2299 10.198.142.33:6805/2299 exists,up 9158613e-449c-484f-9f07-23e776f5f7e4
osd.51 up   in  weight 1 up_from 1912 up_thru 1927 down_at 1906 last_clean_interval [1677,1903) 10.198.141.33:6803/2410 10.198.142.33:6806/2410 10.198.142.33:6807/2410 exists,up 834e9c58-f956-4edb-b5be-478e78ed3b7a
osd.52 up   in  weight 1 up_from 1920 up_thru 2020 down_at 1914 last_clean_interval [1685,1913) 10.198.141.34:6800/2108 10.198.142.34:6800/2108 10.198.142.34:6801/2108 exists,up e8d66816-8ce6-4a7c-8b0a-e535fa5c12ad
osd.53 up   in  weight 1 up_from 1920 up_thru 1928 down_at 1914 last_clean_interval [1685,1913) 10.198.141.34:6801/2202 10.198.142.34:6802/2202 10.198.142.34:6803/2202 exists,up 25dca904-a617-4504-8857-c13c8f31b758
osd.54 up   in  weight 1 up_from 1919 up_thru 1919 down_at 1914 last_clean_interval [1684,1913) 10.198.141.34:6802/2312 10.198.142.34:6804/2312 10.198.142.34:6805/2312 exists,up ad65b25d-2fb9-471e-9cd2-8c08ff86f6d3
osd.55 up   in  weight 1 up_from 1919 up_thru 1926 down_at 1917 last_clean_interval [1685,1913) 10.198.141.34:6803/2414 10.198.142.34:6806/2414 10.198.142.34:6807/2414 exists,up ff0b8f74-8f4d-47eb-a008-8229981f01f7
osd.56 up   in  weight 1 up_from 1935 up_thru 1940 down_at 1922 last_clean_interval [1692,1921) 10.198.141.35:6800/2234 10.198.142.35:6800/2234 10.198.142.35:6801/2234 exists,up 88f2a234-1273-4519-9598-56a071250231
osd.57 up   in  weight 1 up_from 1931 up_thru 1931 down_at 1922 last_clean_interval [1691,1921) 10.198.141.35:6801/2328 10.198.142.35:6802/2328 10.198.142.35:6803/2328 exists,up 2c8f0562-24d8-4bb7-87a6-f58cc125f824
osd.58 up   in  weight 1 up_from 1936 up_thru 1942 down_at 1922 last_clean_interval [1693,1921) 10.198.141.35:6802/2421 10.198.142.35:6804/2421 10.198.142.35:6805/2421 exists,up 687d3a7d-a0f5-4854-acf4-d66695f58e66
osd.59 up   in  weight 1 up_from 1933 up_thru 1952 down_at 1924 last_clean_interval [1692,1921) 10.198.141.35:6803/2557 10.198.142.35:6806/2557 10.198.142.35:6807/2557 exists,up 794d2daf-3f3c-4dda-93c8-4ca4d4369aac
osd.60 up   in  weight 1 up_from 1967 up_thru 1967 down_at 1955 last_clean_interval [1700,1954) 10.198.141.36:6800/2283 10.198.142.36:6800/2283 10.198.142.36:6801/2283 exists,up bbed85bb-7096-4ea6-8df9-04c1a48a0021
osd.61 up   in  weight 1 up_from 1968 up_thru 1968 down_at 1955 last_clean_interval [1701,1954) 10.198.141.36:6801/2378 10.198.142.36:6802/2378 10.198.142.36:6803/2378 exists,up 2e3fc7a7-71b0-477d-a258-e4f7646a6e32
osd.62 up   in  weight 1 up_from 1967 up_thru 1967 down_at 1956 last_clean_interval [1701,1954) 10.198.141.36:6802/2471 10.198.142.36:6804/2471 10.198.142.36:6805/2471 exists,up e2b88a19-da9d-4bc4-b89b-4d467a5da927
osd.63 up   in  weight 1 up_from 1968 up_thru 1977 down_at 1956 last_clean_interval [1701,1954) 10.198.141.36:6803/2580 10.198.142.36:6806/2580 10.198.142.36:6807/2580 exists,up 892ed9bb-a87e-49ce-a6ea-ef434283b9a0
osd.64 up   in  weight 1 up_from 1991 up_thru 1991 down_at 1979 last_clean_interval [1708,1978) 10.198.141.37:6800/2317 10.198.142.37:6800/2317 10.198.142.37:6801/2317 exists,up 0028d026-0c49-470a-921e-237a820ebbf6
osd.65 up   in  weight 1 up_from 1992 up_thru 1992 down_at 1979 last_clean_interval [1708,1978) 10.198.141.37:6801/2411 10.198.142.37:6802/2411 10.198.142.37:6803/2411 exists,up 357c43bb-bb3e-4ff8-b747-0047c1a03996
osd.66 up   in  weight 1 up_from 1992 up_thru 1992 down_at 1982 last_clean_interval [1708,1978) 10.198.141.37:6802/2503 10.198.142.37:6804/2503 10.198.142.37:6805/2503 exists,up 1b45e8fb-ecc5-4fd1-a24a-9b5a7921bb8c
osd.67 up   in  weight 1 up_from 1994 up_thru 1994 down_at 1982 last_clean_interval [1707,1978) 10.198.141.37:6803/2622 10.198.142.37:6806/2622 10.198.142.37:6807/2622 exists,up df199549-ccec-467e-b1a5-635dbca53497
osd.68 up   in  weight 1 up_from 1717 up_thru 1928 down_at 1710 last_clean_interval [1540,1709) 10.198.141.38:6800/2305 10.198.142.38:6800/2305 10.198.142.38:6801/2305 exists,up 3374c038-cbbe-4989-82d0-d0d438e11da9
osd.69 up   in  weight 1 up_from 1717 up_thru 2014 down_at 1711 last_clean_interval [1540,1709) 10.198.141.38:6801/2400 10.198.142.38:6802/2400 10.198.142.38:6803/2400 exists,up 387c4522-f15b-4692-aee8-7317e68b0cb8
osd.70 up   in  weight 1 up_from 1715 up_thru 1926 down_at 1711 last_clean_interval [1539,1709) 10.198.141.38:6802/2504 10.198.142.38:6804/2504 10.198.142.38:6805/2504 exists,up fc362184-4b8e-4680-b189-151a030370b4
osd.71 up   in  weight 1 up_from 1717 up_thru 2024 down_at 1713 last_clean_interval [1540,1709) 10.198.141.38:6803/2623 10.198.142.38:6806/2623 10.198.142.38:6807/2623 exists,up e32596ae-9021-4b12-8a59-86e190d939db
osd.72 up   in  weight 1 up_from 1725 up_thru 1927 down_at 1719 last_clean_interval [1546,1718) 10.198.141.39:6800/2310 10.198.142.39:6800/2310 10.198.142.39:6801/2310 exists,up a8279024-9b81-4824-bc7d-52afa2db4516
osd.73 up   in  weight 1 up_from 1724 up_thru 1901 down_at 1719 last_clean_interval [1545,1718) 10.198.141.39:6801/2405 10.198.142.39:6802/2405 10.198.142.39:6803/2405 exists,up 95502db5-4c4b-4b77-bf76-9e2ef6e6fb9a
osd.74 up   in  weight 1 up_from 1725 up_thru 1927 down_at 1720 last_clean_interval [1547,1718) 10.198.141.39:6802/2497 10.198.142.39:6804/2497 10.198.142.39:6805/2497 exists,up 9aad6a95-a671-4f6f-8489-d4b4a62ba27f
osd.75 up   in  weight 1 up_from 1726 up_thru 1927 down_at 1722 last_clean_interval [1547,1718) 10.198.141.39:6803/2606 10.198.142.39:6806/2606 10.198.142.39:6807/2606 exists,up fe3d0639-824f-49b2-9126-47b881471b2d
osd.76 up   in  weight 1 up_from 1732 up_thru 2017 down_at 1728 last_clean_interval [1556,1727) 10.198.141.40:6800/2409 10.198.142.40:6800/2409 10.198.142.40:6801/2409 exists,up 0eb886e3-8080-4d2c-b45c-02d1bfcde894
osd.77 up   in  weight 1 up_from 1733 up_thru 1927 down_at 1728 last_clean_interval [1555,1727) 10.198.141.40:6801/2503 10.198.142.40:6802/2503 10.198.142.40:6803/2503 exists,up 1cc21081-ca09-4875-865e-5938da044096
osd.78 up   in  weight 1 up_from 1733 up_thru 2008 down_at 1728 last_clean_interval [1556,1727) 10.198.141.40:6802/2596 10.198.142.40:6804/2596 10.198.142.40:6805/2596 exists,up 87910bf8-85f5-4632-a6bc-f540cbeb3699
osd.79 up   in  weight 1 up_from 1733 up_thru 1901 down_at 1730 last_clean_interval [1554,1727) 10.198.141.40:6803/2714 10.198.142.40:6806/2714 10.198.142.40:6807/2714 exists,up 9fb83231-b14f-4a8e-9958-33ad47138050
osd.80 up   in  weight 1 up_from 1738 up_thru 2027 down_at 1735 last_clean_interval [1565,1734) 10.198.141.41:6800/2425 10.198.142.41:6800/2425 10.198.142.41:6801/2425 exists,up a582d3d5-49f7-4f72-9c50-20b940ff3ce0
osd.81 up   in  weight 1 up_from 1738 up_thru 1927 down_at 1735 last_clean_interval [1563,1734) 10.198.141.41:6801/2519 10.198.142.41:6802/2519 10.198.142.41:6803/2519 exists,up fe1f41cd-eb44-4532-9e14-bdb84bb0707d
osd.82 up   in  weight 1 up_from 1739 up_thru 1926 down_at 1736 last_clean_interval [1564,1734) 10.198.141.41:6802/2629 10.198.142.41:6804/2629 10.198.142.41:6805/2629 exists,up d708e2bd-4413-45b4-bb0d-ff06492db571
osd.83 up   in  weight 1 up_from 1741 up_thru 1927 down_at 1736 last_clean_interval [1565,1734) 10.198.141.41:6803/2732 10.198.142.41:6806/2732 10.198.142.41:6807/2732 exists,up 8d0508a1-7fe7-43e4-a679-1a0a552d3da3
osd.84 up   in  weight 1 up_from 1748 up_thru 1901 down_at 1743 last_clean_interval [1572,1742) 10.198.141.42:6800/2528 10.198.142.42:6800/2528 10.198.142.42:6801/2528 exists,up 18bf317c-7b34-4028-9d4e-7100c72a5308
osd.85 up   in  weight 1 up_from 1749 up_thru 1926 down_at 1743 last_clean_interval [1573,1742) 10.198.141.42:6801/2622 10.198.142.42:6802/2622 10.198.142.42:6803/2622 exists,up 440a4b77-59a0-484b-9f29-ea798860e8d5
osd.86 up   in  weight 1 up_from 1749 up_thru 1926 down_at 1744 last_clean_interval [1574,1742) 10.198.141.42:6802/2715 10.198.142.42:6804/2715 10.198.142.42:6805/2715 exists,up f506fbd4-b41c-4bc8-9cfd-4909f02c1ffd
osd.87 up   in  weight 1 up_from 1749 up_thru 2023 down_at 1746 last_clean_interval [1572,1742) 10.198.141.42:6803/2834 10.198.142.42:6806/2834 10.198.142.42:6807/2834 exists,up 922794f2-827b-467c-a18d-685716deb9c1
osd.88 up   in  weight 1 up_from 1755 up_thru 1927 down_at 1751 last_clean_interval [1579,1750) 10.198.141.43:6800/2561 10.198.142.43:6800/2561 10.198.142.43:6801/2561 exists,up f2931004-62c5-4ba4-8fd6-c38ad9c97459
osd.89 up   in  weight 1 up_from 1755 up_thru 2010 down_at 1751 last_clean_interval [1578,1750) 10.198.141.43:6801/2675 10.198.142.43:6802/2675 10.198.142.43:6803/2675 exists,up 79fd1a48-00d2-4655-8334-0e67196a5f27
osd.90 up   in  weight 1 up_from 1755 up_thru 1927 down_at 1753 last_clean_interval [1581,1750) 10.198.141.43:6802/2791 10.198.142.43:6804/2791 10.198.142.43:6805/2791 exists,up 372fb901-4f16-42c6-8e56-cd3c6bd4bac1
osd.91 up   in  weight 1 up_from 1756 up_thru 1927 down_at 1753 last_clean_interval [1580,1750) 10.198.141.43:6803/2883 10.198.142.43:6806/2883 10.198.142.43:6807/2883 exists,up 13c8cd96-45ea-42ed-a7ac-64dcee08d9c2
osd.92 up   in  weight 1 up_from 1762 up_thru 1927 down_at 1758 last_clean_interval [1134,1757) 10.198.141.44:6800/2670 10.198.142.44:6800/2670 10.198.142.44:6801/2670 exists,up b9515f45-f827-4961-8c33-b0970a50828e
osd.93 up   in  weight 1 up_from 1764 up_thru 1927 down_at 1758 last_clean_interval [1135,1757) 10.198.141.44:6801/2763 10.198.142.44:6802/2763 10.198.142.44:6803/2763 exists,up 7f712a96-063e-429b-aa53-303c0f4618de
osd.94 up   in  weight 1 up_from 1763 up_thru 2018 down_at 1758 last_clean_interval [1135,1757) 10.198.141.44:6802/2856 10.198.142.44:6804/2856 10.198.142.44:6805/2856 exists,up b3daae82-6451-4fc0-8fbf-9212b774bac0
osd.95 up   in  weight 1 up_from 1764 up_thru 1927 down_at 1760 last_clean_interval [1135,1757) 10.198.141.44:6803/2948 10.198.142.44:6806/2948 10.198.142.44:6807/2948 exists,up 63911b22-01c4-45b1-95de-97a462f8581f

pg_temp 13.57 [75,33,35]
pg_temp 13.1b8 [94,33,35]
pg_temp 13.234 [69,35,34]
pg_temp 13.2d5 [63,34,33]
pg_temp 13.2db [51,35,34]
pg_temp 13.30a [81,35,34]
pg_temp 13.3ce [35,33,60]
pg_temp 13.400 [74,34,35]
pg_temp 13.46b [35,34,59]
pg_temp 13.47c [90,35,34]
pg_temp 13.4a9 [91,35,34]
pg_temp 13.533 [91,33,34]
pg_temp 13.535 [95,34,33]
pg_temp 13.694 [34,35,66]
pg_temp 13.6d6 [51,33,35]
pg_temp 13.6e0 [74,33,34]
pg_temp 13.6f2 [83,34,35]
pg_temp 13.71c [35,34,65]
pg_temp 13.7e6 [87,33,34]
pg_temp 13.812 [63,35,33]
pg_temp 13.84b [68,34,33]
pg_temp 13.8e0 [88,34,35]
pg_temp 13.8fb [48,35,34]
pg_temp 13.917 [91,33,35]
pg_temp 13.978 [77,35,34]
pg_temp 13.a1b [49,33,35]
pg_temp 13.a57 [51,35,33]
pg_temp 13.a6e [87,35,33]
pg_temp 13.bfb [33,34,56]
pg_temp 13.bff [95,33,35]
pg_temp 13.c33 [81,35,34]
pg_temp 13.c62 [88,33,35]
pg_temp 13.cf0 [94,35,34]
pg_temp 13.d34 [93,35,33]
pg_temp 13.d3b [83,33,35]
pg_temp 13.d53 [95,34,35]
pg_temp 13.dd5 [93,33,35]
pg_temp 13.df0 [68,33,34]
pg_temp 13.e58 [50,33,35]
pg_temp 13.eba [94,35,33]
pg_temp 13.f3d [53,34,35]
pg_temp 13.f5b [34,33,57]
pg_temp 13.f73 [49,33,34]
pg_temp 13.fa7 [72,35,34]
pg_temp 13.10d4 [63,35,33]
pg_temp 13.1107 [53,35,34]
pg_temp 13.1115 [91,33,34]
pg_temp 13.1138 [35,34,66]
pg_temp 13.1191 [34,35,59]
pg_temp 13.1199 [34,33,58]
pg_temp 13.123d [50,35,33]

Nelson Jeppesen
   Disney Technology Solutions and Services
   Phone 206-588-5001


-----Original Message-----
From: Samuel Just [mailto:sam.just@xxxxxxxxxxx]
Sent: Monday, April 15, 2013 10:11 AM
To: Jeppesen, Nelson
Cc: Gregory Farnum; ceph-users@xxxxxxxxxxxxxx
Subject: Re:  ceph -w question

Also the ceph osdmap.  (ceph osd getmap -o /tmp/map will put the osdmap in /tmp/map).
-Sam

On Mon, Apr 15, 2013 at 10:09 AM, Samuel Just <sam.just@xxxxxxxxxxx> wrote:
> Can you post the output of ceph osd tree?
> -Sam
>
> On Mon, Apr 15, 2013 at 9:52 AM, Jeppesen, Nelson
> <Nelson.Jeppesen@xxxxxxxxxx> wrote:
>> Thanks for the help but how do I track down this issue? If data is inaccessible, that's a very bad thing given this is production.
>>
>> # ceph osd dump | grep pool
>> pool 13 '.rgw.buckets' rep size 2 crush_ruleset 0 object_hash
>> rjenkins pg_num 4800 pgp_num 4800 last_change 1198 owner 0 pool 14
>> '.rgw' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8
>> pgp_num 8 last_change 242 owner 18446744073709551615 pool 15
>> '.rgw.gc' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8
>> pgp_num 8 last_change 243 owner 18446744073709551615 pool 16
>> '.rgw.control' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num
>> 8 pgp_num 8 last_change 244 owner 18446744073709551615 pool 17
>> '.users.uid' rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8
>> pgp_num 8 last_change 246 owner 0 pool 18 '.users.email' rep size 2
>> crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change
>> 248 owner 0 pool 19 '.users' rep size 2 crush_ruleset 0 object_hash
>> rjenkins pg_num 8 pgp_num 8 last_change 250 owner 0 pool 20 '.usage'
>> rep size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8
>> last_change 256 owner 18446744073709551615 pool 21 '.users.swift' rep
>> size 2 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8
>> last_change 1138 owner 0
>>
>> Nelson Jeppesen
>>    Disney Technology Solutions and Services
>>    Phone 206-588-5001
>>
>> -----Original Message-----
>> From: Gregory Farnum [mailto:greg@xxxxxxxxxxx]
>> Sent: Monday, April 15, 2013 9:34 AM
>> To: Jeppesen, Nelson
>> Cc: ceph-users@xxxxxxxxxxxxxx
>> Subject: Re:  ceph -w question
>>
>> "Incomplete" means that there are fewer than the minimum copies of the placement group (by default, half of the requested size, rounded up).
>> In general rebooting one node shouldn't do that unless you've changed your minimum size on the pool, and it does mean that data in those PGs is unaccessible.
>> -Greg
>> Software Engineer #42 @ http://inktank.com | http://ceph.com
>>
>>
>> On Mon, Apr 15, 2013 at 9:01 AM, Jeppesen, Nelson <Nelson.Jeppesen@xxxxxxxxxx> wrote:
>>> When I reboot any node in my prod environment with no activity I see
>>> incomplete pgs. Is that a concern? Does that mean some data is unavailable?
>>> Thank you.
>>>
>>>
>>>
>>> # ceph -v
>>>
>>> ceph version 0.56.4 (63b0f854d1cef490624de5d6cf9039735c7de5ca)
>>>
>>>
>>>
>>> # ceph -w
>>>
>>> 2013-04-15 08:57:27.712065 mon.0 [INF] pgmap v585220: 4864 pgs: 4443
>>> active+clean, 1 active+degraded, 420 incomplete; 3177 GB data, 6504
>>> active+GB used,
>>> 38186 GB / 44691 GB avail; 252/8168154 degraded (0.003%)
>>>
>>>
>>> _______________________________________________
>>> ceph-users mailing list
>>> ceph-users@xxxxxxxxxxxxxx
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>>
>> _______________________________________________
>> ceph-users mailing list
>> ceph-users@xxxxxxxxxxxxxx
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux