Hi Aaron,
Thank you for the pointer, I've increased the pg_num and pgp_num and this has resolved the issue! Spot on, I'm going to aim for 100 PG's per OSD, but I've recently expanded the cluster, and neglected to increase the numbers (hence this issue).
Sherry, Aaron, thanks again for your time!
Alex
From: "Aaron Ten Clay" <aarontc@xxxxxxxxxxx>
To: "Alex Pearson" <alex@xxxxxxxxxxx>
Cc: "Sherry Shahbazi" <shoosah@xxxxxxxxx>, ceph-users@xxxxxxxxxxxxxx
Sent: Monday, 13 January, 2014 10:25:59 PM
Subject: Re: [ceph-users] Cluster degraded
Another thing to check is that your "pg_num" and "pgp_num" match for the pool settings, since both affect the CRUSH placement rules.
HTH,
-Aaron
--
Aaron Ten Clay
http://www.aarontc.com/
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
To: "Alex Pearson" <alex@xxxxxxxxxxx>
Cc: "Sherry Shahbazi" <shoosah@xxxxxxxxx>, ceph-users@xxxxxxxxxxxxxx
Sent: Monday, 13 January, 2014 10:25:59 PM
Subject: Re: [ceph-users] Cluster degraded
Alex,
Since it appears at first glance that all your stuck placement groups are in pools 6 and 9, perhaps those pools have too few placement groups? I imagine that's what Sherry meant to say - there shouldn't be a problem with too few OSDs as far as I am aware, but too few placement groups can cause the CRUSH algorithm to fail to locate a place for all the objects. The recommendation is "about" 100 placement groups per OSD (across all pools), but there is some realistic minimum per pool as well. Someone with more hard facts in that domain might be able to chime in here. :)-Aaron
On Mon, Jan 13, 2014 at 3:48 AM, Alex Pearson <alex@xxxxxxxxxxx> wrote:
Sherry,Thank you for the quick reply, but I'm not sure that is the case? Surely you don't need to more OSD's than pools? Could you reference some documentation detailing that? Also when building the cluster you have the three default pools (data, metadata and rbd) and that came up cleanly with a size of two and only two OSDs?Thanks againAlexFrom: "Sherry Shahbazi" <shoosah@xxxxxxxxx>
To: "Alex Pearson" <alex@xxxxxxxxxxx>, ceph-users@xxxxxxxxxxxxxx
Sent: Sunday, 12 January, 2014 10:34:58 PM
Subject: Re: Cluster degradedHi Alex,I think you need to increase number of OSDs, since you have got 10 OSDs for 10 pools which is too few.Best RegardsSherry
On Monday, January 13, 2014 11:27 AM, Alex Pearson <alex@xxxxxxxxxxx> wrote:
Hello All,
I've had a look through the documentation, however I'm unable to find out how to get my PGs back to 'healthy'. I caused the below state by issuing multiple reweight-by-utilization commands (I believe). I have also changed the 'size' of a pool (from 2 to 3) which may also be relevant.Everything seems to be running fine at the moment, but the cluster has a number of PGs showing unhealthy as below. What would be the correct way to get the cluster back healthy? I've reviewed http://ceph.com/docs/master/rados/operations/monitoring-osd-pg/ but can't see what the correct action should be.Thanks in advance for any help on this.Alexroot@osh1:~# ceph --cluster apics -s
cluster 1f8d996d-2fc1-4663-93b2-5e3155ddb3b8
health HEALTH_WARN 49 pgs degraded; 176 pgs stuck unclean; recovery 19479/2079561 objects degraded (0.937%)
monmap e3: 3 mons at {dog=MailScanner has detected a possible fraud attempt from "10.30.10.61:6789" claiming to be MailScanner warning: numerical links are often malicious: 10.30.10.61:6789/0,osh1=10.30.10.101:6789/0,osh2=10.30.10.102:6789/0}, election epoch 1522, quorum 0,1,2 dog,osh1,osh2
mdsmap e294: 1/1/1 up {0=0=up:active}
osdmap e1605: 10 osds: 10 up, 10 in
pgmap v373901: 4528 pgs, 10 pools, 2531 GB data, 982 kobjects
6212 GB used, 2916 GB / 9305 GB avail
19479/2079561 objects degraded (0.937%)
3 active
4329 active+clean
147 active+remapped
26 active+degraded
23 active+clean+degraded
client io 35388 B/s wr, 4 op/sroot@osh1:~# ceph --cluster apics health detail
HEALTH_WARN 49 pgs degraded; 176 pgs stuck unclean; recovery 19479/2079561 objects degraded (0.937%)
pg 9.63 is stuck unclean for 34587.441837, current state active+degraded, last acting [8,2]
pg 6.12d is stuck unclean for 189490.064360, current state active+remapped, last acting [2,8,3]
pg 9.61 is stuck unclean for 25930.260406, current state active+remapped, last acting [3,6,5]
pg 6.1ec is stuck unclean since forever, current state active+remapped, last acting [0,5,2]
pg 6.12e is stuck unclean for 187564.076457, current state active+remapped, last acting [5,0,8]
pg 6.1ef is stuck unclean since forever, current state active+remapped, last acting [0,3,1]
pg 6.60 is stuck unclean for 181657.261045, current state active+remapped, last acting [0,2,1]
pg 9.6f is stuck unclean for 82751.234847, current state active+degraded, last acting [2,0]
pg 6.62 is stuck unclean since forever, current state active+remapped, last acting [4,0,1]
pg 6.122 is stuck unclean for 182927.362958, current state active+remapped, last acting [6,3,1]
pg 6.64 is stuck unclean since forever, current state active, last acting [3,6,5]
pg 6.1e2 is stuck unclean for 187564.120447, current state active+remapped, last acting [5,8,9]
pg 6.67 is stuck unclean for 187564.241127, current state active+remapped, last acting [8,2,3]
pg 6.66 is stuck unclean since forever, current state active+remapped, last acting [8,2,5]
pg 6.58 is stuck unclean for 181656.731823, current state active+remapped, last acting [3,0,1]
pg 9.57 is stuck unclean for 82751.779108, current state active+degraded, last acting [0,2]
pg 6.1e6 is stuck unclean since forever, current state active+remapped, last acting [5,0,8]
pg 6.5a is stuck unclean for 182926.684982, current state active+remapped, last acting [0,2,1]
pg 6.1db is stuck unclean for 181657.277903, current state active+remapped, last acting [0,4,1]
pg 6.11c is stuck unclean for 182927.022009, current state active+remapped, last acting [5,0,1]
pg 6.1dc is stuck unclean for 187564.253602, current state active+remapped, last acting [8,3,5]
pg 9.5f is stuck unclean for 4454.417475, current state active+degraded, last acting [4,0]
pg 6.111 is stuck unclean for 189488.047268, current state active+remapped, last acting [5,6,0]
pg 6.110 is stuck unclean for 182926.646916, current state active+remapped, last acting [2,8,3]
pg 9.5d is stuck unclean for 82751.778493, current state active+degraded, last acting [0,2]
pg 6.113 is stuck unclean for 199785.387076, current state active+remapped, last acting [5,0,1]
pg 6.117 is stuck unclean for 181657.273579, current state active+remapped, last acting [0,2,1]
pg 6.109 is stuck unclean since forever, current state active+remapped, last acting [2,8,5]
pg 6.10a is stuck unclean for 181657.028166, current state active+remapped, last acting [2,8,5]
pg 6.4f is stuck unclean for 181657.256397, current state active+remapped, last acting [0,4,1]
pg 6.4e is stuck unclean since forever, current state active+remapped, last acting [3,8,1]
pg 9.41 is stuck unclean for 95426.196707, current state active+degraded, last acting [2,8]
pg 6.10f is stuck unclean since forever, current state active+remapped, last acting [6,3,1]
pg 6.1cc is stuck unclean for 189489.856555, current state active+remapped, last acting [6,2,5]
pg 6.41 is stuck unclean for 207151.663867, current state active+remapped, last acting [2,8,5]
pg 6.1ce is stuck unclean for 189490.143064, current state active+remapped, last acting [2,0,9]
pg 9.4c is stuck unclean for 4514.849807, current state active+degraded, last acting [0,4]
pg 6.103 is stuck unclean since forever, current state active+remapped, last acting [5,6,1]
pg 6.45 is stuck unclean for 189489.763856, current state active+remapped, last acting [3,8,9]
pg 6.1c3 is stuck unclean since forever, current state active+remapped, last acting [5,6,8]
pg 6.44 is stuck unclean for 189490.015875, current state active+remapped, last acting [2,8,3]
pg 6.1c2 is stuck unclean since forever, current state active+remapped, last acting [2,8,3]
pg 6.1c5 is stuck unclean for 182927.082080, current state active+remapped, last acting [5,6,1]
pg 6.1c6 is stuck unclean for 181657.041183, current state active+remapped, last acting [2,8,5]
pg 9.32 is stuck unclean for 82751.773922, current state active+remapped, last acting [0,2,3]
pg 6.1bc is stuck unclean for 181657.280292, current state active+remapped, last acting [0,2,1]
pg 6.31 is stuck unclean since forever, current state active+remapped, last acting [3,6,0]
pg 9.3e is stuck unclean for 31943.087354, current state active+degraded, last acting [2,8]
pg 6.fe is stuck unclean for 181656.557890, current state active+remapped, last acting [8,3,1]
pg 6.f0 is stuck unclean for 187564.186472, current state active+remapped, last acting [3,0,9]
pg 6.1b0 is stuck unclean for 189488.053293, current state active+remapped, last acting [5,8,9]
pg 6.35 is stuck unclean for 187086.122285, current state active+remapped, last acting [0,2,9]
pg 6.1b2 is stuck unclean for 199788.285859, current state active+remapped, last acting [5,0,6]
pg 6.29 is stuck unclean for 182927.068290, current state active+remapped, last acting [8,5,1]
pg 6.28 is stuck unclean for 181657.032858, current state active+remapped, last acting [5,0,1]
pg 6.e9 is stuck unclean since forever, current state active+remapped, last acting [2,6,3]
pg 6.e8 is stuck unclean for 182926.645708, current state active+remapped, last acting [2,6,3]
pg 6.1a8 is stuck unclean for 181657.279948, current state active+remapped, last acting [0,5,1]
pg 6.2c is stuck unclean for 182927.006868, current state active+remapped, last acting [3,0,1]
pg 9.23 is stuck unclean for 4454.412192, current state active+degraded, last acting [4,6]
pg MailScanner has detected a possible fraud attempt from "6.ec" claiming to be MailScanner warning: numerical links are often malicious: 6.ec is stuck unclean for 182927.360674, current state active+remapped, last acting [6,2,3]
pg 6.2e is stuck unclean since forever, current state active+remapped, last acting [2,6,5]
pg 6.1ae is stuck unclean for 182927.247745, current state active+remapped, last acting [4,0,1]
pg 6.1a1 is stuck unclean for 189489.814916, current state active+remapped, last acting [3,0,9]
pg 6.22 is stuck unclean for 189489.929437, current state active+remapped, last acting [2,8,4]
pg 6.24 is stuck unclean for 181656.550140, current state active+remapped, last acting [8,3,1]
pg 9.2b is stuck unclean for 173816.252908, current state active+degraded, last acting [2,6]
pg 6.1a5 is stuck unclean since forever, current state active+remapped, last acting [3,6,9]
pg 6.26 is stuck unclean for 187564.099158, current state active+remapped, last acting [4,6,8]
pg 9.17 is stuck unclean for 25930.040655, current state active+remapped, last acting [6,2,7]
pg 6.1a6 is stuck unclean for 187564.104902, current state active+remapped, last acting [5,6,9]
pg 6.199 is stuck unclean for 182927.022250, current state active+remapped, last acting [3,6,1]
pg 6.1a is stuck unclean for 25930.023296, current state active+remapped, last acting [6,2,7]
pg 6.da is stuck unclean for 187564.245796, current state active+remapped, last acting [8,5,9]
pg 6.1c is stuck unclean since forever, current state active+remapped, last acting [6,2,5]
pg 6.19d is stuck unclean since forever, current state active+remapped, last acting [8,2,3]
pg 9.1f is stuck unclean for 159167.265649, current state active+degraded, last acting [2,8]
pg 6.d1 is stuck unclean for 182927.018091, current state active+remapped, last acting [5,8,0]
pg 6.19e is stuck unclean for 182927.080993, current state active+remapped, last acting [8,4,1]
pg 6.13 is stuck unclean for 187084.294984, current state active+remapped, last acting [3,0,9]
pg 6.191 is stuck unclean for 189489.855254, current state active+remapped, last acting [6,3,9]
pg 6.d2 is stuck unclean for 182927.018037, current state active+remapped, last acting [5,0,1]
pg 6.250 is stuck unclean for 182927.034444, current state active+remapped, last acting [3,6,0]
pg 6.d4 is stuck unclean for 182927.013012, current state active+remapped, last acting [3,6,1]
pg 6.252 is stuck unclean since forever, current state active+remapped, last acting [2,0,1]
pg 9.19 is stuck unclean for 82751.368622, current state active+degraded, last acting [6,2]
pg 6.c9 is stuck unclean for 187086.138042, current state active+remapped, last acting [8,3,9]
pg 6.196 is stuck unclean for 182927.021856, current state active+remapped, last acting [3,6,1]
pg 6.257 is stuck unclean for 182926.944790, current state active+remapped, last acting [0,5,1]
pg 6.249 is stuck unclean for 182927.381620, current state active+remapped, last acting [6,2,4]
pg 6.248 is stuck unclean since forever, current state active+remapped, last acting [6,2,0]
pg 6.18a is stuck unclean since forever, current state active+remapped, last acting [3,6,1]
pg 6.24b is stuck unclean for 182926.541677, current state active+remapped, last acting [2,8,5]
pg 6.cc is stuck unclean for 182927.171805, current state active+remapped, last acting [4,0,1]
pg 9.c3 is stuck unclean for 4469.112833, current state active+degraded, last acting [4,6]
pg 6.e is stuck unclean for 181656.740682, current state active+remapped, last acting [2,6,3]
pg 9.1 is stuck unclean for 68383.647655, current state active+remapped, last acting [2,0,5]
pg 6.24c is stuck unclean for 182927.032144, current state active+remapped, last acting [3,6,0]
pg 6.c0 is stuck unclean for 187564.230695, current state active+remapped, last acting [6,5,3]
pg 6.24e is stuck unclean for 181657.258973, current state active+remapped, last acting [6,3,1]
pg 6.2 is stuck unclean for 181657.231274, current state active+remapped, last acting [6,5,1]
pg 6.4 is stuck unclean since forever, current state active, last acting [2,0,5]
pg 9.b is stuck unclean for 160369.847661, current state active+degraded, last acting [2,6]
pg 6.c5 is stuck unclean for 182927.018313, current state active+remapped, last acting [5,8,1]
pg 6.c6 is stuck unclean for 182927.171553, current state active+remapped, last acting [4,6,1]
pg 6.246 is stuck unclean for 189490.025903, current state active+remapped, last acting [0,3,9]
pg MailScanner has detected a possible fraud attempt from "6.bd" claiming to be MailScanner warning: numerical links are often malicious: 6.bd is stuck unclean since forever, current state active+remapped, last acting [2,8,4]
pg 6.23a is stuck unclean for 187086.153951, current state active+remapped, last acting [0,2,9]
pg MailScanner has detected a possible fraud attempt from "6.bf" claiming to be MailScanner warning: numerical links are often malicious: 6.bf is stuck unclean since forever, current state active+remapped, last acting [2,0,1]
pg 9.b1 is stuck unclean for 4469.112012, current state active+degraded, last acting [4,0]
pg 6.b1 is stuck unclean since forever, current state active+remapped, last acting [5,8,2]
pg 6.17e is stuck unclean for 189490.015499, current state active+remapped, last acting [8,3,9]
pg 6.23f is stuck unclean for 182927.010113, current state active+remapped, last acting [5,8,1]
pg 9.bc is stuck unclean for 178360.379416, current state active+degraded, last acting [2,0]
pg MailScanner has detected a possible fraud attempt from "9.ba" claiming to be MailScanner warning: numerical links are often malicious: 9.ba is stuck unclean for 146602.612400, current state active+degraded, last acting [2,8]
pg 6.172 is stuck unclean since forever, current state active+remapped, last acting [0,5,1]
pg 6.b4 is stuck unclean for 181657.091254, current state active+remapped, last acting [4,0,1]
pg 6.174 is stuck unclean for 187564.241472, current state active+remapped, last acting [2,8,3]
pg 6.a8 is stuck unclean for 197921.381643, current state active+remapped, last acting [3,6,0]
pg 6.169 is stuck unclean for 182926.652321, current state active+remapped, last acting [2,8,5]
pg 9.a5 is stuck unclean for 83212.585593, current state active+degraded, last acting [3,6]
pg MailScanner has detected a possible fraud attempt from "6.af" claiming to be MailScanner warning: numerical links are often malicious: 6.af is stuck unclean for 207081.360042, current state active+remapped, last acting [0,5,4]
pg 6.16c is stuck unclean since forever, current state active+remapped, last acting [2,6,5]
pg 6.22d is stuck unclean since forever, current state active+remapped, last acting [2,6,3]
pg MailScanner has detected a possible fraud attempt from "9.ae" claiming to be MailScanner warning: numerical links are often malicious: 9.ae is stuck unclean for 83238.657997, current state active+degraded, last acting [5,8]
pg 6.16e is stuck unclean for 189490.015156, current state active+remapped, last acting [8,3,9]
pg 6.223 is stuck unclean for 181657.047363, current state active+remapped, last acting [2,0,1]
pg 6.165 is stuck unclean for 182927.247437, current state active+remapped, last acting [4,6,1]
pg 6.224 is stuck unclean for 25930.448314, current state active+remapped, last acting [5,0,7]
pg 6.99 is stuck unclean since forever, current state active+remapped, last acting [3,0,1]
pg 6.9d is stuck unclean for 181656.555581, current state active+remapped, last acting [8,2,5]
pg 6.21a is stuck unclean for 189489.819499, current state active+remapped, last acting [3,6,0]
pg 6.15c is stuck unclean since forever, current state active, last acting [3,6,5]
pg 6.15f is stuck unclean since forever, current state active+remapped, last acting [0,4,1]
pg 6.21c is stuck unclean since forever, current state active+remapped, last acting [2,6,3]
pg 6.91 is stuck unclean for 182926.643616, current state active+remapped, last acting [2,0,1]
pg 6.21f is stuck unclean for 182927.089147, current state active+remapped, last acting [8,5,1]
pg 6.150 is stuck unclean since forever, current state active+remapped, last acting [5,0,1]
pg 6.210 is stuck unclean for 182927.026746, current state active+remapped, last acting [3,6,1]
pg 9.9a is stuck unclean for 82751.873851, current state active+degraded, last acting [8,2]
pg 6.152 is stuck unclean for 182927.247268, current state active+remapped, last acting [4,8,1]
pg 6.155 is stuck unclean for 189489.995299, current state active+remapped, last acting [0,4,9]
pg 6.154 is stuck unclean since forever, current state active+remapped, last acting [3,6,0]
pg 6.214 is stuck unclean since forever, current state active+remapped, last acting [6,2,5]
pg 6.149 is stuck unclean since forever, current state active+remapped, last acting [2,8,3]
pg 6.14a is stuck unclean for 182926.649578, current state active+remapped, last acting [2,6,5]
pg 6.20a is stuck unclean for 187086.145054, current state active+remapped, last acting [0,5,9]
pg 9.8e is stuck unclean for 154670.986075, current state active+degraded, last acting [2,0]
pg 6.80 is stuck unclean for 182927.071220, current state active+remapped, last acting [8,5,1]
pg 6.83 is stuck unclean for 197921.387641, current state active+remapped, last acting [3,8,1]
pg 6.82 is stuck unclean for 181656.554868, current state active+remapped, last acting [8,3,1]
pg 6.142 is stuck unclean since forever, current state active+remapped, last acting [8,2,5]
pg 9.77 is stuck unclean for 82751.780914, current state active+degraded, last acting [0,2]
pg 6.7b is stuck unclean for 182927.168302, current state active+remapped, last acting [4,8,1]
pg 6.138 is stuck unclean for 199785.426014, current state active+remapped, last acting [5,6,1]
pg 6.7a is stuck unclean for 187086.123925, current state active+remapped, last acting [0,2,5]
pg 6.13b is stuck unclean for 181657.133217, current state active+remapped, last acting [5,6,0]
pg 6.13c is stuck unclean for 181657.030277, current state active+remapped, last acting [2,6,3]
pg 6.1fd is stuck unclean for 182927.024824, current state active+remapped, last acting [3,0,1]
pg 6.7e is stuck unclean for 181656.733443, current state active+remapped, last acting [3,8,1]
pg 6.71 is stuck unclean for 182927.014811, current state active+remapped, last acting [5,6,0]
pg 6.13e is stuck unclean since forever, current state active+remapped, last acting [2,0,1]
pg 6.1fe is stuck unclean for 182927.375724, current state active+remapped, last acting [6,2,3]
pg 6.73 is stuck unclean for 182927.014794, current state active+remapped, last acting [5,6,1]
pg 6.1f1 is stuck unclean for 182927.375694, current state active+remapped, last acting [6,2,5]
pg 6.72 is stuck unclean for 181656.815359, current state active+remapped, last acting [2,0,1]
pg 6.75 is stuck unclean for 182927.070651, current state active+remapped, last acting [8,5,1]
pg 6.74 is stuck unclean since forever, current state active+remapped, last acting [8,5,0]
pg 6.77 is stuck unclean since forever, current state active+remapped, last acting [3,0,4]
pg 9.78 is stuck unclean for 4454.429800, current state active+degraded, last acting [4,8]
pg 6.1f5 is stuck unclean since forever, current state active+remapped, last acting [3,0,2]
pg 6.69 is stuck unclean for 189489.885122, current state active+remapped, last acting [4,6,9]
pg 9.66 is stuck unclean for 4454.422339, current state active+degraded, last acting [4,6]
pg 6.68 is stuck unclean for 182927.357434, current state active+remapped, last acting [6,4,1]
pg 9.64 is stuck unclean for 81616.028304, current state active+degraded, last acting [8,2]
pg 9.65 is stuck unclean for 4519.092258, current state active+degraded, last acting [6,4]
pg 9.c6 is active+clean+degraded, acting [8,3]
pg 9.c2 is active+clean+degraded, acting [5,8]
pg 9.c3 is active+degraded, acting [4,6]
pg 9.b1 is active+degraded, acting [4,0]
pg 9.bc is active+degraded, acting [2,0]
pg MailScanner has detected a possible fraud attempt from "9.bd" claiming to be MailScanner warning: numerical links are often malicious: 9.bd is active+clean+degraded, acting [6,5]
pg MailScanner has detected a possible fraud attempt from "9.ba" claiming to be MailScanner warning: numerical links are often malicious: 9.ba is active+degraded, acting [2,8]
pg 9.a5 is active+degraded, acting [3,6]
pg MailScanner has detected a possible fraud attempt from "9.ae" claiming to be MailScanner warning: numerical links are often malicious: 9.ae is active+degraded, acting [5,8]
pg MailScanner has detected a possible fraud attempt from "9.ac" claiming to be MailScanner warning: numerical links are often malicious: 9.ac is active+clean+degraded, acting [0,5]
pg 9.96 is active+clean+degraded, acting [3,0]
pg 9.9a is active+degraded, acting [8,2]
pg 9.80 is active+clean+degraded, acting [3,8]
pg 9.8e is active+degraded, acting [2,0]
pg 9.77 is active+degraded, acting [0,2]
pg 9.74 is active+clean+degraded, acting [3,0]
pg 9.72 is active+clean+degraded, acting [8,5]
pg 9.70 is active+clean+degraded, acting [5,6]
pg 9.71 is active+clean+degraded, acting [8,5]
pg 9.7f is active+clean+degraded, acting [8,3]
pg 9.7d is active+clean+degraded, acting [8,5]
pg 9.7b is active+clean+degraded, acting [3,8]
pg 9.78 is active+degraded, acting [4,8]
pg 9.66 is active+degraded, acting [4,6]
pg 9.64 is active+degraded, acting [8,2]
pg 9.65 is active+degraded, acting [6,4]
pg 9.63 is active+degraded, acting [8,2]
pg 9.6e is active+clean+degraded, acting [5,6]
pg 9.6f is active+degraded, acting [2,0]
pg 9.57 is active+degraded, acting [0,2]
pg 9.55 is active+clean+degraded, acting [3,0]
pg 9.5f is active+degraded, acting [4,0]
pg 9.5d is active+degraded, acting [0,2]
pg 9.42 is active+clean+degraded, acting [3,8]
pg 9.41 is active+degraded, acting [2,8]
pg 9.4c is active+degraded, acting [0,4]
pg 9.4b is active+clean+degraded, acting [3,8]
pg 9.3e is active+degraded, acting [2,8]
pg 9.26 is active+clean+degraded, acting [8,5]
pg 9.25 is active+clean+degraded, acting [5,0]
pg 9.23 is active+degraded, acting [4,6]
pg 9.21 is active+clean+degraded, acting [8,3]
pg 9.2e is active+clean+degraded, acting [3,6]
pg 9.2b is active+degraded, acting [2,6]
pg 9.29 is active+clean+degraded, acting [3,0]
pg 9.10 is active+clean+degraded, acting [3,0]
pg 9.1f is active+degraded, acting [2,8]
pg 9.19 is active+degraded, acting [6,2]
pg 9.b is active+degraded, acting [2,6]
recovery 19479/2079561 objects degraded (0.937%)root@osh1:~# ceph --cluster apics pg dump_stuck
ok
pg_stat objects mip degr unf bytes log disklog state state_stamp v reported up acting last_scrub scrub_stamp last_deep_scrub deep_scrub_stamp
9.63 519 0 519 0 1042014 3003 3003 active+degraded 2014-01-12 10:16:02.499133 1527'8548 1605:19000 [8,2] [8,2] 615'5995 2014-01-09 12:29:18.871498 342'5982 2014-01-06 00:37:57.667418
6.12d 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:29.248336 0'0 1605:1467 [2,8] [2,8,3] 0'0 2014-01-09 09:03:27.120625 0'0 2014-01-08 09:01:18.371990
9.61 506 0 0 0 1077912 3000 3000 active+remapped 2014-01-12 16:56:36.923766 1501'7214 1605:18207 [3,6] [3,6,5] 1501'7214 2014-01-11 17:42:38.863720 299'6209 2014-01-06 09:26:59.628384
6.1ec 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.301048 0'0 1605:269 [0,5] [0,5,2] 0'0 2014-01-09 12:24:21.891960 0'0 2014-01-08 11:48:09.345325
6.12e 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:35:42.537436 0'0 1605:1315 [5,0] [5,0,8] 0'0 2014-01-09 08:41:13.417066 0'0 2014-01-08 08:40:05.539008
6.1ef 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.360917 0'0 1605:235 [0,3] [0,3,1] 0'0 2014-01-09 12:25:01.976062 0'0 2014-01-08 12:02:39.055719
6.60 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.761453 0'0 1605:510 [0,2] [0,2,1] 0'0 2014-01-09 10:45:18.158406 0'0 2014-01-08 10:08:18.336081
9.6f 517 0 518 0 5841250 4234 4234 active+degraded 2014-01-12 10:21:29.279125 1565'10260 1605:16979 [2,0] [2,0] 1491'7790 2014-01-10 22:21:34.519687 600'6420 2014-01-06 21:49:48.363891
6.62 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:23:47.670890 0'0 1605:702 [4,0] [4,0,1] 0'0 2014-01-09 16:09:40.034782 0'0 2014-01-08 13:17:31.452140
6.122 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.415398 0'0 1605:719 [6,3] [6,3,1] 0'0 2014-01-09 08:18:09.795832 0'0 2014-01-08 08:18:06.126540
6.64 0 0 0 0 0 0 0 active 2014-01-12 16:14:33.704017 0'0 1605:33 [3,6] [3,6,5] 0'0 2014-01-09 09:03:48.527003 0'0 2014-01-08 09:01:43.855932
6.1e2 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:29.131493 0'0 1605:1339 [5,8] [5,8,9] 0'0 2014-01-09 09:10:43.968654 0'0 2014-01-08 09:07:03.260069
6.67 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:20:57.093502 0'0 1605:701 [8,2] [8,2,3] 0'0 2014-01-09 15:07:17.081140 0'0 2014-01-08 15:06:59.897113
6.66 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:20:56.082554 0'0 1605:578 [8,2] [8,2,5] 0'0 2014-01-09 12:24:21.956577 0'0 2014-01-08 11:48:32.823357
6.58 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.433527 0'0 1605:1551 [3,0] [3,0,1] 0'0 2014-01-09 10:18:53.259457 0'0 2014-01-08 09:46:49.094908
9.57 484 0 484 0 1006483 3002 3002 active+degraded 2014-01-12 10:15:56.115195 1109'5736 1605:14446 [0,2] [0,2] 884'5735 2014-01-09 20:55:09.218442 299'5550 2014-01-06 00:37:38.343410
6.1e6 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:35:42.536393 0'0 1605:662 [5,0] [5,0,8] 0'0 2014-01-09 16:25:13.350510 0'0 2014-01-08 14:03:29.209531
6.5a 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.765400 0'0 1605:1825 [0,2] [0,2,1] 0'0 2014-01-09 19:18:59.709030 0'0 2014-01-09 19:18:59.709030
6.1db 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:20.785977 0'0 1605:670 [0,4] [0,4,1] 0'0 2014-01-10 06:37:03.579999 0'0 2014-01-10 06:37:03.579999
6.11c 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.358395 0'0 1605:1314 [5,0] [5,0,1] 0'0 2014-01-09 09:04:45.646165 0'0 2014-01-08 09:02:38.922668
6.1dc 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:35:28.396308 0'0 1605:711 [8,3] [8,3,5] 0'0 2014-01-09 19:56:49.359394 0'0 2014-01-09 19:56:49.359394
9.5f 514 0 514 0 726167 3002 3002 active+degraded 2014-01-12 16:23:50.792494 1491'7306 1605:16463 [4,0] [4,0] 1491'7306 2014-01-11 18:07:46.192152 299'7125 2014-01-07 02:16:57.159837
6.111 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:07:00.617463 0'0 1605:1291 [5,6] [5,6,0] 0'0 2014-01-09 08:40:28.369866 0'0 2014-01-08 08:39:48.560967
6.110 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:29.248910 0'0 1605:1528 [2,8] [2,8,3] 0'0 2014-01-09 09:00:52.018913 0'0 2014-01-08 09:00:45.358662
9.5d 504 0 505 0 5894969 4241 4241 active+degraded 2014-01-12 10:16:02.372640 1586'6561 1605:15652 [0,2] [0,2] 884'5308 2014-01-09 14:54:18.368889 299'5263 2014-01-06 14:46:01.474050
6.113 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.359390 0'0 1605:1001 [5,0] [5,0,1] 0'0 2014-01-09 16:26:30.767291 0'0 2014-01-08 14:03:39.687286
6.117 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.570030 0'0 1605:1786 [0,2] [0,2,1] 0'0 2014-01-09 09:06:19.414663 0'0 2014-01-08 09:06:07.543870
6.109 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:20:23.141017 0'0 1605:77 [2,8] [2,8,5] 0'0 2014-01-09 16:54:43.106979 0'0 2014-01-08 15:33:51.778814
6.10a 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:20:23.139547 0'0 1605:1502 [2,8] [2,8,5] 0'0 2014-01-09 09:24:12.925192 0'0 2014-01-08 09:16:57.973042
6.4f 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:20.950677 0'0 1605:517 [0,4] [0,4,1] 0'0 2014-01-09 10:17:59.246328 0'0 2014-01-08 09:45:50.088888
6.4e 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.447561 0'0 1605:205 [3,8] [3,8,1] 0'0 2014-01-09 10:07:42.946438 0'0 2014-01-08 09:37:30.906450
9.41 285 0 284 0 455560 3002 3002 active+degraded 2014-01-12 10:21:25.393688 1565'5155 1605:10472 [2,8] [2,8] 615'3783 2014-01-09 10:45:50.619414 299'3781 2014-01-06 08:51:18.437773
6.10f 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.426273 0'0 1605:547 [6,3] [6,3,1] 0'0 2014-01-09 16:54:57.680559 0'0 2014-01-08 15:08:47.013771
6.1cc 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:14:33.093331 0'0 1605:682 [6,2] [6,2,5] 0'0 2014-01-10 02:50:32.084624 0'0 2014-01-10 02:50:32.084624
6.41 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:00.360657 0'0 1605:603 [2,8] [2,8,5] 0'0 2014-01-09 14:30:45.305420 0'0 2014-01-08 12:43:26.784150
6.1ce 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:23.961210 0'0 1605:1495 [2,0] [2,0,9] 0'0 2014-01-09 09:05:16.346117 0'0 2014-01-08 09:03:25.479488
9.4c 511 0 511 0 881371 3002 3002 active+degraded 2014-01-12 16:21:20.952410 1565'5905 1605:14560 [0,4] [0,4] 615'4516 2014-01-09 08:58:52.309443 299'4492 2014-01-07 03:01:43.374142
6.103 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.369690 0'0 1605:895 [5,6] [5,6,1] 0'0 2014-01-09 16:23:26.473540 0'0 2014-01-08 13:20:41.951785
6.45 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:56:15.121244 0'0 1605:1581 [3,8] [3,8,9] 0'0 2014-01-09 10:16:28.274734 0'0 2014-01-08 09:45:48.564931
6.1c3 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:07:00.520696 0'0 1605:869 [5,6] [5,6,8] 0'0 2014-01-09 16:55:27.692213 0'0 2014-01-08 15:09:34.039603
6.44 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:29.277955 0'0 1605:1494 [2,8] [2,8,3] 0'0 2014-01-10 03:07:39.229560 0'0 2014-01-07 22:27:32.496363
6.1c2 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:29.236995 0'0 1605:73 [2,8] [2,8,3] 0'0 2014-01-09 10:41:09.575115 0'0 2014-01-08 09:54:16.500084
6.1c5 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.270096 0'0 1605:1340 [5,6] [5,6,1] 0'0 2014-01-09 09:08:39.895969 0'0 2014-01-08 09:06:57.244843
6.1c6 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:19:32.778447 0'0 1605:1537 [2,8] [2,8,5] 0'0 2014-01-09 09:04:56.271867 0'0 2014-01-08 09:03:16.470590
9.32 267 0 267 0 760405 1732 1732 active+remapped 2014-01-12 10:16:08.293588 1109'1732 1605:6199 [0,2] [0,2,3] 884'1731 2014-01-09 20:54:35.857181 299'1712 2014-01-06 00:37:16.909432
6.1bc 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.389568 0'0 1605:521 [0,2] [0,2,1] 0'0 2014-01-09 08:49:16.759156 0'0 2014-01-08 08:49:14.982061
6.31 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:48:36.832214 0'0 1605:194 [3,6] [3,6,0] 0'0 2014-01-09 14:33:15.491643 0'0 2014-01-08 12:44:54.931011
9.3e 257 0 257 0 4863385 4266 4266 active+degraded 2014-01-12 10:19:43.301020 1527'5478 1605:10677 [2,8] [2,8] 615'3980 2014-01-09 12:40:29.560774 299'3969 2014-01-06 17:48:58.838951
6.fe 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.384469 0'0 1605:524 [8,3] [8,3,1] 0'0 2014-01-09 09:17:40.417863 0'0 2014-01-08 09:09:11.171327
6.f0 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:56:01.445331 0'0 1605:1576 [3,0] [3,0,9] 0'0 2014-01-09 10:26:18.577847 0'0 2014-01-08 09:53:57.245027
6.1b0 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:29.364499 0'0 1605:1323 [5,8] [5,8,9] 0'0 2014-01-09 08:43:40.523357 0'0 2014-01-08 08:43:22.620376
6.35 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:38.392468 0'0 1605:1801 [0,2] [0,2,9] 0'0 2014-01-09 19:18:47.707930 0'0 2014-01-09 19:18:47.707930
6.1b2 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:07:00.543523 0'0 1605:1347 [5,0] [5,0,6] 0'0 2014-01-09 11:12:38.981953 0'0 2014-01-08 11:11:22.856320
6.29 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.523288 0'0 1605:513 [8,5] [8,5,1] 0'0 2014-01-09 10:44:23.127433 0'0 2014-01-08 10:07:09.308621
6.28 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.454218 0'0 1605:429 [5,0] [5,0,1] 0'0 2014-01-09 10:44:21.126616 0'0 2014-01-08 10:05:23.287340
6.e9 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.180901 0'0 1605:78 [2,6] [2,6,3] 0'0 2014-01-09 16:23:26.274441 0'0 2014-01-08 13:37:04.794046
6.e8 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.182154 0'0 1605:1502 [2,6] [2,6,3] 0'0 2014-01-09 09:23:27.822206 0'0 2014-01-08 09:12:56.813109
6.1a8 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.393001 0'0 1605:673 [0,5] [0,5,1] 0'0 2014-01-09 08:29:55.042014 0'0 2014-01-08 08:29:54.548487
6.2c 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.454337 0'0 1605:1585 [3,0] [3,0,1] 0'0 2014-01-09 10:15:49.174461 0'0 2014-01-08 09:43:10.046785
9.23 281 0 281 0 1203992 3001 3001 active+degraded 2014-01-12 16:23:53.986197 1109'3245 1605:8663 [4,6] [4,6] 1109'3245 2014-01-11 10:30:31.471057 299'3077 2014-01-06 14:33:55.527018
MailScanner has detected a possible fraud attempt from "6.ec" claiming to be MailScanner warning: numerical links are often malicious: 6.ec 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.267184 0'0 1605:659 [6,2] [6,2,3] 0'0 2014-01-09 15:19:52.865780 0'0 2014-01-08 15:18:57.258859
6.2e 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:06.572899 0'0 1605:76 [2,6] [2,6,5] 0'0 2014-01-09 08:55:56.840643 0'0 2014-01-08 08:54:48.264306
6.1ae 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:22:15.852876 0'0 1605:1363 [4,0] [4,0,1] 0'0 2014-01-09 10:57:26.601672 0'0 2014-01-08 10:14:36.158317
6.1a1 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:30.364236 0'0 1605:1606 [3,0] [3,0,9] 0'0 2014-01-09 09:51:37.383129 0'0 2014-01-08 09:31:18.688919
6.22 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:09.469724 0'0 1605:1429 [2,8] [2,8,4] 0'0 2014-01-09 08:55:55.854115 0'0 2014-01-08 08:54:37.256926
6.24 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.530486 0'0 1605:702 [8,3] [8,3,1] 0'0 2014-01-09 19:18:34.690876 0'0 2014-01-09 19:18:34.690876
9.2b 276 0 276 0 994133 3001 3001 active+degraded 2014-01-12 10:21:07.380808 1109'3229 1605:7774 [2,6] [2,6] 615'3213 2014-01-09 10:43:26.465712 299'3202 2014-01-06 08:45:29.474538
6.1a5 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:30.365180 0'0 1605:849 [3,6] [3,6,9] 0'0 2014-01-09 12:34:15.079349 0'0 2014-01-08 12:34:11.416407
6.26 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:23:53.989924 0'0 1605:1281 [4,6] [4,6,8] 0'0 2014-01-09 10:43:40.110560 0'0 2014-01-08 10:05:22.274204
9.17 257 0 0 0 4933022 3001 3001 active+remapped 2014-01-12 16:14:25.437802 1527'3806 1605:10986 [6,2] [6,2,7] 884'2624 2014-01-10 06:12:17.756516 884'2624 2014-01-10 06:12:17.756516
6.1a6 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:29.714289 0'0 1605:1340 [5,6] [5,6,9] 0'0 2014-01-09 09:06:12.797997 0'0 2014-01-08 09:06:12.193747
6.199 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.321114 0'0 1605:1570 [3,6] [3,6,1] 0'0 2014-01-09 10:29:18.663287 0'0 2014-01-08 09:54:08.265485
6.1a 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:14:25.438292 0'0 1605:1363 [6,2] [6,2,7] 0'0 2014-01-09 20:36:30.994093 0'0 2014-01-09 20:36:30.994093
6.da 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:24.188309 0'0 1605:702 [8,5] [8,5,9] 0'0 2014-01-10 06:38:52.549091 0'0 2014-01-10 06:38:52.549091
6.1c 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:06.617744 0'0 1605:748 [6,2] [6,2,5] 0'0 2014-01-09 16:51:59.504404 0'0 2014-01-08 15:08:10.305595
6.19d 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:25.010947 0'0 1605:395 [8,2] [8,2,3] 0'0 2014-01-09 16:55:01.114467 0'0 2014-01-08 15:34:09.794056
9.1f 256 0 256 0 505448 2911 2911 active+degraded 2014-01-12 10:15:17.820311 1491'2911 1605:7124 [2,8] [2,8] 1109'2910 2014-01-10 21:21:06.415076 299'2831 2014-01-06 19:09:13.654085
6.d1 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:48:20.302744 0'0 1605:1270 [5,8] [5,8,0] 0'0 2014-01-09 08:40:31.389855 0'0 2014-01-08 08:39:49.560872
6.19e 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:09.549937 0'0 1605:1696 [8,4] [8,4,1] 0'0 2014-01-09 08:48:57.237465 0'0 2014-01-08 08:48:55.964886
6.13 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:56:15.155440 0'0 1605:1565 [3,0] [3,0,9] 0'0 2014-01-09 09:54:05.418547 0'0 2014-01-08 09:32:49.785038
6.191 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:24.020591 0'0 1605:1281 [6,3] [6,3,9] 0'0 2014-01-10 02:50:30.042711 0'0 2014-01-10 02:50:30.042711
6.d2 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.371571 0'0 1605:1288 [5,0] [5,0,1] 0'0 2014-01-09 09:03:58.637975 0'0 2014-01-08 09:02:33.917141
6.250 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:47:51.620257 0'0 1605:1581 [3,6] [3,6,0] 0'0 2014-01-09 10:32:08.844964 0'0 2014-01-08 09:55:38.287309
6.d4 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.347861 0'0 1605:416 [3,6] [3,6,1] 0'0 2014-01-09 10:51:55.467448 0'0 2014-01-08 10:13:14.112677
6.252 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.297932 0'0 1605:503 [2,0] [2,0,1] 0'0 2014-01-09 12:25:14.982234 0'0 2014-01-08 12:02:51.052423
9.19 253 0 253 0 606223 3002 3002 active+degraded 2014-01-12 10:16:02.453285 1109'3153 1605:7462 [6,2] [6,2] 884'3152 2014-01-09 16:26:55.642238 299'3011 2014-01-07 08:37:09.343261
6.c9 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:24.189323 0'0 1605:719 [8,3] [8,3,9] 0'0 2014-01-09 19:27:09.848901 0'0 2014-01-09 19:27:09.848901
6.196 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.323930 0'0 1605:1576 [3,6] [3,6,1] 0'0 2014-01-09 10:27:18.612860 0'0 2014-01-08 09:54:06.250691
6.257 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.257906 0'0 1605:989 [0,5] [0,5,1] 0'0 2014-01-09 20:06:21.532104 0'0 2014-01-09 20:06:21.532104
6.249 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:10.026851 0'0 1605:522 [6,2] [6,2,4] 0'0 2014-01-09 09:12:36.014578 0'0 2014-01-08 09:08:16.330104
6.248 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:14:25.356629 0'0 1605:44 [6,2] [6,2,0] 0'0 2014-01-09 16:25:16.309428 0'0 2014-01-08 14:03:38.247840
6.18a 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.324766 0'0 1605:197 [3,6] [3,6,1] 0'0 2014-01-09 12:34:12.103024 0'0 2014-01-08 12:34:09.938504
6.24b 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:00.360878 0'0 1605:1491 [2,8] [2,8,5] 0'0 2014-01-09 09:08:37.598426 0'0 2014-01-08 09:06:22.577679
6.cc 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:23:39.871509 0'0 1605:1311 [4,0] [4,0,1] 0'0 2014-01-09 10:32:08.775105 0'0 2014-01-08 09:55:42.034457
9.c3 240 0 240 0 767354 2882 2882 active+degraded 2014-01-12 16:23:39.869188 884'2882 1605:6555 [4,6] [4,6] 884'2882 2014-01-12 13:55:50.007941 299'2824 2014-01-07 05:18:32.908294
6.e 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.220860 0'0 1605:1496 [2,6] [2,6,3] 0'0 2014-01-09 08:55:54.839006 0'0 2014-01-08 08:53:27.205348
9.1 284 0 2 0 8773472 3001 3001 active+remapped 2014-01-12 16:26:24.582499 1605'4995 1605:1908 [2,0] [2,0,5] 1199'2589 2014-01-11 06:07:27.851789 299'2152 2014-01-07 05:12:42.379851
6.24c 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:48:12.669333 0'0 1605:1604 [3,6] [3,6,0] 0'0 2014-01-09 10:31:28.803424 0'0 2014-01-08 09:55:02.271086
6.c0 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:35:19.595209 0'0 1605:673 [6,5] [6,5,3] 0'0 2014-01-07 15:09:56.028935 0'0 2013-12-31 19:07:00.165926
6.24e 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.182787 0'0 1605:1325 [6,3] [6,3,1] 0'0 2014-01-10 05:43:23.038175 0'0 2014-01-10 05:43:23.038175
6.2 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.710914 0'0 1605:1214 [6,5] [6,5,1] 0'0 2014-01-09 11:08:44.798110 0'0 2014-01-08 11:07:54.973330
6.4 0 0 0 0 0 0 0 active 2014-01-12 16:16:02.584767 0'0 1605:40 [2,0] [2,0,5] 0'0 2014-01-09 10:39:45.988813 0'0 2014-01-08 10:03:25.712846
9.b 255 0 255 0 511815 2597 2597 active+degraded 2014-01-12 10:15:17.832137 1491'2597 1605:7550 [2,6] [2,6] 1109'2596 2014-01-10 21:01:03.832837 299'2579 2014-01-06 19:08:34.842640
6.c5 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.373055 0'0 1605:408 [5,8] [5,8,1] 0'0 2014-01-09 10:51:52.465758 0'0 2014-01-08 10:13:10.108756
6.c6 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:23:24.146939 0'0 1605:1320 [4,6] [4,6,1] 0'0 2014-01-09 10:36:18.881569 0'0 2014-01-08 09:59:22.108028
6.246 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:38.136103 0'0 1605:1641 [0,3] [0,3,9] 0'0 2014-01-09 09:23:27.018453 0'0 2014-01-08 09:17:05.086984
MailScanner has detected a possible fraud attempt from "6.bd" claiming to be MailScanner warning: numerical links are often malicious: 6.bd 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:09.458198 0'0 1605:107 [2,8] [2,8,4] 0'0 2014-01-09 12:19:43.882949 0'0 2014-01-08 11:47:30.792530
6.23a 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:38.136777 0'0 1605:1864 [0,2] [0,2,9] 0'0 2014-01-09 20:06:18.530400 0'0 2014-01-09 20:06:18.530400
MailScanner has detected a possible fraud attempt from "6.bf" claiming to be MailScanner warning: numerical links are often malicious: 6.bf 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.237895 0'0 1605:94 [2,0] [2,0,1] 0'0 2014-01-09 16:49:41.912921 0'0 2014-01-08 14:40:41.355242
9.b1 278 0 278 0 1019316 3001 3001 active+degraded 2014-01-12 16:23:30.058082 1109'3071 1605:6640 [4,0] [4,0] 1109'3071 2014-01-12 11:03:50.240907 299'2661 2014-01-07 05:18:29.080755
6.b1 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.848134 0'0 1605:238 [5,8] [5,8,2] 0'0 2014-01-09 12:30:47.991392 0'0 2014-01-08 12:30:07.382088
6.17e 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:24.092695 0'0 1605:715 [8,3] [8,3,9] 0'0 2014-01-10 06:39:01.563804 0'0 2014-01-10 06:39:01.563804
6.23f 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.458123 0'0 1605:1285 [5,8] [5,8,1] 0'0 2014-01-09 08:51:50.913292 0'0 2014-01-08 08:51:29.800394
9.bc 249 0 249 0 366022 1985 1985 active+degraded 2014-01-12 10:15:17.715812 884'1985 1605:847 [2,0] [2,0] 884'1985 2014-01-09 16:26:06.346292 299'1966 2014-01-07 08:43:56.689069
MailScanner has detected a possible fraud attempt from "9.ba" claiming to be MailScanner warning: numerical links are often malicious: 9.ba 282 0 282 0 713504 3001 3001 active+degraded 2014-01-12 10:15:45.114064 1491'3548 1605:9295 [2,8] [2,8] 615'3148 2014-01-09 11:30:53.430338 615'3148 2014-01-08 11:14:00.957418
6.172 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.429703 0'0 1605:221 [0,5] [0,5,1] 0'0 2014-01-09 11:18:07.081397 0'0 2014-01-08 10:24:17.153406
6.b4 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:23:30.143976 0'0 1605:1298 [4,0] [4,0,1] 0'0 2014-01-09 10:47:47.203140 0'0 2014-01-08 10:09:23.363418
6.174 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:29.243245 0'0 1605:312 [2,8] [2,8,3] 0'0 2014-01-09 10:53:42.519222 0'0 2014-01-08 10:13:58.149248
6.a8 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:48:37.071210 0'0 1605:331 [3,6] [3,6,0] 0'0 2014-01-09 10:31:41.761935 0'0 2014-01-08 09:55:18.027269
6.169 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:19:49.723024 0'0 1605:1501 [2,8] [2,8,5] 0'0 2014-01-09 09:31:25.136044 0'0 2014-01-08 09:18:41.073577
9.a5 261 0 261 0 681314 3045 3045 active+degraded 2014-01-11 18:33:15.638697 1501'4656 1605:12077 [3,6] [3,6] 884'3629 2014-01-09 15:05:31.498011 299'3603 2014-01-06 15:01:16.697258
MailScanner has detected a possible fraud attempt from "6.af" claiming to be MailScanner warning: numerical links are often malicious: 6.af 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:20.920953 0'0 1605:464 [0,5] [0,5,4] 0'0 2014-01-09 09:00:14.242287 0'0 2014-01-08 09:00:09.061867
6.16c 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:04.813484 0'0 1605:85 [2,6] [2,6,5] 0'0 2014-01-09 17:00:10.356004 0'0 2014-01-08 15:52:27.147650
6.22d 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.135565 0'0 1605:77 [2,6] [2,6,3] 0'0 2014-01-09 16:08:10.003447 0'0 2014-01-08 13:16:22.419681
MailScanner has detected a possible fraud attempt from "9.ae" claiming to be MailScanner warning: numerical links are often malicious: 9.ae 279 0 280 0 4929245 4262 4262 active+degraded 2014-01-11 18:32:33.273086 1565'5245 1605:12112 [5,8] [5,8] 615'3912 2014-01-09 12:25:39.417078 299'3902 2014-01-06 16:40:48.555270
6.16e 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:24.102547 0'0 1605:707 [8,3] [8,3,9] 0'0 2014-01-09 21:58:00.116420 0'0 2014-01-09 21:58:00.116420
6.223 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.175408 0'0 1605:338 [2,0] [2,0,1] 0'0 2014-01-09 10:58:10.620538 0'0 2014-01-08 10:14:45.164997
6.165 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:22:22.739013 0'0 1605:1355 [4,6] [4,6,1] 0'0 2014-01-09 10:53:40.526426 0'0 2014-01-08 10:13:57.144933
6.224 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:16:35.166879 0'0 1605:1318 [5,0] [5,0,7] 0'0 2014-01-11 01:55:28.428832 0'0 2014-01-08 08:51:21.796833
6.99 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.366371 0'0 1605:206 [3,0] [3,0,1] 0'0 2014-01-09 09:39:51.857459 0'0 2014-01-08 09:27:28.474615
6.9d 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:20:48.760442 0'0 1605:712 [8,2] [8,2,5] 0'0 2014-01-09 19:27:06.862888 0'0 2014-01-09 19:27:06.862888
6.21a 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:05:46.937303 0'0 1605:1632 [3,6] [3,6,0] 0'0 2014-01-09 10:30:49.725194 0'0 2014-01-08 09:54:13.254937
6.15c 0 0 0 0 0 0 0 active 2014-01-12 16:14:33.270607 0'0 1605:34 [3,6] [3,6,5] 0'0 2014-01-09 08:41:16.439686 0'0 2014-01-08 08:40:48.567231
6.15f 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:20.825238 0'0 1605:231 [0,4] [0,4,1] 0'0 2014-01-09 16:55:22.690895 0'0 2014-01-08 15:09:02.011332
6.21c 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.140501 0'0 1605:78 [2,6] [2,6,3] 0'0 2014-01-09 10:42:21.603282 0'0 2014-01-08 09:54:59.513208
6.91 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.239622 0'0 1605:1510 [2,0] [2,0,1] 0'0 2014-01-09 09:21:54.744981 0'0 2014-01-08 09:11:59.770748
6.21f 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.206178 0'0 1605:506 [8,5] [8,5,1] 0'0 2014-01-10 03:03:47.780276 0'0 2014-01-10 03:03:47.780276
6.150 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.331047 0'0 1605:206 [5,0] [5,0,1] 0'0 2014-01-09 12:24:18.888704 0'0 2014-01-08 11:48:00.298061
6.210 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.306686 0'0 1605:456 [3,6] [3,6,1] 0'0 2014-01-09 10:39:54.993102 0'0 2014-01-08 10:03:30.224818
9.9a 227 0 227 0 956824 3000 3000 active+degraded 2014-01-12 10:16:02.400086 884'3906 1605:9239 [8,2] [8,2] 884'3906 2014-01-09 20:56:55.174021 299'3819 2014-01-06 00:38:22.792384
6.152 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:22:34.779934 0'0 1605:1336 [4,8] [4,8,1] 0'0 2014-01-09 10:38:02.914887 0'0 2014-01-08 10:02:11.190879
6.155 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:38.266827 0'0 1605:670 [0,4] [0,4,9] 0'0 2014-01-09 09:33:01.719863 0'0 2014-01-08 09:20:32.349939
6.154 0 0 0 0 0 0 0 active+remapped 2014-01-10 13:44:12.933852 0'0 1605:224 [3,6] [3,6,0] 0'0 2014-01-09 16:55:20.691398 0'0 2014-01-08 15:09:00.010145
6.214 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:06.493689 0'0 1605:238 [6,2] [6,2,5] 0'0 2014-01-09 10:40:00.998246 0'0 2014-01-08 10:03:35.229280
6.149 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:25.016020 0'0 1605:93 [2,8] [2,8,3] 0'0 2014-01-09 09:03:46.143118 0'0 2014-01-08 09:01:38.386273
6.14a 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:06.551389 0'0 1605:1507 [2,6] [2,6,5] 0'0 2014-01-09 09:26:58.010643 0'0 2014-01-08 09:17:15.010940
6.20a 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:38.145184 0'0 1605:1306 [0,5] [0,5,9] 0'0 2014-01-09 07:55:41.720768 0'0 2014-01-08 07:54:46.098197
9.8e 281 0 281 0 597395 3003 3003 active+degraded 2014-01-12 10:21:27.943479 1109'3302 1605:7601 [2,0] [2,0] 1109'3302 2014-01-10 22:36:07.774680 342'3053 2014-01-06 22:26:44.662514
6.80 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.444524 0'0 1605:693 [8,5] [8,5,1] 0'0 2014-01-09 19:23:11.775546 0'0 2014-01-09 19:23:11.775546
6.83 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.393136 0'0 1605:808 [3,8] [3,8,1] 0'0 2014-01-09 16:45:24.854229 0'0 2014-01-08 14:40:31.349407
6.82 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.460593 0'0 1605:700 [8,3] [8,3,1] 0'0 2014-01-09 19:23:12.776633 0'0 2014-01-09 19:23:12.776633
6.142 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:20:21.411519 0'0 1605:244 [8,2] [8,2,5] 0'0 2014-01-09 10:40:40.595621 0'0 2014-01-08 09:54:13.504283
9.77 501 0 501 0 1136032 3003 3003 active+degraded 2014-01-12 10:16:02.368324 1109'4746 1605:12686 [0,2] [0,2] 884'4744 2014-01-09 20:56:18.077813 299'4557 2014-01-06 00:38:05.128010
6.7b 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:23:41.787528 0'0 1605:1336 [4,8] [4,8,1] 0'0 2014-01-09 10:32:00.769992 0'0 2014-01-08 09:55:39.034384
6.138 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.343884 0'0 1605:817 [5,6] [5,6,1] 0'0 2014-01-09 12:24:17.896516 0'0 2014-01-08 11:47:59.308707
6.7a 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:01.319589 0'0 1605:668 [0,2] [0,2,5] 0'0 2014-01-09 19:19:09.701568 0'0 2014-01-09 19:19:09.701568
6.13b 0 0 0 0 0 0 0 active+remapped 2014-01-10 15:17:27.246371 0'0 1605:1324 [5,6] [5,6,0] 0'0 2014-01-09 09:05:13.736448 0'0 2014-01-08 09:04:25.122441
6.13c 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.172013 0'0 1605:1557 [2,6] [2,6,3] 0'0 2014-01-09 22:44:24.068932 0'0 2014-01-07 22:27:04.477927
6.1fd 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.312842 0'0 1605:1594 [3,0] [3,0,1] 0'0 2014-01-09 09:56:37.535956 0'0 2014-01-08 09:34:03.817682
6.7e 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.416132 0'0 1605:1543 [3,8] [3,8,1] 0'0 2014-01-09 10:21:38.409338 0'0 2014-01-08 09:47:43.130429
6.71 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:47:48.287766 0'0 1605:1279 [5,6] [5,6,0] 0'0 2014-01-09 09:03:49.538744 0'0 2014-01-08 09:02:17.908254
6.13e 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.208592 0'0 1605:94 [2,0] [2,0,1] 0'0 2014-01-09 22:45:15.057054 0'0 2014-01-07 22:36:27.054235
6.1fe 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:02.181911 0'0 1605:509 [6,2] [6,2,3] 0'0 2014-01-09 10:56:20.577888 0'0 2014-01-08 10:14:18.147755
6.73 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.410402 0'0 1605:388 [5,6] [5,6,1] 0'0 2014-01-09 10:46:06.164304 0'0 2014-01-08 10:08:28.342536
6.1f1 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:06.503325 0'0 1605:677 [6,2] [6,2,5] 0'0 2014-01-10 02:50:48.083370 0'0 2014-01-10 02:50:48.083370
6.72 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.245430 0'0 1605:1493 [2,0] [2,0,1] 0'0 2014-01-09 08:57:24.880524 0'0 2014-01-08 08:55:05.238720
6.75 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:21:43.462504 0'0 1605:699 [8,5] [8,5,1] 0'0 2014-01-09 21:20:57.327577 0'0 2014-01-09 21:20:57.327577
6.74 0 0 0 0 0 0 0 active+remapped 2014-01-10 14:47:47.808109 0'0 1605:225 [8,5] [8,5,0] 0'0 2014-01-09 10:31:58.783857 0'0 2014-01-08 09:55:30.035789
6.77 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:24:05.866553 0'0 1605:231 [3,0] [3,0,4] 0'0 2014-01-09 16:05:27.947031 0'0 2014-01-08 13:05:01.210834
9.78 483 0 483 0 1397009 3002 3002 active+degraded 2014-01-12 16:23:48.106924 1491'10499 1605:21936 [4,8] [4,8] 1491'10499 2014-01-11 17:07:40.143810 299'9912 2014-01-06 14:57:57.183660
6.1f5 0 0 0 0 0 0 0 active+remapped 2014-01-12 10:16:00.112840 0'0 1605:250 [3,0] [3,0,2] 0'0 2014-01-09 11:19:14.116060 0'0 2014-01-08 10:24:42.161664
6.69 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:55:40.680517 0'0 1605:1314 [4,6] [4,6,9] 0'0 2014-01-09 10:31:09.756309 0'0 2014-01-08 09:54:04.000963
9.66 514 0 514 0 5941247 4254 4254 active+degraded 2014-01-12 16:23:46.541017 1527'5604 1605:14508 [4,6] [4,6] 1199'4351 2014-01-11 17:04:53.443119 299'4300 2014-01-06 14:55:47.126133
6.68 0 0 0 0 0 0 0 active+remapped 2014-01-12 16:21:10.236438 0'0 1605:1362 [6,4] [6,4,1] 0'0 2014-01-09 20:37:01.966498 0'0 2014-01-09 20:37:01.966498
9.64 494 0 493 0 1001770 3586 3586 active+degraded 2014-01-12 10:15:56.081499 1605'7607 1605:19399 [8,2] [8,2] 879'6275 2014-01-09 08:51:53.242171 299'6023 2014-01-07 02:25:39.810228
9.65 509 0 509 0 1128369 3002 3002 active+degraded 2014-01-12 16:21:10.241191 1535'7350 1605:12720 [6,4] [6,4] 884'6107 2014-01-09 18:05:34.531831 884'6107 2014-01-09 18:05:34.531831--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean._______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Aaron Ten Clay
http://www.aarontc.com/
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
