Thanks for the answer. Now the meaning of "MB data" and "MB used" is clear, and if all the pools have size=3 I expect a ratio 1 to 3 of the two values. I still can't understand why "MB used" is so big in my setup. All my pools are size =3 but the ratio "MB data" and "MB used" is 1 to 5 instead of 1 to 3. My first guess was that I wrote a wrong crushmap that was making more than 3 copies.. (is it really possible to make such a mistake?) So I changed my crushmap and I put the default one, that just spreads data across hosts, but I see no change, the ratio is still 1 to 5. I thought maybe my 3 monitors have different views of the pgmap, so I tried to restart the monitors but this also did not help. What useful information may I share here to troubleshoot this issue further ? ceph version 0.87.1 (283c2e7cfa2457799f534744d7d549f83ea1335e) Thank you Saverio 2015-03-25 14:55 GMT+01:00 Gregory Farnum <greg@xxxxxxxxxxx>: > On Wed, Mar 25, 2015 at 1:24 AM, Saverio Proto <zioproto@xxxxxxxxx> wrote: >> Hello there, >> >> I started to push data into my ceph cluster. There is something I >> cannot understand in the output of ceph -w. >> >> When I run ceph -w I get this kinkd of output: >> >> 2015-03-25 09:11:36.785909 mon.0 [INF] pgmap v278788: 26056 pgs: 26056 >> active+clean; 2379 MB data, 19788 MB used, 33497 GB / 33516 GB avail >> >> >> 2379MB is actually the data I pushed into the cluster, I can see it >> also in the "ceph df" output, and the numbers are consistent. >> >> What I dont understand is 19788MB used. All my pools have size 3, so I >> expected something like 2379 * 3. Instead this number is very big. >> >> I really need to understand how "MB used" grows because I need to know >> how many disks to buy. > > "MB used" is the summation of (the programmatic equivalent to) "df" > across all your nodes, whereas "MB data" is calculated by the OSDs > based on data they've written down. Depending on your configuration > "MB used" can include thing like the OSD journals, or even totally > unrelated data if the disks are shared with other applications. > > "MB used" including the space used by the OSD journals is my first > guess about what you're seeing here, in which case you'll notice that > it won't grow any faster than "MB data" does once the journal is fully > allocated. > -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |