On Thu, Mar 26, 2015 at 2:56 AM, Saverio Proto <zioproto@xxxxxxxxx> wrote: > Thanks for the answer. Now the meaning of "MB data" and "MB used" is > clear, and if all the pools have size=3 I expect a ratio 1 to 3 of the > two values. > > I still can't understand why "MB used" is so big in my setup. > All my pools are size =3 but the ratio "MB data" and "MB used" is 1 to > 5 instead of 1 to 3. > > My first guess was that I wrote a wrong crushmap that was making more > than 3 copies.. (is it really possible to make such a mistake?) > > So I changed my crushmap and I put the default one, that just spreads > data across hosts, but I see no change, the ratio is still 1 to 5. > > I thought maybe my 3 monitors have different views of the pgmap, so I > tried to restart the monitors but this also did not help. > > What useful information may I share here to troubleshoot this issue further ? > ceph version 0.87.1 (283c2e7cfa2457799f534744d7d549f83ea1335e) You just need to go look at one of your OSDs and see what data is stored on it. Did you configure things so that the journals are using a file on the same storage disk? If so, *that* is why the "data used" is large. I promise that your 5:1 ratio won't persist as you write more than 2GB of data into the cluster. -Greg > > Thank you > > Saverio > > > > 2015-03-25 14:55 GMT+01:00 Gregory Farnum <greg@xxxxxxxxxxx>: >> On Wed, Mar 25, 2015 at 1:24 AM, Saverio Proto <zioproto@xxxxxxxxx> wrote: >>> Hello there, >>> >>> I started to push data into my ceph cluster. There is something I >>> cannot understand in the output of ceph -w. >>> >>> When I run ceph -w I get this kinkd of output: >>> >>> 2015-03-25 09:11:36.785909 mon.0 [INF] pgmap v278788: 26056 pgs: 26056 >>> active+clean; 2379 MB data, 19788 MB used, 33497 GB / 33516 GB avail >>> >>> >>> 2379MB is actually the data I pushed into the cluster, I can see it >>> also in the "ceph df" output, and the numbers are consistent. >>> >>> What I dont understand is 19788MB used. All my pools have size 3, so I >>> expected something like 2379 * 3. Instead this number is very big. >>> >>> I really need to understand how "MB used" grows because I need to know >>> how many disks to buy. >> >> "MB used" is the summation of (the programmatic equivalent to) "df" >> across all your nodes, whereas "MB data" is calculated by the OSDs >> based on data they've written down. Depending on your configuration >> "MB used" can include thing like the OSD journals, or even totally >> unrelated data if the disks are shared with other applications. >> >> "MB used" including the space used by the OSD journals is my first >> guess about what you're seeing here, in which case you'll notice that >> it won't grow any faster than "MB data" does once the journal is fully >> allocated. >> -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |