On 04/28/2011 10:02 PM, Gregory Farnum wrote: [various explanations] Thanks Greg, that's very helpful towards graspings ceph's workings. I'll put it in the wiki. > The relation between these reports and your data can be a bit fuzzy, > though. When looking at the disk space used the OSD is just relying on > a df for the mount it's on -- if it's sharing that mount with anything > else (eg, the node OS) then it's not distinguishing between OSD data, > and data on the disk. Something like that must be going on if you've > got a 4.4x ratio. (An example is below. [1]) Based on what you're giving > us here: > 1) You have 9791 MB of data in the filesystem. > 2) You have (12222MB - 9791 MB=) 2431MB of metadata maintaining the Ceph tree. > 3) RADOS is using 24444MB of disk space amongst all your OSDs to store this. > 4) Your nodes have other stuff installed to the tune of (29135MB/2=)14567MB or (29135/3=)9711MB per OSD. 1 and 3 are correct. 2 is presumably correct; it makes perfect sense and there's no reason to question it. 4 is not correct: # df -m [...] /dev/mapper/sda6 232003 26913 191832 13% /mnt/osd # grep /mnt/osd /etc/ceph/ceph.conf osd data = /mnt/osd # ls -a /mnt/osd/ . .. ceph_fsid current fsid lost+found magic whoami So the OSD lives in its own exclusive partition and nothing else uses that partition. The other node is done the same way. The "53579 MB used" reported by ceph matches the aggregated "Used" output of df -m on both nodes. And I checked, lost+found is empty on both. Something here is trying to be elusive (and is succeeding). Z -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
