tried ceph on 3 kvm instances, each with a root 40G drive, and 6 virtio disks of 4G each. when i look at available space, instead of some number less than 72G, i get 689G, and 154G used. the journal is in a folder on the root drive. the images were made with virt-builder using ubuntu-14.04 and virsh to attach the osd disks. set up was done with this set of playbooks, https://github.com/ceph/ceph-ansible with these settings, monitor_secret: FFFFFFFFFFFFFFFFFFFFFFFFF== monitor_interface: eth0 mon_osd_min_down_reporters: 7 mon_osd_full_ratio: .611 mon_osd_nearfull_ratio: .60 disable_swap: false cluster_network: 192.168.16.0/24 public_network: 192.168.16.0/24 devices: - /dev/vdb - /dev/vdc - /dev/vdd - /dev/vde - /dev/vdf - /dev/vdg journal_size: 1000 journal_collocation: false osd_directory: true osd_directories: - /var/lib/ceph/osd/j0 - /var/lib/ceph/osd/j1 - /var/lib/ceph/osd/j2 - /var/lib/ceph/osd/j3 - /var/lib/ceph/osd/j4 - /var/lib/ceph/osd/j5 root@ceph1:~# ceph -s cluster 2198abdb-2669-438a-8673-fc4f226a226c health HEALTH_OK monmap e1: 3 mons at {ceph1=192.168.16.31:6789/0,ceph2=192.168.16.32:6789/0,ceph3=192.168.16.33:6789/0}, election epoch 12, quorum 0,1,2 ceph1,ceph2,ceph3 osdmap e48: 18 osds: 18 up, 18 in pgmap v14445: 664 pgs, 2 pools, 1280 MB data, 337 objects 154 GB used, 499 GB / 689 GB avail 664 active+clean host:~/vstore$ for i in ceph*;do qemu-img info $i | grep "virtual size";done # the 40gig disks are the root and journal volumes. virtual size: 40G (42949672960 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 40G (42949672960 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 40G (42949672960 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) virtual size: 4.0G (4294967296 bytes) _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |