On Mon, 16 Sep 2013, Yan, Zheng wrote: > For cephfs, the size reported by 'ls -s' is the same as file size. see > http://ceph.com/docs/next/dev/differences-from-posix/ ...but the files are still in fact stored sparsely. It's just hard to tell. sage > > Regards > Yan, Zheng > > > On Mon, Sep 16, 2013 at 5:12 PM, Jens-Christian Fischer > <jens-christian.fischer@xxxxxxxxx> wrote: > > > > Hi all > > > > as part of moving our OpenStack VM instance store from dedicated disks on the physical hosts to a CephFS backed by an SSD pool, we noticed that the files created on CephFS aren't sparse, even though the original files were. > > > > This is on > > root@s2:~# ls -lhs /var/lib/nova/instances/_base > > total 63G > > 750M -rw-r--r-- 1 nova nova 2.0G Jul 10 21:40 1a11de23fe75a210b4da631366513cb7c22ef311 > > 750M -rw-r--r-- 1 libvirt-qemu kvm 10G Jul 10 21:40 1a11de23fe75a210b4da631366513cb7c22ef311_10 > > ? > > > > vs > > > > root@s2:~# ls -lhs /mnt/instances/instances/_base/1a11de23fe75a210b4da631366513cb7c22ef311* > > 1.2G -rw-r--r-- 1 nova nova 1.2G Sep 5 16:56 /mnt/instances/instances/_base/1a11de23fe75a210b4da631366513cb7c22ef311 > > 10G -rw-r--r-- 1 libvirt-qemu kvm 10G Jul 10 21:40 /mnt/instances/instances/_base/1a11de23fe75a210b4da631366513cb7c22ef311_10 > > > > We have used different ways of copying the files (tar and rsync) and specified the sparse options: > > > > # rsync -rtvupogS -h /var/lib/nova/instances/ /mnt/instances/instances > > or > > # (cd /var/lib/nova/instances ; tar -Svcf - .)|(cd /mnt/instances/instances ; tar Sxpf -) > > > > The OSDs we use for this pool are backed by XFS (which has a problem with sparse files, unless one specifies allocation block size options in the mounts) http://serverfault.com/questions/406069/why-are-my-xfs-filesystems-suddenly-consuming-more-space-and-full-of-sparse-file, https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=055388a3188f56676c21e92962fc366ac8b5cb72. We have mounted the XFS partitions for the OSDs with this option, but I assume that this shouldn't impact the way CephFS handles sparse files. > > > > I seem to remember that the copying of sparse files worked a couple of months ago (ceph-fs kernel 3.5 on btrfs OSDs), but now we used Kernel 3.10 and recently ceph-fuse to mount the CephFS. > > > > Are we doing something wrong, or is this not supported by CephFS? > > > > cheers > > jc > > > > > > > > > > > > -- > > SWITCH > > Jens-Christian Fischer, Peta Solutions > > Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland > > phone +41 44 268 15 15, direct +41 44 268 15 71 > > jens-christian.fischer@xxxxxxxxx > > http://www.switch.ch > > > > http://www.switch.ch/socialmedia > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@xxxxxxxxxxxxxx > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |