Hello, We redid our setup with ext4 instead of xfs - and it seems to work fine. It sounds like there may be a bug in the integration of xfs + glusterfs. Thanks. On Fri, Jul 6, 2012 at 11:03 AM, Khawaja Shams <kshams at usc.edu> wrote: > Hello, > Thanks for the response. Does our create volume look correct? > 1) We are using XFS > 2) We are using: > > dd if=/dev/zero of=/mnt/home/testfile bs=16k count=16384 > > to write a 256MB file filled with 0s. I don't think that is a sparse file. > We also tried other methods of writing the file to the system, and each > file is consistently 40 times larger than we expect it to be. Should we > try a different fs? > > 3) We haven't tried the cluster.stripe-coalesce option. We will > investigate it now. Google didn't turn up any useful info on how to use > that at a first glance, but we will look a little further. > > > Thank you. > > On Fri, Jul 6, 2012 at 9:13 AM, Jeff Darcy <jdarcy at redhat.com> wrote: > >> On 07/06/2012 11:29 AM, Khawaja Shams wrote: >> > Hello, >> > We are trying to create a 40 volume gluster across 4 machines. For >> the >> > purposes of our benchmarks, we are trying to set it up without any >> replication. >> > However, files are taking up 40 times the storage space than we think. >> Putting >> > out a 256MB file with dd puts a 9.2GB file on the file system. The >> gluster FAQs >> > claim that ls doesn't track the file space properly and du does: >> > >> http://gluster.org/community/documentation//index.php/GlusterFS_Technical_FAQ#Stripe_behavior_not_working_as_expected >> > >> > However, in our case, ls shows the file to be 256MB, while du shows it >> to be >> > 9.2GB. After looking at the individual drives, we also noticed that >> each drive >> > has 256MB of data and it seems to be getting replicated. Here is how we >> create >> > the volumes: >> > >> > gluster create volume gluster-test stripe 40 transport tcp >> Gluster1:/data_f >> > Gluster1:/data_g Gluster1:/data_h Gluster1:/data_i Gluster1:/data_j >> > Gluster1:/data_k Gluster1:/data_l Gluster1:/data_m Gluster1:/data_n >> > Gluster1:/data_o Gluster2:/data_f Gluster2:/data_g Gluster2:/data_h >> > Gluster2:/data_i Gluster2:/data_j Gluster2:/data_k Gluster2:/data_l >> > Gluster2:/data_m Gluster2:/data_n Gluster2:/data_o Gluster3:/data_f >> > Gluster3:/data_g Gluster3:/data_h Gluster3:/data_i Gluster3:/data_j >> > Gluster3:/data_k Gluster3:/data_l Gluster3:/data_m Gluster3:/data_n >> > Gluster3:/data_o Gluster4:/data_f Gluster4:/data_g Gluster4:/data_h >> > Gluster4:/data_i Gluster4:/data_j Gluster4:/data_k Gluster4:/data_l >> > Gluster4:/data_m Gluster4:/data_n Gluster4:/data_o >> > >> > gluster --version >> > *glusterfs 3.3.0 built on Jul 2 2012 23:26:48* >> > Repository revision: git://git.gluster.com/glusterfs.git >> > <http://git.gluster.com/glusterfs.git> >> > Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com> >> > GlusterFS comes with ABSOLUTELY NO WARRANTY. >> > You may redistribute copies of GlusterFS under the terms of the GNU >> General >> > Public License. >> > >> > We do not have this issue if all the bricks are on the same machine. >> What are >> > we doing wrong? Are there specific logs we should be looking at? >> >> Three questions come to mind. >> >> (1) What underlying local filesystem (e.g. XFS, ext4) are you using for >> the bricks? >> >> (2) What exactly is the "dd" command you're using? Block size is >> particularly >> important, also whether you're trying to write a sparse file. >> >> (3) Have you tried turning on the "cluster.stripe-coalesce" volume >> option, and >> did it make a difference? >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://gluster.org/pipermail/gluster-users/attachments/20120706/ce50e98f/attachment.htm>
