Let me tell how it works currently, and why. When a file is created on filesystem, there is no way a for a filesystem to know about its future size. It may be just a 0byte file, may be 4k, may be 1GB, may be 100GB. Hence, while creating a file, GlusterFS just creates it on the best possible node (depending on the scheduler) at the time of creation. Hope that explains the reason why the 40GB file got created on the afr (gluster1/2). ---- snip ---- |-------------- u n i f y ----------------| | gluster1 <-- afr ---> gluster2 | | gluster3 <-- afr ---> gluster4 | | gluster5 <-- afr ---> gluster6 | | gluster7 <-- afr ---> gluster8 | |-------------- u n i f y ----------------| gluster1 / 2 has 8gb free diskspace gluster3 / 4 has 2 gb free diskspace (*) gluster5 / 6 has 2 gb free diskspace (*) gluster7 / 8 has 2 tb free diskspace (*) => no writes due to alu.limits.min-free-disk 6GB Now, when i try to copy a large file (40gb) into the gluster it starts writing to gluster1/2. after 8gb gluster1/2 has no more free diskspace and the copy process dies. Is there a configuration option for unify to write on an other gluster-server if one server run out of space during a write? ---- end-snip ---- Well, then it would be great if we could handle such cases too. (Its on the roadmap) currently there is a switch scheduler in 1.3.8pre1 or latest tla archives, where you can specify where the file should go if it matches a specific pattern. (this helps if you have a pattern for all the large files). Another way of tackling this problem is using stripe, where you can stripe your file across, but well, it may not suit everyone. It would be an ideal case if users had a way to tell about the possible filesize during the creation. There is no such option in any of the currently available kernels/system calls. Now, we can have discussion on how to handle such situation if anyone has idea. Please do think about all the parameters before suggesting one. For example I can say "well, when the file reaches 6GB on afr (gluster1/2) you are just left with 2GB there, hence, move it to afr (gluster7/8) where you have 2TB', but now, while a single 'write()' gets the error ENOSPC, I have to move whole of 6GB from one node to another internally, which stops the application for that long (by that time, 'transport-timeout' would be called too). Next problem is where will you go if the file size reaches 2TB ? Open for design ideas. Thanks and regards, Amar > Regards, > > jan > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxx > http://lists.nongnu.org/mailman/listinfo/gluster-devel > -- Amar Tumballi Gluster/GlusterFS Hacker [bulde on #gluster/irc.gnu.org] http://www.zresearch.com - Commoditizing Supercomputing and Superstorage!
