Dear all,
today, we started using GlusterFS at our institute at a large scale. A
problem occurred for the first migrated users, that was already
discussed on this mailing list, but with an older version of glusterFS
and, as far as I could tell, without a solution.
When users wants to create or move a file, they sometimes get the error
message:
mv: cannot create regular file `/storage/cluster/<etc...>': No space
left on device
Doing df -h tells me, however:
glusterfs#192.168.101.247:/lemmy
104T 69T 36T 66% /storage/cluster
It may be of importance that one brick in the cluster is actually 100%
used. Others are almost completely empty. I am using GlusterFS.3.1.1,
the file servers are running debian lenny or ubuntu server 10.04,
clients are SLC4, SLC5, CentOS and ubuntu server 10.04.
The migration of the users was done in the following way:
The following volume was created:
Volume Name: lemmy
Type: Distribute
Status: Started
Number of Bricks: 5
Transport-type: tcp
Bricks:
Brick1: 192.168.101.249:/storage/4/cluster
Brick2: 192.168.101.248:/storage/5/cluster
Brick3: 192.168.101.250:/storage/6/cluster
Brick4: 192.168.101.247:/storage/7/cluster
Brick5: 192.168.101.246:/storage/8/cluster
The user directories were moved into one of the above folders
(whichever required the least amount of network traffic), using "find
-print" a file list was created and an "ls" was done on each line. All
the data can be read by the users, and for the most part, they can also
write to their directories. Not always, however, as described above.
Any advice is very much appreciated,
Best Regards,
Daniel
