Latest updates:
no error messages were found on the log files of the bricks.
The error messages appear on the client log files. Writing
from a second client also has the same errors.
Note that if I try to write a directory with 480 6Mb files
to /projects, over 3/4 of the files are written. It is
random which files are not written (i.e. it is not the
last 1/4 of the files which fail)
Hi,
Some additional data
[root@mseas-data save]# gluster volume info
Volume Name: gdata
Type: Distribute
Volume ID: eccc3a90-212d-4563-ae8d-10a77758738d
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: gluster-0-0:/mseas-data-0-0
Brick2: gluster-0-1:/mseas-data-0-1
Brick3: gluster-data:/data
[root@mseas-data save]# gluster volume status
Status of volume: gdata
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick gluster-0-0:/mseas-data-0-0 24009 Y 27006
Brick gluster-0-1:/mseas-data-0-1 24009 Y
7063
Brick gluster-data:/data 24010 N
8007
NFS Server on localhost 38467 Y
8013
NFS Server on gluster-0-1 38467 Y 10228
NFS Server on 10.1.1.10 38467 Y
3867
Noticing that the brick gluster-data:/data was appearing as N
in the "online" column, I tried (1) detaching gluster-data (using
gluster peer detach gluster-data issued from gluster-0-0), (2) removing
/var/lib/glusterd, (3) restarting glusterd on gluster-data,
(4) reattaching /gluster-data (using gluster peer probe gluster-data
issued from gluster-0-0) then (5) restart glusterd one more time on all
3 bricks. The brick gluster-data:/data still appears as N in the Online
column.
[root@mseas-data save]# gluster peer status
Number of Peers: 2
Hostname: gluster-0-1
Uuid: 393fc4a6-1573-4564-971e-1b1aec434167
State: Peer in Cluster (Connected)
Hostname: 10.1.1.10
Uuid: 3619440a-4ca3-4151-b62e-d4d6bf2e0c03
State: Peer in Cluster (Connected)
(similarly from the other bricks)
Ping works between all bricks too.
Hi,
We are using gluster to present 3 bricks as a single name space.
We appear to have a situation in which gluster thinks there
is no disk space when there is actually plenty. I have restarted
the glusterd deamons on all three bricks and I still get the
following message
/bin/cp: cannot create regular file
`./Bottom_Gravity_Current_25/344.mat': No space left on device
This is a 6Mbyte file. The total space available on
gluster is 3.6T
Filesystem Size Used Avail Use% Mounted on
mseas-data:/gdata 55T 51T 3.6T 94% /gdata
Also, no single brick is full:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/the_raid-lv_data
15T 14T 804G 95% /data
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 21T 18T 2.1T 90% /mseas-data-0-0
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 21T 20T 784G 97% /mseas-data-0-1
What should we do to fix this problem or look at to diagnose
this problem?
Thanks.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email: phaley@xxxxxxx
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA 02139-4301
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email: phaley@xxxxxxx
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213 http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA 02139-4301
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users
