Re: Upgraded from 3.4.1 to 3.5.2, quota no longer working

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Certainly, thank you for your response:

Quotad is running on all nodes:
[root@gfs-a-1 ~]# ps aux | grep quotad
root      3004  0.0  0.4 241368 68552 ?        Ssl  Nov30   0:07 /usr/local/sbin/glusterfs -s localhost --volfile-id gluster/quotad -p /var/lib/glusterd/quotad/run/quotad.pid -l /usr/local/var/log/glusterfs/quotad.log -S /var/run/9d02605105ef0e74d913a4671c1143a1.socket --xlator-option *replicate*.data-self-heal=off --xlator-option *replicate*.metadata-self-heal=off --xlator-option *replicate*.entry-self-heal=off

And the relevant output from gluster volume status shares per your request:
[root@gfs-a-1 ~]# gluster volume status shares | grep Quota
Quota Daemon on localhost                               N/A     Y       3004
Quota Daemon on gfs-a-3                                 N/A     Y       32307
Quota Daemon on gfs-a-4                                 N/A     Y       10818
Quota Daemon on gfs-a-2                                 N/A     Y       12292
 
 No log entries are created in /var/log/glusterfs/quotad.log when I run a quota list; all of the log entries are from yesterday. They do indicate a version mis-match, although I can't seem to locate where that version is specified:
[2014-11-30 13:21:55.173081] I [client-handshake.c:1474:client_setvolume_cbk] 0-shares-client-14: Server and Client lk-version numbers are not same, reopening the fds
[2014-11-30 13:21:55.173170] I [client-handshake.c:450:client_set_lk_version_cbk] 0-shares-client-14: Server lk version = 1
[2014-11-30 13:21:55.178739] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-shares-client-9: changing port to 49154 (from 0)
[2014-11-30 13:21:55.181170] I [client-handshake.c:1677:select_server_supported_programs] 0-shares-client-9: Using Program GlusterFS 3.3, Num (1298437), Version (330)
[2014-11-30 13:21:55.181386] I [client-handshake.c:1462:client_setvolume_cbk] 0-shares-client-9: Connected to 172.16.10.13:49154, attached to remote volume '/mnt/a-3-shares-brick-3/brick'.
[2014-11-30 13:21:55.181401] I [client-handshake.c:1474:client_setvolume_cbk] 0-shares-client-9: Server and Client lk-version numbers are not same, reopening the fds
[2014-11-30 13:21:55.181535] I [client-handshake.c:450:client_set_lk_version_cbk] 0-shares-client-9: Server lk version = 1

I see the operational mode for the volume as "3". I saw a non-related thread that indicated this number should be more digits on a cluster running 3.5.2. The other thread also indicated that quota may not work if the volume version number was not compatible with the quota version running on the cluster. I can't seem to find the link right now.

It's almost as if the volume version did not get upgraded when the server version was upgraded. Is that possible?

Cheers,
Dave


On Sun, Nov 30, 2014 at 11:46 PM, Krutika Dhananjay <kdhananj@xxxxxxxxxx> wrote:
Hi,

Could you confirm whether quotad (Quota Daemon) is online from the output of `gluster volume status shares`?

Also, could you share quota daemon's log file from the node where you executed `quota list` command, which you will find at @ /var/log/glusterfs/quotad.log?

-Krutika


From: "David Gibbons" <david.c.gibbons@xxxxxxxxx>
To: "gluster-users" <gluster-users@xxxxxxxxxxx>
Sent: Sunday, November 30, 2014 9:57:02 PM
Subject: Upgraded from 3.4.1 to 3.5.2,        quota no longer working


Hi All,

I performed a long-awaited upgrade from 3.4.1 to 3.5.2 today following the instructions for an offline upgrade outlined here:

I ran the pre- and post- upgrade scripts as instructed, intending to move the quotas over to the new version. The upgrade seemed to go well, the volume is online and it appears to be functioning properly.

When I attempt to check quotas, the list is empty:
[root@gfs-a-1 glusterfs]# gluster volume quota shares list
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
[root@gfs-a-1 glusterfs]#


And upon execution of that command, the cli.log file fills up with entries like this. I am assuming it's one cli log entry per quota entry:
[2014-11-30 14:00:02.154143] W [cli-rpc-ops.c:2469:print_quota_list_from_quotad] 0-cli: path key is not present in dict
[2014-11-30 14:00:02.160507] W [cli-rpc-ops.c:2469:print_quota_list_from_quotad] 0-cli: path key is not present in dict
[2014-11-30 14:00:02.167947] W [cli-rpc-ops.c:2469:print_quota_list_from_quotad] 0-cli: path key is not present in dic

So, it appears that somehow the quota database has become offline or corrupt. Any thoughts on what I can do to resolve this?

I have checked all of the binaries on all 4 machines in the cluster, and they all appear to be running the correct version:
[root@gfs-a-1 glusterfs]# glusterfsd --version
glusterfs 3.5.2 built on Nov 30 2014 08:16:37

Cheers,
Dave 

_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users


_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://supercolony.gluster.org/mailman/listinfo/gluster-users

[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux