I've just upgraded my gluster hosts from gluster 3.8 to 3.10 and it appears that geo-replication on my volume is now broken. Here are the log entries from the master. I've tried restarting the geo-replication process several times which did not help. Is there any way to resolve this? 2017-04-28 19:03:56.477895] I [monitor(monitor):275:monitor] Monitor: starting gsyncd worker(/var/mnt/gluster/brick2). Slave node: ssh://root@mdct-gluster-srv3:gluster://localhost:slavevol [2017-04-28 19:03:56.616667] I [changelogagent(/var/mnt/gluster/brick2):73:__init__] ChangelogAgent: Agent listining... [2017-04-28 19:04:07.780885] I [master(/var/mnt/gluster/brick2):1328:register] _GMaster: Working dir: /var/lib/misc/glusterfsd/gv0/ssh%3A%2F%2Froot%4010.112.215.10%3Agluster%3A%2F%2F127.0.0.1%3Aslavevol/920a96ad4a5f9c0c2bdbd24a14eeb1af [2017-04-28 19:04:07.781146] I [resource(/var/mnt/gluster/brick2):1604:service_loop] GLUSTER: Register time: 1493406247 [2017-04-28 19:04:08.143078] I [gsyncdstatus(/var/mnt/gluster/brick2):272:set_active] GeorepStatus: Worker Status: Active [2017-04-28 19:04:08.254343] I [gsyncdstatus(/var/mnt/gluster/brick2):245:set_worker_crawl_status] GeorepStatus: Crawl Status: History Crawl [2017-04-28 19:04:08.254739] I [master(/var/mnt/gluster/brick2):1244:crawl] _GMaster: starting history crawl... turns: 1, stime: (1493382958, 0), etime: 1493406248, entry_stime: (1493382958, 0) [2017-04-28 19:04:09.256428] I [master(/var/mnt/gluster/brick2):1272:crawl] _GMaster: slave's time: (1493382958, 0) [2017-04-28 19:04:09.381602] E [syncdutils(/var/mnt/gluster/brick2):297:log_raise_exception] <top>: FAIL: Traceback (most recent call last): File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 204, in main main_i() File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 780, in main_i local.service_loop(*[r for r in [remote] if r]) File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1610, in service_loop g3.crawlwrap(oneshot=True) File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 600, in crawlwrap self.crawl() File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1281, in crawl self.changelogs_batch_process(changes) File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1184, in changelogs_batch_process self.process(batch) File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1039, in process self.process_change(change, done, retry) File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 969, in process_change entry_stime_to_update[0]) File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 200, in set_field return self._update(merger) File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 161, in _update data = mergerfunc(data) File "/usr/libexec/glusterfs/python/syncdaemon/gsyncdstatus.py", line 194, in merger if data[key] == value: KeyError: 'last_synced_entry' [2017-04-28 19:04:09.383002] I [syncdutils(/var/mnt/gluster/brick2):238:finalize] <top>: exiting. [2017-04-28 19:04:09.387280] I [repce(/var/mnt/gluster/brick2):92:service_loop] RepceServer: terminating on reaching EOF. [2017-04-28 19:04:09.387507] I [syncdutils(/var/mnt/gluster/brick2):238:finalize] <top>: exiting. [2017-04-28 19:04:09.764077] I [monitor(monitor):357:monitor] Monitor: worker(/var/mnt/gluster/brick2) died in startup phase [2017-04-28 19:04:09.768179] I [gsyncdstatus(monitor):241:set_worker_status] GeorepStatus: Worker Status: Faulty _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users