Hi Strahil,
Thank you for your response. when I tail logs on both master and slave I get this:
on slave, from /var/log/glusterfs/geo-replication-slaves/<geo-session>/mnt-XXX.log file:
[2020-03-11 19:53:32.721509] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f78e10488ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f78d83f6221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f78d83f7998] (--> /lib64/libpthread.so.0(+0x7e65)[0x7f78dfe89e65] (--> /lib64/libc.so.6(clone+0x6d)[0x7f78df74f88d] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:53:32.723758] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f78e10488ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f78d83f6221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f78d83f7998] (--> /lib64/libpthread.so.0(+0x7e65)[0x7f78dfe89e65] (--> /lib64/libc.so.6(clone+0x6d)[0x7f78df74f88d] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:53:32.723758] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f78e10488ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f78d83f6221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f78d83f7998] (--> /lib64/libpthread.so.0(+0x7e65)[0x7f78dfe89e65] (--> /lib64/libc.so.6(clone+0x6d)[0x7f78df74f88d] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
on master, from /var/log/glusterfs/geo-replication/<geo-session>/mnt-XXX.log file:
[2020-03-11 19:40:55.872002] E [fuse-bridge.c:4188:fuse_xattr_cbk] 0-glusterfs-fuse: extended attribute not supported by the backend storage
[2020-03-11 19:40:58.389748] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f1f4b9108ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f1f42cc2221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f1f42cc3998] (--> /lib64/libpthread.so.0(+0x7e25)[0x7f1f4a751e25] (--> /lib64/libc.so.6(clone+0x6d)[0x7f1f4a01abad] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:41:08.214591] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f1f4b9108ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f1f42cc2221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f1f42cc3998] (--> /lib64/libpthread.so.0(+0x7e25)[0x7f1f4a751e25] (--> /lib64/libc.so.6(clone+0x6d)[0x7f1f4a01abad] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:53:59.275469] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f1f4b9108ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f1f42cc2221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f1f42cc3998] (--> /lib64/libpthread.so.0(+0x7e25)[0x7f1f4a751e25] (--> /lib64/libc.so.6(clone+0x6d)[0x7f1f4a01abad] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:40:58.389748] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f1f4b9108ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f1f42cc2221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f1f42cc3998] (--> /lib64/libpthread.so.0(+0x7e25)[0x7f1f4a751e25] (--> /lib64/libc.so.6(clone+0x6d)[0x7f1f4a01abad] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:41:08.214591] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f1f4b9108ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f1f42cc2221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f1f42cc3998] (--> /lib64/libpthread.so.0(+0x7e25)[0x7f1f4a751e25] (--> /lib64/libc.so.6(clone+0x6d)[0x7f1f4a01abad] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
[2020-03-11 19:53:59.275469] E [fuse-bridge.c:227:check_and_dump_fuse_W] (--> /lib64/libglusterfs.so.0(_gf_log_callingfn+0x13a)[0x7f1f4b9108ea] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x8221)[0x7f1f42cc2221] (--> /usr/lib64/glusterfs/7.3/xlator/mount/fuse.so(+0x9998)[0x7f1f42cc3998] (--> /lib64/libpthread.so.0(+0x7e25)[0x7f1f4a751e25] (--> /lib64/libc.so.6(clone+0x6d)[0x7f1f4a01abad] ))))) 0-glusterfs-fuse: writing to fuse device failed: No such file or directory
####################gsyncd.log outputs:######################
from slave:
[2020-03-11 08:55:16.384085] I [repce(slave master-node/srv/media-storage):96:service_loop] RepceServer: terminating on reaching EOF.
[2020-03-11 08:57:55.87364] I [resource(slave master-node/srv/media-storage):1105:connect] GLUSTER: Mounting gluster volume locally...
[2020-03-11 08:57:56.171372] I [resource(slave master-node/srv/media-storage):1128:connect] GLUSTER: Mounted gluster volume duration=1.0837
[2020-03-11 08:57:56.173346] I [resource(slave master-node/srv/media-storage):1155:service_loop] GLUSTER: slave listening
[2020-03-11 08:57:55.87364] I [resource(slave master-node/srv/media-storage):1105:connect] GLUSTER: Mounting gluster volume locally...
[2020-03-11 08:57:56.171372] I [resource(slave master-node/srv/media-storage):1128:connect] GLUSTER: Mounted gluster volume duration=1.0837
[2020-03-11 08:57:56.173346] I [resource(slave master-node/srv/media-storage):1155:service_loop] GLUSTER: slave listening
from master:
[2020-03-11 20:08:55.145453] I [master(worker /srv/media-storage):1991:syncjob] Syncer: Sync Time Taken duration=134.9987num_files=4661 job=2 return_code=0
[2020-03-11 20:08:55.285871] I [master(worker /srv/media-storage):1421:process] _GMaster: Entry Time Taken MKD=83 MKN=8109 LIN=0 SYM=0 REN=0 RMD=0 CRE=0 duration=17.0358 UNL=0
[2020-03-11 20:08:55.286082] I [master(worker /srv/media-storage):1431:process] _GMaster: Data/Metadata Time Taken SETA=83 SETX=0 meta_duration=0.9334 data_duration=135.2497 DATA="" XATT=0
[2020-03-11 20:08:55.286410] I [master(worker /srv/media-storage):1441:process] _GMaster: Batch Completed changelog_end=1583917610 entry_stime=None changelog_start=1583917610 stime=None duration=153.5185 num_changelogs=1 mode=xsync
[2020-03-11 20:08:55.315442] I [master(worker /srv/media-storage):1681:crawl] _GMaster: processing xsync changelog path=/var/lib/misc/gluster/gsyncd/media-storage_daredevil01.zingat.com_dr-media/srv-media-storage/xsync/XSYNC-CHANGELOG.1583917613
[2020-03-11 20:08:55.285871] I [master(worker /srv/media-storage):1421:process] _GMaster: Entry Time Taken MKD=83 MKN=8109 LIN=0 SYM=0 REN=0 RMD=0 CRE=0 duration=17.0358 UNL=0
[2020-03-11 20:08:55.286082] I [master(worker /srv/media-storage):1431:process] _GMaster: Data/Metadata Time Taken SETA=83 SETX=0 meta_duration=0.9334 data_duration=135.2497 DATA="" XATT=0
[2020-03-11 20:08:55.286410] I [master(worker /srv/media-storage):1441:process] _GMaster: Batch Completed changelog_end=1583917610 entry_stime=None changelog_start=1583917610 stime=None duration=153.5185 num_changelogs=1 mode=xsync
[2020-03-11 20:08:55.315442] I [master(worker /srv/media-storage):1681:crawl] _GMaster: processing xsync changelog path=/var/lib/misc/gluster/gsyncd/media-storage_daredevil01.zingat.com_dr-media/srv-media-storage/xsync/XSYNC-CHANGELOG.1583917613
Thank you..
Strahil Nikolov <hunter86_bg@xxxxxxxxx>, 11 Mar 2020 Çar, 12:28 tarihinde şunu yazdı:
On March 11, 2020 10:09:27 AM GMT+02:00, "Etem Bayoğlu" <etembayoglu@xxxxxxxxx> wrote:
>Hello community,
>
>I've set up a glusterfs geo-replication node for disaster recovery. I
>manage about 10TB media data on a gluster volume and I want to sync all
>data to remote location over WAN. So, I created a slave node volume on
>disaster recovery center on remote location and I've started geo-rep
>session. It has been transferred data fine up to about 800GB, but
>syncing
>has stopped for three days despite gluster geo-rep status active and
>hybrid
>crawl. There is no sending data. I've recreated session and restarted
>but
>still the same.
>
>#gluster volu geo-rep status
>
>MASTER NODE MASTER VOL MASTER BRICK SLAVE
>USER
>SLAVE SLAVE NODE
>STATUS
> CRAWL STATUS LAST_SYNCED
>------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>master-node media-storage /srv/media-storage root
> ssh://slave-node::dr-media slave-node Active
>Hybrid Crawl N/A
>
>Any idea? please. Thank you.
Hi Etem,
Have you checked the log on both source and destination. Maybe they can hint you what the issue is.
Best Regards,
Strahil Nikolov
________ Community Meeting Calendar: Schedule - Every Tuesday at 14:30 IST / 09:00 UTC Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
