Hi,
Geo-replication has three types of Change detection(To identify the list
of files changed and to sync only those files)
1. XTime based Brick backend Crawl for initial sync
2. Historical Changelogs to sync backlogs(Files created/modified/deleted
between Worker down and start)
3. Live Changelogs - As and when changelog is rolled over, process it
and sync the changes
If initial data available in Master Volume before Geo-replication
session is created, then it does XTime based Crawl(Hybrid Crawl) and
then switches to Live Changelog mode.
After initial sync, Xtime crawl will not be used. On worker restart it
uses Historical changelogs and then switches to Live Changelogs.
Geo-replication is very slow during History Crawl if backlog changelogs
grows(If Geo-rep session was down for long time).
- If a same file is Created, deleted and again created, Geo-rep is
replaying the changelogs in the same manner in Slave side.
- Data sync happens GFID to GFID, So except the final GFID sync all the
other sync will fail since file not exists in Master(File may exist but
with different GFID)
Due to these data sync and retries, Geo-rep performance is affected.
Me and Kotresh discussed about the same and came up with following
changes to Geo-replication
While processing History,
- Collect all the entry, data and meta operations in a temporary database
- Delete all Data and Meta GFIDs which are already unlinked as per
Changelogs
- Process all Entry operations in batch
- Process data and meta operations in batch
- Once the sync is complete, Update last Changelog's time as last_synced
time as usual.
Challenges:
- If worker crashes in between while doing above steps, on restart same
changelogs will be reprocessed.(Crawl done in small batches in existing,
so on failure reprocess only last partially completed last batch)
Some of the retries can be avoided if we start maintaining details
about entry_last_synced(entry_stime) and data_last_synced(stime) separately.
Let us know if any suggestions.
@Kotresh, Please add if I missed anything.
--
regards
Aravinda
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
