[bug] git svn fetch: defect history - missing merges and wrong tag ancestors

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Dear all,

I finally had the time to start converting some older Subversion repositories to git repositiers and run into issues with repos using "parallel" branches, so-called vendor branches [1] in Subversion to track upstream changes via snapshots in a separate branch and merge them into the custom build on trunk.

Just wanting to convert the Subversion history as-is to git.
I studied the git-svn reference docs [2] plus the related chapter of the ProGit book [3] and assume that I understood how git-svn works.
Still I'm not a git expert, just a sporadic user.

Somehow `git svn fetch` (2.29.2.windows.3) looses merge information between vendor branch and trunk plus tags are referencing the predecessors instead of the original ancestor. Maybe there is a manual way to fix this for that small repository, but it wouldn't be feasible for larger repositories, that's why I deceided to write this bug report.
Is there anything I missed? (see my procedure below)

Fortunately I ran quite early into these issues and also with a very small repository of just 11 commits, so I can provide a small reproduction case (see below after the links).
Hoping you can enhance `git svn fetch`.

Any help is appreciated, thanks in advance
Matthias Bücher


[Links]
[1] http://svnbook.red-bean.com/en/1.8/svn.advanced.vendorbr.html#svn.advanced.vendorbr.mirrored-sources
[2] https://git-scm.com/docs/git-svn
[3] https://git-scm.com/book/en/v2/Git-and-Other-Systems-Migrating-to-Git


[System Info]
git version:
git version 2.29.2.windows.3
cpu: x86_64
built from commit: d054eb1fc46ff23e7c95756a7c747e2f2864b478
sizeof-long: 4
sizeof-size_t: 8
shell-path: /bin/sh
uname: Windows 10.0 19042
compiler info: gnuc: 10.2
libc info: no libc information available
$SHELL (typically, interactive shell): C:\Program Files\Git\usr\bin\bash.exe

[Enabled Hooks]
none


[Commits in Subversion]
A Subversion repository dump is attached, plus a test where I recreated the same history directly in a git repository without an issue.

* trunk:  a------------d--e--h--i--j--k
*                     /     /
* vendor: a (empty)--b-----f
*                    ^     ^
* tags:              c     g

* Vendor releases: b, f
* Custom modifications: e, i, j, k
* Tags: really just used as tags, although Subversion internally they are branches. Therefore wanting to create lightweight git tags, although annotated git tags would be fine too.


[Expected Subversion to git repo conversion]
* trunk => branch "main"
* branches/* => branch "*"
* tags/* => tag "*"
* vendor/current => branch "vendor/current" (can be renamed later to just "vendor")
* vendor/* => tag "vendor/*" (except for vendor/current)

Expected "tags":
vendor/5.4 = rev b (maybe c when annotated)
vendor/5.8 = rev f (maybe g when annotated)


[Wrong Results]
Merges from "vendor/current" branch to trunk get lost.
Tags are referencing the predecessors of the expected commit.


[Procedure]
```
### a) preparation
cd /d/Coding
#
cd dd-formmailer
svn log --xml --quiet | grep author | sort -u | perl -pe 's/.*>(.*?)<.*/$1 = /' > authors-transform.txt
## edit authors-transform.txt accordingly

### b) git-svn adapted from ProGit book, but with "svn/" prefix
cd /d/Coding
git svn init --stdlayout --no-metadata --prefix="svn/" -- 'svn+ssh://svn@vcs/dd-formmailer' dd-formmailer-git
cd dd-formmailer-git
## edit .git/config for additional vendor branches and tags
<< __EOF
...
[svn-remote "svn"]
	...
	fetch = trunk:refs/remotes/svn/trunk
# branches = vendor/current:refs/remotes/svn/vendor/current ## non-glob definition not working for branches
	branches = vendor/{current}:refs/remotes/svn/vendor/*
	branches = branches/*:refs/remotes/svn/*
	tags = vendor/*:refs/remotes/svn/tags/vendor/*
	tags = tags/*:refs/remotes/svn/tags/*
__EOF
##
cat .git/config
git svn fetch --authors-file ../dd-formmailer/authors-transform.txt
#
git branch -vv --list ; git for-each-ref
gitk --all &
```

[Log]
$ cat .git/config
[core]
        repositoryformatversion = 0
        filemode = false
        bare = false
        logallrefupdates = true
        symlinks = false
        ignorecase = true
[svn-remote "svn"]
        noMetadata = 1
        url = svn+ssh://svn@vcs/dd-formmailer
        fetch = trunk:refs/remotes/svn/trunk
        branches = vendor/{current}:refs/remotes/svn/vendor/*
        branches = branches/*:refs/remotes/svn/*
        tags = vendor/*:refs/remotes/svn/tags/vendor/*
        tags = tags/*:refs/remotes/svn/tags/*

$ git svn fetch --authors-file ../dd-formmailer/authors-transform.txt
r1 = 158d53044a5628897379403647a19ea13594b532 (refs/remotes/svn/vendor/current)
        A       _svn__guideline.txt
        A       _svn_client_config.txt
        A       _svn_dir_ignore_list.txt
r1 = a795b654edbc296e6f38da398f032a4851fd0a9e (refs/remotes/svn/trunk)
        A       dd-formmailer.css
        A       dd-formmailer.php
        A       lang/BrazilianPortuguese.php
        A       lang/Catalan.php
        A       lang/Danish.php
        A       lang/Deutsch.php
        A       lang/Dutch.php
        A       lang/English.php
        A       lang/Finnish.php
        A       lang/French.php
        A       lang/Greek.php
        A       lang/Italian.php
        A       lang/NorwegianBokmaal.php
        A       lang/Polish.php
        A       lang/Portuguese.php
        A       lang/Romanian.php
        A       lang/Russian.php
        A       lang/Slovak.php
        A       lang/Slovene.php
        A       lang/Spanish.php
        A       lang/Swedish.php
        A       lang/Turkish.php
        A       recaptchalib.php
r2 = 8fd8668dbed2dde6b55306c71b3b629f5ed794ec (refs/remotes/svn/vendor/current) Found possible branch point: svn+ssh://svn@vcs/dd-formmailer/vendor/current => svn+ssh://svn@vcs/dd-formmailer/vendor/5.4, 1 Found branch parent: (refs/remotes/svn/tags/vendor/5.4) 158d53044a5628897379403647a19ea13594b532
Following parent with do_switch
        A       dd-formmailer.css
        A       dd-formmailer.php
        A       lang/BrazilianPortuguese.php
        A       lang/Catalan.php
        A       lang/Danish.php
        A       lang/Deutsch.php
        A       lang/Dutch.php
        A       lang/English.php
        A       lang/Finnish.php
        A       lang/French.php
        A       lang/Greek.php
        A       lang/Italian.php
        A       lang/NorwegianBokmaal.php
        A       lang/Polish.php
        A       lang/Portuguese.php
        A       lang/Romanian.php
        A       lang/Russian.php
        A       lang/Slovak.php
        A       lang/Slovene.php
        A       lang/Spanish.php
        A       lang/Swedish.php
        A       lang/Turkish.php
        A       recaptchalib.php
Successfully followed parent
r3 = 7ab4f436cafc8af3ed2e727a6d8cbef1a8f8b39f (refs/remotes/svn/tags/vendor/5.4)
        A       dd-formmailer.css
        A       dd-formmailer.php
        A       lang/BrazilianPortuguese.php
        A       lang/Catalan.php
        A       lang/Danish.php
        A       lang/Deutsch.php
        A       lang/Dutch.php
        A       lang/English.php
        A       lang/Finnish.php
        A       lang/French.php
        A       lang/Greek.php
        A       lang/Italian.php
        A       lang/NorwegianBokmaal.php
        A       lang/Polish.php
        A       lang/Portuguese.php
        A       lang/Romanian.php
        A       lang/Russian.php
        A       lang/Slovak.php
        A       lang/Slovene.php
        A       lang/Spanish.php
        A       lang/Swedish.php
        A       lang/Turkish.php
        A       recaptchalib.php
r4 = 7ec3663fd8e7c9fcfeb2742968a948d6978776d4 (refs/remotes/svn/trunk)
        M       dd-formmailer.css
        M       dd-formmailer.php
        A       dd-verify.php
r5 = e65ba2bdcd12a1935c1a327507dad6b7117f452b (refs/remotes/svn/trunk)
        A       calendar.gif
        A       date_chooser.js
        M       dd-formmailer.css
        M       dd-formmailer.php
        A       lang/Belarussian.php
        A       lang/Czech.php
        A       lang/Estonian.php
        A       lang/Japanese.php
        A       lang/Vietnamese.php
        M       recaptchalib.php
r6 = 9f95df7ce49c5d1cb4715017d223a8bd1c8dcffc (refs/remotes/svn/vendor/current) Found possible branch point: svn+ssh://svn@vcs/dd-formmailer/vendor/current => svn+ssh://svn@vcs/dd-formmailer/vendor/5.8, 3 Found branch parent: (refs/remotes/svn/tags/vendor/5.8) 8fd8668dbed2dde6b55306c71b3b629f5ed794ec
Following parent with do_switch
        A       calendar.gif
        A       date_chooser.js
        M       dd-formmailer.css
        M       dd-formmailer.php
        A       lang/Belarussian.php
        A       lang/Czech.php
        A       lang/Estonian.php
        A       lang/Japanese.php
        A       lang/Vietnamese.php
        M       recaptchalib.php
Successfully followed parent
r7 = cc00cb187386298cf974dda69f151d2ad4795917 (refs/remotes/svn/tags/vendor/5.8)
        A       calendar.gif
        A       date_chooser.js
        M       dd-formmailer.css
        M       dd-formmailer.php
        M       dd-verify.php
        A       lang/Belarussian.php
        A       lang/Czech.php
        A       lang/Estonian.php
        A       lang/Japanese.php
        A       lang/Vietnamese.php
        M       recaptchalib.php
Checking svn:mergeinfo changes since r5: 1 sources, 1 changed
W: Cannot find common ancestor between e65ba2bdcd12a1935c1a327507dad6b7117f452b and 9f95df7ce49c5d1cb4715017d223a8bd1c8dcffc. Ignoring merge info.
r8 = 9a5cd8f55f377f469280171cd87219b8f528c693 (refs/remotes/svn/trunk)
        M       dd-formmailer.php
r9 = 8143782376d9fbc91a9181b367b72701205b017f (refs/remotes/svn/trunk)
        M       dd-formmailer.php
r10 = 6f563fc4c511ddcc53c2ffc5d26c1e116725bc6b (refs/remotes/svn/trunk)
        M       _svn_client_config.txt
r11 = e9130854178d2c2743a981f303cdd7f34e54b052 (refs/remotes/svn/trunk)
svnserve: E210002: Network connection closed unexpectedly
Checked out HEAD:
  svn+ssh://svn@vcs/dd-formmailer/trunk r11

$ git branch -vv --list ; git for-each-ref
* main e913085 Updated client config for Windows Scripts
e9130854178d2c2743a981f303cdd7f34e54b052 commit refs/heads/main
7ab4f436cafc8af3ed2e727a6d8cbef1a8f8b39f commit refs/remotes/svn/tags/vendor/5.4 cc00cb187386298cf974dda69f151d2ad4795917 commit refs/remotes/svn/tags/vendor/5.8
e9130854178d2c2743a981f303cdd7f34e54b052 commit refs/remotes/svn/trunk
9f95df7ce49c5d1cb4715017d223a8bd1c8dcffc commit refs/remotes/svn/vendor/current
Procerdure:
===========

### a) preparation
cd /d/Coding
#
cd dd-formmailer
svn log --xml --quiet | grep author | sort -u | perl -pe 's/.*>(.*?)<.*/$1 = /' > authors-transform.txt
## edit authors-transform.txt accordingly

### b) git-svn adapted from ProGit book, but with "svn/" prefix
cd /d/Coding
git svn init --stdlayout --no-metadata --prefix="svn/" -- 'svn+ssh://svn@vcs/dd-formmailer' dd-formmailer-git
cd dd-formmailer-git
## edit .git/config for additional branches and tags
<< __EOF
...
[svn-remote "svn"]
	...
	fetch = trunk:refs/remotes/svn/trunk
	branches = vendor/{current}:refs/remotes/svn/vendor/*
	branches = branches/*:refs/remotes/svn/*
	tags = vendor/*:refs/remotes/svn/tags/vendor/*
	tags = tags/*:refs/remotes/svn/tags/*
__EOF
##
cat .git/config
git svn fetch --authors-file ../dd-formmailer/authors-transform.txt
#
git branch -vv --list ; git for-each-ref
gitk --all &


Log:
====
$ cat .git/config
[core]
        repositoryformatversion = 0
        filemode = false
        bare = false
        logallrefupdates = true
        symlinks = false
        ignorecase = true
[svn-remote "svn"]
        noMetadata = 1
        url = svn+ssh://svn@vcs/dd-formmailer
        fetch = trunk:refs/remotes/svn/trunk
        branches = vendor/{current}:refs/remotes/svn/vendor/*
        branches = branches/*:refs/remotes/svn/*
        tags = vendor/*:refs/remotes/svn/tags/vendor/*
        tags = tags/*:refs/remotes/svn/tags/*

$ git svn fetch --authors-file ../dd-formmailer/authors-transform.txt
r1 = 158d53044a5628897379403647a19ea13594b532 (refs/remotes/svn/vendor/current)
        A       _svn__guideline.txt
        A       _svn_client_config.txt
        A       _svn_dir_ignore_list.txt
r1 = a795b654edbc296e6f38da398f032a4851fd0a9e (refs/remotes/svn/trunk)
        A       dd-formmailer.css
        A       dd-formmailer.php
        A       lang/BrazilianPortuguese.php
        A       lang/Catalan.php
        A       lang/Danish.php
        A       lang/Deutsch.php
        A       lang/Dutch.php
        A       lang/English.php
        A       lang/Finnish.php
        A       lang/French.php
        A       lang/Greek.php
        A       lang/Italian.php
        A       lang/NorwegianBokmaal.php
        A       lang/Polish.php
        A       lang/Portuguese.php
        A       lang/Romanian.php
        A       lang/Russian.php
        A       lang/Slovak.php
        A       lang/Slovene.php
        A       lang/Spanish.php
        A       lang/Swedish.php
        A       lang/Turkish.php
        A       recaptchalib.php
r2 = 8fd8668dbed2dde6b55306c71b3b629f5ed794ec (refs/remotes/svn/vendor/current)
Found possible branch point: svn+ssh://svn@vcs/dd-formmailer/vendor/current => svn+ssh://svn@vcs/dd-formmailer/vendor/5.4, 1
Found branch parent: (refs/remotes/svn/tags/vendor/5.4) 158d53044a5628897379403647a19ea13594b532
Following parent with do_switch
        A       dd-formmailer.css
        A       dd-formmailer.php
        A       lang/BrazilianPortuguese.php
        A       lang/Catalan.php
        A       lang/Danish.php
        A       lang/Deutsch.php
        A       lang/Dutch.php
        A       lang/English.php
        A       lang/Finnish.php
        A       lang/French.php
        A       lang/Greek.php
        A       lang/Italian.php
        A       lang/NorwegianBokmaal.php
        A       lang/Polish.php
        A       lang/Portuguese.php
        A       lang/Romanian.php
        A       lang/Russian.php
        A       lang/Slovak.php
        A       lang/Slovene.php
        A       lang/Spanish.php
        A       lang/Swedish.php
        A       lang/Turkish.php
        A       recaptchalib.php
Successfully followed parent
r3 = 7ab4f436cafc8af3ed2e727a6d8cbef1a8f8b39f (refs/remotes/svn/tags/vendor/5.4)
        A       dd-formmailer.css
        A       dd-formmailer.php
        A       lang/BrazilianPortuguese.php
        A       lang/Catalan.php
        A       lang/Danish.php
        A       lang/Deutsch.php
        A       lang/Dutch.php
        A       lang/English.php
        A       lang/Finnish.php
        A       lang/French.php
        A       lang/Greek.php
        A       lang/Italian.php
        A       lang/NorwegianBokmaal.php
        A       lang/Polish.php
        A       lang/Portuguese.php
        A       lang/Romanian.php
        A       lang/Russian.php
        A       lang/Slovak.php
        A       lang/Slovene.php
        A       lang/Spanish.php
        A       lang/Swedish.php
        A       lang/Turkish.php
        A       recaptchalib.php
r4 = 7ec3663fd8e7c9fcfeb2742968a948d6978776d4 (refs/remotes/svn/trunk)
        M       dd-formmailer.css
        M       dd-formmailer.php
        A       dd-verify.php
r5 = e65ba2bdcd12a1935c1a327507dad6b7117f452b (refs/remotes/svn/trunk)
        A       calendar.gif
        A       date_chooser.js
        M       dd-formmailer.css
        M       dd-formmailer.php
        A       lang/Belarussian.php
        A       lang/Czech.php
        A       lang/Estonian.php
        A       lang/Japanese.php
        A       lang/Vietnamese.php
        M       recaptchalib.php
r6 = 9f95df7ce49c5d1cb4715017d223a8bd1c8dcffc (refs/remotes/svn/vendor/current)
Found possible branch point: svn+ssh://svn@vcs/dd-formmailer/vendor/current => svn+ssh://svn@vcs/dd-formmailer/vendor/5.8, 3
Found branch parent: (refs/remotes/svn/tags/vendor/5.8) 8fd8668dbed2dde6b55306c71b3b629f5ed794ec
Following parent with do_switch
        A       calendar.gif
        A       date_chooser.js
        M       dd-formmailer.css
        M       dd-formmailer.php
        A       lang/Belarussian.php
        A       lang/Czech.php
        A       lang/Estonian.php
        A       lang/Japanese.php
        A       lang/Vietnamese.php
        M       recaptchalib.php
Successfully followed parent
r7 = cc00cb187386298cf974dda69f151d2ad4795917 (refs/remotes/svn/tags/vendor/5.8)
        A       calendar.gif
        A       date_chooser.js
        M       dd-formmailer.css
        M       dd-formmailer.php
        M       dd-verify.php
        A       lang/Belarussian.php
        A       lang/Czech.php
        A       lang/Estonian.php
        A       lang/Japanese.php
        A       lang/Vietnamese.php
        M       recaptchalib.php
Checking svn:mergeinfo changes since r5: 1 sources, 1 changed
W: Cannot find common ancestor between e65ba2bdcd12a1935c1a327507dad6b7117f452b and 9f95df7ce49c5d1cb4715017d223a8bd1c8dcffc. Ignoring merge info.
r8 = 9a5cd8f55f377f469280171cd87219b8f528c693 (refs/remotes/svn/trunk)
        M       dd-formmailer.php
r9 = 8143782376d9fbc91a9181b367b72701205b017f (refs/remotes/svn/trunk)
        M       dd-formmailer.php
r10 = 6f563fc4c511ddcc53c2ffc5d26c1e116725bc6b (refs/remotes/svn/trunk)
        M       _svn_client_config.txt
r11 = e9130854178d2c2743a981f303cdd7f34e54b052 (refs/remotes/svn/trunk)
svnserve: E210002: Network connection closed unexpectedly
Checked out HEAD:
  svn+ssh://svn@vcs/dd-formmailer/trunk r11

$ git branch -vv --list ; git for-each-ref
* main e913085 Updated client config for Windows Scripts
e9130854178d2c2743a981f303cdd7f34e54b052 commit refs/heads/main
7ab4f436cafc8af3ed2e727a6d8cbef1a8f8b39f commit refs/remotes/svn/tags/vendor/5.4
cc00cb187386298cf974dda69f151d2ad4795917 commit refs/remotes/svn/tags/vendor/5.8
e9130854178d2c2743a981f303cdd7f34e54b052 commit refs/remotes/svn/trunk
9f95df7ce49c5d1cb4715017d223a8bd1c8dcffc commit refs/remotes/svn/vendor/current

Attachment: 20201225-dd-formmailer-gitk--all.png
Description: PNG image

Attachment: dd-formmailer.svndump.7z
Description: application/7z-compressed

Attachment: dd-formmailer-git-expected.git.7z
Description: application/7z-compressed


[Index of Archives]     [Linux Kernel Development]     [Gcc Help]     [IETF Annouce]     [DCCP]     [Netdev]     [Networking]     [Security]     [V4L]     [Bugtraq]     [Yosemite]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Linux SCSI]     [Fedora Users]

  Powered by Linux