Ping, i'd like to get this merged before we get close to the freeze so we have debugging time if something unexpected happens... On Thu, Apr 12, 2018 at 02:28:17PM +0100, Daniel P. Berrangé wrote: > This series aims to improve the way we handle translations. > > The key problems with our current approach > > - The pot & po files stored in GIT contain huge set of > annotations about source file names & line numbers. > These are out of date as soon as a change is commited > to git following a translation refresh. This makes > diffs impossible to meaningfully review, as they are > 98% noise, 2% signal. > > - The po file messages are sorted by source location, > so when we move code between files, or rename files, > the po file message order changes for no good reason. > This makes diffs even more impossible to review. > > - The po files contain entries for all messages even > if most have no translation, bloating size of po/ > data stored in git > > - Whenever 'make dist' is run, it alters all the pot > and po files, so developers need to then reset their > content to match git HEAD manually. This is caused > by having auto-generated content (source file locations) > mixed in with the static content (the actual translated > strings) > > Overall the *.po files we store consume 100MB, and when > I refresh content from Zanata, it results in git commits > that are many 10's of MB in size and impossible todo any > meaningful review on. > > After this series, we only minimized po files in git, with > the redundated & outdated source locations info stripped. > This stripped info is re-added automatically during build > to create the real .po files, that we distribute, and/or > upload to translators in Zanata. > > As a result the po directory is only 19MB in size, and > when refreshing from Zanata, we have git commits that > clearly show *only* the altered translations, nothing > else. The importance of this cannot be overstated - by > having these clear diffs I discovered a serious bug in > the Zanata client that has been screwing up translations > in every project that uses Zanata by adding bogus "fuzzy" > annotations. > > Note that what I've posted here is a cut-down version of > what will actually be pushed. I have cut out the 99 other > patches that actally change the .po files. There's no > meaningful way to review those commits because of the > noise in the original .po files we're replacing. I have > gone through manual steps, however, to validate that the > .po files we generate from the new .mini.po files are > identical to the original .po files we used to store, > aside from the bogus fuzzy annotations due to broken > Zanata. > > The diffstat shown below covers the full set of patches > on the branch at: > > https://github.com/berrange/libvirt/commits/i18n-4 > > The killer statistic is: > > 418787 insertions(+), 4388281 deletions(-) > > Daniel P. Berrangé (5): > po: provide custom make rules for po file management > po: remove language list from zanata configuration > po: add rules for integration with zanata > po: stop storing libvirt.pot in git > po: minimize & canonicalize translations stored in git > > .gitignore | 9 +- > ABOUT-NLS | 1 + > autogen.sh | 3 +- > bootstrap.conf | 38 - > build-aux/minimize-po.pl | 37 + > configure.ac | 33 +- > m4/virt-nls.m4 | 70 + > po/Makefile.am | 107 + > po/{POTFILES.in => POTFILES} | 0 > po/README.md | 75 + > po/af.mini.po | 20 + > po/af.po | 46395 ------------------------------ > po/am.mini.po | 20 + > po/am.po | 46395 ------------------------------ > po/anp.mini.po | 20 + > po/anp.po | 46395 ------------------------------ > po/ar.mini.po | 762 + > po/ar.po | 46685 ------------------------------ > po/{as.po => as.mini.po} | 54246 ++++++++++------------------------- > po/ast.mini.po | 20 + > po/ast.po | 46395 ------------------------------ > po/bal.mini.po | 20 + > po/bal.po | 46395 ------------------------------ > po/be.mini.po | 21 + > po/be.po | 46396 ------------------------------ > po/bg.mini.po | 1204 + > po/bg.po | 46733 ------------------------------ > po/bn.mini.po | 761 + > po/bn.po | 46684 ------------------------------ > po/bn_IN.mini.po | 9131 ++++++ > po/bn_IN.po | 46729 ------------------------------ > po/bo.mini.po | 20 + > po/bo.po | 46395 ------------------------------ > po/br.mini.po | 20 + > po/br.po | 46395 ------------------------------ > po/brx.mini.po | 20 + > po/brx.po | 46395 ------------------------------ > po/bs.mini.po | 767 + > po/bs.po | 46693 ------------------------------ > po/ca.mini.po | 1618 ++ > po/ca.po | 46555 ------------------------------ > po/cs.mini.po | 4146 +++ > po/cs.po | 46751 ------------------------------ > po/cy.mini.po | 762 + > po/cy.po | 46701 ------------------------------ > po/da.mini.po | 982 + > po/da.po | 46704 ------------------------------ > po/de.mini.po | 22654 +++++++++++++++ > po/de.po | 47656 ------------------------------- > po/de_CH.mini.po | 20 + > po/de_CH.po | 46395 ------------------------------ > po/el.mini.po | 724 + > po/el.po | 46857 ------------------------------ > po/en_GB.mini.po | 22271 +++++++++++++++ > po/en_GB.po | 47573 ------------------------------- > po/eo.mini.po | 20 + > po/eo.po | 46395 ------------------------------ > po/es.mini.po | 22622 +++++++++++++++ > po/es.po | 47748 ------------------------------- > po/et.mini.po | 761 + > po/et.po | 46684 ------------------------------ > po/eu.mini.po | 761 + > po/eu.po | 46684 ------------------------------ > po/fa.mini.po | 20 + > po/fa.po | 46395 ------------------------------ > po/fi.mini.po | 1281 + > po/fi.po | 46744 ------------------------------ > po/fil.mini.po | 20 + > po/fr.mini.po | 4258 +++ > po/fr.po | 46907 ------------------------------ > po/fur.mini.po | 20 + > po/ga.mini.po | 21 + > po/gl.mini.po | 761 + > po/gl.po | 46700 ------------------------------ > po/{gu.po => gu.mini.po} | 54191 ++++++++++------------------------- > po/he.mini.po | 761 + > po/he.po | 46684 ------------------------------ > po/hi.mini.po | 13772 +++++++++ > po/hi.po | 47013 ------------------------------- > po/hr.mini.po | 21 + > po/hr.po | 46396 ------------------------------ > po/hu.mini.po | 987 + > po/hu.po | 46709 ------------------------------ > po/ia.mini.po | 20 + > po/ia.po | 46395 ------------------------------ > po/id.mini.po | 728 + > po/id.po | 46691 ------------------------------ > po/ilo.mini.po | 20 + > po/ilo.po | 46395 ------------------------------ > po/is.mini.po | 761 + > po/is.po | 46684 ------------------------------ > po/it.mini.po | 6588 +++++ > po/it.po | 46663 ------------------------------ > po/{ja.po => ja.mini.po} | 54737 ++++++++++------------------------- > po/ka.mini.po | 761 + > po/ka.po | 46684 ------------------------------ > po/kk.mini.po | 20 + > po/kk.po | 46395 ------------------------------ > po/km.mini.po | 20 + > po/km.po | 46395 ------------------------------ > po/{kn.po => kn.mini.po} | 54156 ++++++++++------------------------- > po/ko.mini.po | 12838 +++++++++ > po/ko.po | 46723 ------------------------------ > po/kw.mini.po | 21 + > po/kw.po | 46396 ------------------------------ > po/kw@xxxxxxxxxxxxx | 21 + > po/kw@xxxxxxxx | 46396 ------------------------------ > po/kw@xxxxxxxxxxxxx | 21 + > po/kw@xxxxxxxx | 46396 ------------------------------ > po/kw_GB.mini.po | 21 + > po/kw_GB.po | 46396 ------------------------------ > po/ky.mini.po | 20 + > po/ky.po | 46395 ------------------------------ > po/libvirt.pot | 46394 ------------------------------ > po/lt.mini.po | 762 + > po/lt.po | 46685 ------------------------------ > po/lv.mini.po | 762 + > po/lv.po | 46685 ------------------------------ > po/mai.mini.po | 20 + > po/mai.po | 46395 ------------------------------ > po/mk.mini.po | 1138 + > po/mk.po | 46724 ------------------------------ > po/{ml.po => ml.mini.po} | 53963 ++++++++++------------------------- > po/mn.mini.po | 20 + > po/mn.po | 46395 ------------------------------ > po/{mr.po => mr.mini.po} | 55484 +++++++++++------------------------- > po/ms.mini.po | 189 + > po/ms.po | 46580 ------------------------------ > po/my.mini.po | 20 + > po/nb.mini.po | 492 + > po/nb.po | 46667 ------------------------------ > po/nds.mini.po | 20 + > po/nds.po | 46395 ------------------------------ > po/ne.mini.po | 20 + > po/ne.po | 46395 ------------------------------ > po/nl.mini.po | 9868 +++++++ > po/nl.po | 47062 ------------------------------- > po/nn.mini.po | 761 + > po/nn.po | 46684 ------------------------------ > po/nso.mini.po | 761 + > po/nso.po | 46684 ------------------------------ > po/{or.po => or.mini.po} | 53161 +++++++++------------------------- > po/{pa.po => pa.mini.po} | 53711 ++++++++++------------------------- > po/pl.mini.po | 10274 +++++++ > po/pl.po | 47166 ------------------------------- > po/pt.mini.po | 1243 + > po/pt.po | 46587 ------------------------------ > po/pt_BR.mini.po | 22821 +++++++++++++++ > po/pt_BR.po | 47565 ------------------------------- > po/ro.mini.po | 762 + > po/ro.po | 46685 ------------------------------ > po/{ru.po => ru.mini.po} | 54065 ++++++++++------------------------- > po/si.mini.po | 761 + > po/si.po | 46684 ------------------------------ > po/sk.mini.po | 761 + > po/sk.po | 46684 ------------------------------ > po/sl.mini.po | 762 + > po/sl.po | 46685 ------------------------------ > po/sq.mini.po | 24 + > po/sq.po | 46399 ------------------------------ > po/sr.mini.po | 2203 ++ > po/sr.po | 46910 ------------------------------ > po/sr@xxxxxxxxxxxxx | 2204 ++ > po/sr@xxxxxxxx | 46911 ------------------------------ > po/sv.mini.po | 2559 ++ > po/sv.po | 46696 ------------------------------ > po/{ta.po => ta.mini.po} | 54968 +++++++++++------------------------- > po/{te.po => te.mini.po} | 53427 ++++++++++------------------------- > po/tg.mini.po | 20 + > po/tg.po | 46395 ------------------------------ > po/th.mini.po | 761 + > po/th.po | 46684 ------------------------------ > po/tr.mini.po | 761 + > po/tr.po | 46684 ------------------------------ > po/tw.mini.po | 20 + > po/tw.po | 46395 ------------------------------ > po/{uk.po => uk.mini.po} | 62575 +++++++++++++++++------------------------ > po/ur.mini.po | 761 + > po/ur.po | 46684 ------------------------------ > po/vi.mini.po | 8674 ++++++ > po/vi.po | 47171 ------------------------------- > po/wba.mini.po | 20 + > po/wba.po | 46395 ------------------------------ > po/yo.mini.po | 20 + > po/yo.po | 46395 ------------------------------ > po/zanata.xml | 99 - > po/zh_CN.mini.po | 23613 ++++++++++++++++ > po/zh_CN.po | 46701 ------------------------------ > po/zh_HK.mini.po | 20 + > po/zh_HK.po | 46395 ------------------------------ > po/zh_TW.mini.po | 958 + > po/zh_TW.po | 46725 ------------------------------ > po/zu.mini.po | 761 + > po/zu.po | 46684 ------------------------------ > src/internal.h | 16 +- > src/libvirt-admin.c | 2 + > src/libvirt.c | 2 + > src/util/virgettext.c | 3 +- > tools/virt-host-validate.c | 4 +- > 199 files changed, 418787 insertions(+), 4388281 deletions(-) > > -- > 2.14.3 > Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| -- libvir-list mailing list libvir-list@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/libvir-list
