[PATCH 0/5] Improve translation handling

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



This series aims to improve the way we handle translations.

The key problems with our current approach

 - The pot & po files stored in GIT contain huge set of
   annotations about source file names & line numbers.
   These are out of date as soon as a change is commited
   to git following a translation refresh. This makes
   diffs impossible to meaningfully review, as they are
   98% noise, 2% signal.

 - The po file messages are sorted by source location,
   so when we move code between files, or rename files,
   the po file message order changes for no good reason.
   This makes diffs even more impossible to review.

 - The po files contain entries for all messages even
   if most have no translation, bloating size of po/
   data stored in git

 - Whenever 'make dist' is run, it alters all the pot
   and po files, so developers need to then reset their
   content to match git HEAD manually. This is caused
   by having auto-generated content (source file locations)
   mixed in with the static content (the actual translated
   strings)

Overall the *.po  files we store consume 100MB, and when
I refresh content from Zanata, it results in git commits
that are many 10's of MB in size and impossible todo any
meaningful review on.

After this series, we only minimized po files in git, with
the redundated & outdated source locations info stripped.
This stripped info is re-added automatically during build
to create the real .po files, that we distribute, and/or
upload to translators in Zanata.

As a result the po directory is only 19MB in size, and
when refreshing from Zanata, we have git commits that
clearly show *only* the altered translations, nothing
else. The importance of this cannot be overstated - by
having these clear diffs I discovered a serious bug in
the Zanata client that has been screwing up translations
in every project that uses Zanata by adding bogus "fuzzy"
annotations.

Note that what I've posted here is a cut-down version of
what will actually be pushed. I have cut out the 99 other
patches that actally change the .po files. There's no
meaningful way to review those commits because of the
noise in the original .po files we're replacing. I have
gone through manual steps, however, to validate that the
.po files we generate from the new .mini.po files are
identical to the original .po files we used to store,
aside from the bogus fuzzy annotations due to broken
Zanata.

The diffstat shown below covers the full set of patches
on the branch at:

   https://github.com/berrange/libvirt/commits/i18n-4

The killer statistic is:

   418787 insertions(+), 4388281 deletions(-)

Daniel P. Berrangé (5):
  po: provide custom make rules for po file management
  po: remove language list from zanata configuration
  po: add rules for integration with zanata
  po: stop storing libvirt.pot in git
  po: minimize & canonicalize translations stored in git

 .gitignore                   |     9 +-
 ABOUT-NLS                    |     1 +
 autogen.sh                   |     3 +-
 bootstrap.conf               |    38 -
 build-aux/minimize-po.pl     |    37 +
 configure.ac                 |    33 +-
 m4/virt-nls.m4               |    70 +
 po/Makefile.am               |   107 +
 po/{POTFILES.in => POTFILES} |     0
 po/README.md                 |    75 +
 po/af.mini.po                |    20 +
 po/af.po                     | 46395 ------------------------------
 po/am.mini.po                |    20 +
 po/am.po                     | 46395 ------------------------------
 po/anp.mini.po               |    20 +
 po/anp.po                    | 46395 ------------------------------
 po/ar.mini.po                |   762 +
 po/ar.po                     | 46685 ------------------------------
 po/{as.po => as.mini.po}     | 54246 ++++++++++-------------------------
 po/ast.mini.po               |    20 +
 po/ast.po                    | 46395 ------------------------------
 po/bal.mini.po               |    20 +
 po/bal.po                    | 46395 ------------------------------
 po/be.mini.po                |    21 +
 po/be.po                     | 46396 ------------------------------
 po/bg.mini.po                |  1204 +
 po/bg.po                     | 46733 ------------------------------
 po/bn.mini.po                |   761 +
 po/bn.po                     | 46684 ------------------------------
 po/bn_IN.mini.po             |  9131 ++++++
 po/bn_IN.po                  | 46729 ------------------------------
 po/bo.mini.po                |    20 +
 po/bo.po                     | 46395 ------------------------------
 po/br.mini.po                |    20 +
 po/br.po                     | 46395 ------------------------------
 po/brx.mini.po               |    20 +
 po/brx.po                    | 46395 ------------------------------
 po/bs.mini.po                |   767 +
 po/bs.po                     | 46693 ------------------------------
 po/ca.mini.po                |  1618 ++
 po/ca.po                     | 46555 ------------------------------
 po/cs.mini.po                |  4146 +++
 po/cs.po                     | 46751 ------------------------------
 po/cy.mini.po                |   762 +
 po/cy.po                     | 46701 ------------------------------
 po/da.mini.po                |   982 +
 po/da.po                     | 46704 ------------------------------
 po/de.mini.po                | 22654 +++++++++++++++
 po/de.po                     | 47656 -------------------------------
 po/de_CH.mini.po             |    20 +
 po/de_CH.po                  | 46395 ------------------------------
 po/el.mini.po                |   724 +
 po/el.po                     | 46857 ------------------------------
 po/en_GB.mini.po             | 22271 +++++++++++++++
 po/en_GB.po                  | 47573 -------------------------------
 po/eo.mini.po                |    20 +
 po/eo.po                     | 46395 ------------------------------
 po/es.mini.po                | 22622 +++++++++++++++
 po/es.po                     | 47748 -------------------------------
 po/et.mini.po                |   761 +
 po/et.po                     | 46684 ------------------------------
 po/eu.mini.po                |   761 +
 po/eu.po                     | 46684 ------------------------------
 po/fa.mini.po                |    20 +
 po/fa.po                     | 46395 ------------------------------
 po/fi.mini.po                |  1281 +
 po/fi.po                     | 46744 ------------------------------
 po/fil.mini.po               |    20 +
 po/fr.mini.po                |  4258 +++
 po/fr.po                     | 46907 ------------------------------
 po/fur.mini.po               |    20 +
 po/ga.mini.po                |    21 +
 po/gl.mini.po                |   761 +
 po/gl.po                     | 46700 ------------------------------
 po/{gu.po => gu.mini.po}     | 54191 ++++++++++-------------------------
 po/he.mini.po                |   761 +
 po/he.po                     | 46684 ------------------------------
 po/hi.mini.po                | 13772 +++++++++
 po/hi.po                     | 47013 -------------------------------
 po/hr.mini.po                |    21 +
 po/hr.po                     | 46396 ------------------------------
 po/hu.mini.po                |   987 +
 po/hu.po                     | 46709 ------------------------------
 po/ia.mini.po                |    20 +
 po/ia.po                     | 46395 ------------------------------
 po/id.mini.po                |   728 +
 po/id.po                     | 46691 ------------------------------
 po/ilo.mini.po               |    20 +
 po/ilo.po                    | 46395 ------------------------------
 po/is.mini.po                |   761 +
 po/is.po                     | 46684 ------------------------------
 po/it.mini.po                |  6588 +++++
 po/it.po                     | 46663 ------------------------------
 po/{ja.po => ja.mini.po}     | 54737 ++++++++++-------------------------
 po/ka.mini.po                |   761 +
 po/ka.po                     | 46684 ------------------------------
 po/kk.mini.po                |    20 +
 po/kk.po                     | 46395 ------------------------------
 po/km.mini.po                |    20 +
 po/km.po                     | 46395 ------------------------------
 po/{kn.po => kn.mini.po}     | 54156 ++++++++++-------------------------
 po/ko.mini.po                | 12838 +++++++++
 po/ko.po                     | 46723 ------------------------------
 po/kw.mini.po                |    21 +
 po/kw.po                     | 46396 ------------------------------
 po/kw@xxxxxxxxxxxxx          |    21 +
 po/kw@xxxxxxxx               | 46396 ------------------------------
 po/kw@xxxxxxxxxxxxx          |    21 +
 po/kw@xxxxxxxx               | 46396 ------------------------------
 po/kw_GB.mini.po             |    21 +
 po/kw_GB.po                  | 46396 ------------------------------
 po/ky.mini.po                |    20 +
 po/ky.po                     | 46395 ------------------------------
 po/libvirt.pot               | 46394 ------------------------------
 po/lt.mini.po                |   762 +
 po/lt.po                     | 46685 ------------------------------
 po/lv.mini.po                |   762 +
 po/lv.po                     | 46685 ------------------------------
 po/mai.mini.po               |    20 +
 po/mai.po                    | 46395 ------------------------------
 po/mk.mini.po                |  1138 +
 po/mk.po                     | 46724 ------------------------------
 po/{ml.po => ml.mini.po}     | 53963 ++++++++++-------------------------
 po/mn.mini.po                |    20 +
 po/mn.po                     | 46395 ------------------------------
 po/{mr.po => mr.mini.po}     | 55484 +++++++++++-------------------------
 po/ms.mini.po                |   189 +
 po/ms.po                     | 46580 ------------------------------
 po/my.mini.po                |    20 +
 po/nb.mini.po                |   492 +
 po/nb.po                     | 46667 ------------------------------
 po/nds.mini.po               |    20 +
 po/nds.po                    | 46395 ------------------------------
 po/ne.mini.po                |    20 +
 po/ne.po                     | 46395 ------------------------------
 po/nl.mini.po                |  9868 +++++++
 po/nl.po                     | 47062 -------------------------------
 po/nn.mini.po                |   761 +
 po/nn.po                     | 46684 ------------------------------
 po/nso.mini.po               |   761 +
 po/nso.po                    | 46684 ------------------------------
 po/{or.po => or.mini.po}     | 53161 +++++++++-------------------------
 po/{pa.po => pa.mini.po}     | 53711 ++++++++++-------------------------
 po/pl.mini.po                | 10274 +++++++
 po/pl.po                     | 47166 -------------------------------
 po/pt.mini.po                |  1243 +
 po/pt.po                     | 46587 ------------------------------
 po/pt_BR.mini.po             | 22821 +++++++++++++++
 po/pt_BR.po                  | 47565 -------------------------------
 po/ro.mini.po                |   762 +
 po/ro.po                     | 46685 ------------------------------
 po/{ru.po => ru.mini.po}     | 54065 ++++++++++-------------------------
 po/si.mini.po                |   761 +
 po/si.po                     | 46684 ------------------------------
 po/sk.mini.po                |   761 +
 po/sk.po                     | 46684 ------------------------------
 po/sl.mini.po                |   762 +
 po/sl.po                     | 46685 ------------------------------
 po/sq.mini.po                |    24 +
 po/sq.po                     | 46399 ------------------------------
 po/sr.mini.po                |  2203 ++
 po/sr.po                     | 46910 ------------------------------
 po/sr@xxxxxxxxxxxxx          |  2204 ++
 po/sr@xxxxxxxx               | 46911 ------------------------------
 po/sv.mini.po                |  2559 ++
 po/sv.po                     | 46696 ------------------------------
 po/{ta.po => ta.mini.po}     | 54968 +++++++++++-------------------------
 po/{te.po => te.mini.po}     | 53427 ++++++++++-------------------------
 po/tg.mini.po                |    20 +
 po/tg.po                     | 46395 ------------------------------
 po/th.mini.po                |   761 +
 po/th.po                     | 46684 ------------------------------
 po/tr.mini.po                |   761 +
 po/tr.po                     | 46684 ------------------------------
 po/tw.mini.po                |    20 +
 po/tw.po                     | 46395 ------------------------------
 po/{uk.po => uk.mini.po}     | 62575 +++++++++++++++++------------------------
 po/ur.mini.po                |   761 +
 po/ur.po                     | 46684 ------------------------------
 po/vi.mini.po                |  8674 ++++++
 po/vi.po                     | 47171 -------------------------------
 po/wba.mini.po               |    20 +
 po/wba.po                    | 46395 ------------------------------
 po/yo.mini.po                |    20 +
 po/yo.po                     | 46395 ------------------------------
 po/zanata.xml                |    99 -
 po/zh_CN.mini.po             | 23613 ++++++++++++++++
 po/zh_CN.po                  | 46701 ------------------------------
 po/zh_HK.mini.po             |    20 +
 po/zh_HK.po                  | 46395 ------------------------------
 po/zh_TW.mini.po             |   958 +
 po/zh_TW.po                  | 46725 ------------------------------
 po/zu.mini.po                |   761 +
 po/zu.po                     | 46684 ------------------------------
 src/internal.h               |    16 +-
 src/libvirt-admin.c          |     2 +
 src/libvirt.c                |     2 +
 src/util/virgettext.c        |     3 +-
 tools/virt-host-validate.c   |     4 +-
 199 files changed, 418787 insertions(+), 4388281 deletions(-)

-- 
2.14.3

--
libvir-list mailing list
libvir-list@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/libvir-list




[Index of Archives]     [Virt Tools]     [Libvirt Users]     [Lib OS Info]     [Fedora Users]     [Fedora Desktop]     [Fedora SELinux]     [Big List of Linux Books]     [Yosemite News]     [KDE Users]     [Fedora Tools]

  Powered by Linux