On 17.06.2015 18:42, Junio C Hamano wrote:
Jan-Philip Gehrcke <jgehrcke@xxxxxxxxxxxxxx> writes:
I was surprised to see that the output of
git log --encoding=utf-8 "--format=format:%b"
can contain byte sequences that are invalid in UTF-8. Note: I am using
git 2.1.4 and the %b format specifier represents the commit message
body.
Yeah, if the original was bad and cannot be sanely expressed in
UTF-8, you have two options. You can show the contents as raw bytes
recorded in the object with a warning so that the user can use it as
such (e.g. perhaps the original was indeed an iso8859-2 but was
incorrectly marked as UTF-8, or something like that, and a human
that is more intelligent than a tool _could_ guess and attempt to
recover). Or you can error out and refuse to produce output.
The two-option scenario is totally clear. Although one must stress that
the "error-out" option can, as discussed, be kept minimally invasive: it
is sufficient (and common) to just skip those byte sequences (and
replace them with a replacement symbol) that would be invalid in the
requested output encoding. This would retain as much information as
possible while guaranteeing a subsequent decoder to retrieve valid input.
We deliberately made a design choice to take the former option.
I totally support this design choice in general, especially when
invoking `git whatever` without options. This here is, I think, mainly
about documentation and the semantics of "--encoding". From my point of
view, `--encoding=utf-8` semantically suggests that the output *is*
valid UTF-8. But it is not, not always. May initial question was: what
do you think about this? Should we
* just make this more clear in the docs and/or
* should we adjust the behavior of --encoding or
* should we do something entirely different, like adding a new command
line option or
* should we just leave things as they are?
Thanks and cheers,
Jan-Philip
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html