Hi Junio,
after upgrading from GIT 1.4 to 1.5, I noticed that I had got
errors when applying patches from mailboxes and that I had to
manually add "--no-utf8" to git-am. The error was always :
"fatal: cannot convert from latin1 to utf-8".
This morning, I decided to take a closer look at it, first
trying to fix it in the configuration. Even this fails :
[i18n]
commitencoding = latin1
$ git-am -3 username.patch
fatal: cannot convert from latin1 to latin1
So I took a look at the code and noticed that in builtin-mailinfo,
convert_to_utf8() is always called because metainfo_charset is
always set. Also, this function always calls reencode_string(),
no matter what input and outputs are passed. And since it fails
for an unknown reason, I get the error message.
Now I have several options :
- always pass --no-utf8 with git-am to leave metainfo_charset
NULL and avoid the call to convert_to_utf8()
=> very annoying and I have to rm -rf .dotest every time I forget
about it
- add a check to convert_to_utf8() so that when input and output
charsets are equal, it does not malloc/convert/copy/free the
result. This would seem necessary anyway.
- check for empty charset name instead of just a NULL pointer,
so that it is still possible by configuration to stay compatible
with old default behaviour which is : not to convert by default.
- switch back to 1.4 : this will be my immediate option while
waiting for any advice on the right long-term solution.
I would be glad to get a recommendation for one of the solutions
above. I can workout a patch, but I don't want to waste my time
doing crappy patches that will not be retained.
Please note that I don't know a damn thing about the way iconv
works, which may be one of the reason I really hate it. So it
will take far too much time to me to dig below reencode_string().
Also, I am really worried for buffer overflows here. Looking at
the code, it seems like headers from input mails are passed to
decode_header(), which itself calls decode_header_bq() which in
turns calls convert_to_utf8() with input and output buffers of
256 bytes. Same for mailinfo() which calls check_header_line()
with <line> which has been filled up to sizeof(line) (1000 bytes).
Unless I'm mistaken, encoding from latin1 to utf-8 can grow the
string by a 5-to-1 ratio. So this means that the blind strcpy()
in convert_to_utf8() could overflow on user-crafted mail headers.
Maybe I overlooked something :-/
Regards,
Willy
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html