On Thu, 17 Jan 2008, Johannes Schindelin wrote: > On Thu, 17 Jan 2008, Wincent Colaiuta wrote: > > > El 17/1/2008, a las 1:40, Pedro Melo escribió: > > > > > That's the point I'm making. The fact that I need to set LANG across > > > all users of a project is insane... > > FWIW if you use another filesystem, such as reiserfs or ext[2-4], the > filenames will be _unaffected_ by your particular setting of LANG. They > will be stored byte-wise exactly like asked for. That's why I call them > "sane". One of the advantages (the biggest one, in fact, apart from the obvious US-ASCII down-compatibility and the fact that you can do C-compatible NUL-terminated strings) of UTF-8 is that it's locale-independent, and doesn't care about LANG, because it's valid in all languages. And that's really important. It's important for a very simple reason: there is almost never such a thing as "a locale" except for US-ASCII. Once you move away from US-ASCII, it actually tends to be much more common that you have a *mixture* of locales - often in the same "document" - than to have one single locale. It very much happens even in filenames - people "mix" locales in trivial ways even within a single pathname component (non-US-ASCII filename, but with a regular file extension), but much more interestingly they do so within a directory tree (ie you have have translation subdirectories where the filenames themselves are in another language, and you can have full pathnames where different components are in different languages, for example). And UTF-8 is _wonderful_ for this, because LANG doesn't matter, and cannot matter, and thus mixing isn't a problem. Of course, you can screw it up. Locales still can change things like sort order and capitalization etc, so even if you use UTF-8, you sure can get into trouble with LANG and thinking that a per-session locale makes sense. So choosing UTF-8 for the filesystem isn't wrong per se. It's a fine choice, and has no issues with LANG in itself. Limiting it to strictly valid UTF-8 encodings is also fine. Limiting it (further) to only character normalized UTF-8 is also fine. Most Linux filesystems don't limit it in any way, so you can make filenames that aren't valid UTF-8 at all, much less normalizing multi-character sequences. I personally think that's the best option, but I probably do so mostly because I know some people still use Latin1 as their only locale (and I suspect Asia will take decades before it has converted to UTF-8 and will also have cases where they use other non-UTF locales). But enforcing clean UTF-8 is not a bad idea per se. Not allowing byte sequences that aren't a valid UTF-8 encoding (eg \xc0\xc0 is not a valid UTF-8 character) is fine. I wouldn't call people crazy for doing that, although it does mean that you cannot, for example, decide to write a Latin1 filename (which is not necessarily a *good* idea in this day and age, but I think there's a difference between "that's not a good idea" and "you cannot do that"). And even limiting the UTF-8 charset further to only the minimal representation of one particular glyph (ie not allowing multi-character sequences that can be represented more simply) may be even *more* big-brother, but would at least not cause the technical aliasing issues. I personally think that's so controlling as to be stupid (and has no real advantage), but hey, at least it doesn't *corrupt* anything silently. So I think that using UTF-8 as a character encoding is a *good* thing to do, and that automatically means that LANG shouldn't matter for filenames, but within that choice of UTF-8 there are still mistakes that you can make. Notably multi-character normalization and case-insensitivity. Linus - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
