I just installed Git for Windows 2.43.0 and noticed the installer offers three options for altering the PATH: 1) Run git from git bash only 2) Run git from git bash, cmd.exe and PowerShell (RECOMMENDED) 3) Run git from git bash, cmd.exe and PowerShell with optional utilities (warning: will override find, sort and other system utilities). It turns out iconv.exe is accessible from cmd.exe (Command Prompt) only when you take the third option. But iconv.exe is NOT optional. It is required for git to deal with UTF-16LE with BOM text conversions (and probably for numerous other encoding conversions). But when PATH option #2 is chosen, and iconv.exe is unreachable from a Windows Command Prompt, the git commands which call upon iconv.exe do NOT indicate the error. The call to iconv.exe fails silently. It is only later after you commit, push and clone the repo again that you see the encoding failures. And the warning about overriding find and sort must be taken with a grain of salt, since the Windows versions of those programs are accessed via a Windows folder which appears earlier in the PATH. So this Git for Windows installer screen is misleading. And perhaps iconv.exe should be relocated so it is accessible even when PATH option #2 is chosen. I intend to submit an issue on the Git for Windows issue tracker regarding this. I'll also submit an issue about the lack of an error when running 'git add' for a UTF-16LE with BOM file under PATH option #2. Thanks, - Michael -----Original Message----- From: brian m. carlson <sandals@xxxxxxxxxxxxxxxxxxxx> Sent: Friday, January 12, 2024 1:50 PM To: Michael Litwak <michael.litwak@xxxxxxxx> Cc: git@xxxxxxxxxxxxxxx Subject: [EXTERNAL]Re: Suggested clarification for .gitattributes reference documentation On 2024-01-12 at 21:25:19, Michael Litwak wrote: > Please note, Git for Windows does not support UTF-16LE encoding when running git > commands from an ordinary Command Prompt. Use a git bash console instead. This sounds like a Git for Windows bug. Rather than documenting it, could you open an issue for it on their project? > NEW TEXT: > > You can get a list of all available encodings on your platform with the following command: > > iconv --list > > For Git for Windows users the command, above, is only supported when running in a 'git bash' console. That sounds like a PATH misconfiguration on your part. Have you checked your PATH settings to make sure that the path including the binary is included? > CONCLUSION: > > Text files encoded with UTF-16LE with BOM are common in the Windows > world, as some versions of Visual Studio will use this as the default > encoding for .rc or .mc files. Solution files, project files and > other Visual Studio files can also be in this format. Other encodings > are common, too, e.g. some older versions of PowerShell defaulted to > UTF-16BE with BOM for new .ps1 files. Yet users continue to experience > encoding errors even when they are using the proper > working-tree-encoding in their .gitattributes file. Part of this is > due to the complexity of Git and the number of different platforms it > supports. I should point out that UTF-8 is pretty much the standard these days in many domains, even on Windows. For example, nobody is going to be pleased if you write a web page in any variant of UTF-16, and some languages, such as Rust, are simply defined to be in UTF-8 and won't work if you put them in any other encoding. Almost all editors these days do support UTF-8 (without BOM), even on Windows, so we do want to strongly encourage that rather than having people use UTF-16. The Git FAQ specifically outlines UTF-8 as the recommended way, which is most portable and most functional. We have also documented the UTF-16LE-BOM case specifically in the Git FAQ (git help gitfaq) under "I'm on Windows and my text files are detected as binary". Answering questions on Stack Overflow, I realize that nobody actually reads the FAQ, but we did clearly document how to do it. That being said, I'm not opposed to an additional mention in the gitattributes(5) page if you want to send an actual patch. -- brian m. carlson (he/him or they/them) Toronto, Ontario, CA
