Re: Sphinx parallel build error: UnicodeEncodeError: 'latin-1' codec can't encode characters in position 18-20: ordinal not in range(256)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Am 12.05.21 um 09:01 schrieb Michal Suchánek:
On Wed, May 12, 2021 at 08:22:38AM +0200, Mauro Carvalho Chehab wrote:
Hi Michal,

Em Thu, 6 May 2021 19:48:49 +0200
Michal Suchánek <msuchanek@xxxxxxx> escreveu:

[  127s] + :
[  127s] + locale
[  128s] LANG=en_US
[  128s] LC_CTYPE="en_US"
[  128s] LC_NUMERIC="en_US"
[  128s] LC_TIME="en_US"
[  128s] LC_COLLATE="en_US"
[  128s] LC_MONETARY="en_US"
[  128s] LC_MESSAGES="en_US"
[  128s] LC_PAPER="en_US"
[  128s] LC_NAME="en_US"
[  128s] LC_ADDRESS="en_US"
[  128s] LC_TELEPHONE="en_US"
[  128s] LC_MEASUREMENT="en_US"
[  128s] LC_IDENTIFICATION="en_US"
[  128s] LC_ALL=
[  128s] + echo LC_ALL=
[  128s] LC_ALL=
[  128s] + echo LANG=en_US
[  128s] LANG=en_US

Where those the locale settings that you used when the build
failed?

I tried to reproduce the bug here with, disabling the parallel run (as
it masks the real error) with both:

	$ for i in LANG LC_ALL LC_ADDRESS LC_IDENTIFICATION LC_MEASUREMENT LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME; do echo $i=en_US; done
	$ make cleandocs && make SPHINXOPTS=-j1 htmldocs

(this one caused lots of warnings on Debian, due to the
  settings at /etc/locale.gen)

and:

	$ for i in LANG LC_ALL LC_ADDRESS LC_IDENTIFICATION LC_MEASUREMENT LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE LC_TIME; do echo $i=en_US.ISO-8859-1; done
	$ make cleandocs && make SPHINXOPTS=-j1 htmldocs

Without any success.

Could you please provide more details about the build VM and the git
changeset that caused the issue?

It depends on what character set your en_US locale implements.

~> cat test.py
print("↑ᛏ个")
~> locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=
~> python3 test.py
↑ᛏ个
~> LANG=en_US python3 test.py
Traceback (most recent call last):
   File "test.py", line 1, in <module>
     print("\u2191\u16cf\u4e2a\uf8f9")
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 0-3: ordinal not in range(256)
~> LANG=C python3 test.py
↑ᛏ个

You can easily test if your python version can print UTF-8 in a specific
locale, and if necessary define an ISO-8859-1 locale for testing.
On some systems the situation is reversed - C locale is ASCII only, and
en_US is UTF-8, and it is possible that some systems don't ship an 8bit
locale at all.

Thats my problem :-) On my system (terminal) I can't reproduce the
issue since stdout always support utf-8, no matter what LANG
environment is set.

$ LANG=en_US.ISO-8859-1 python3
...
>>> import sys
>>> print (sys.stdout.encoding)
utf-8

>>> import locale
>>> locale.getdefaultlocale()
('en_US', 'UTF-8')

I'm not familar with POSIX's locale [1] in detail and in particular
on my system (gnome terminal), I can't say how I can change to 8 bit
coding to reproduce the issue.

[1] https://docs.python.org/3/library/locale.html

-- Markus --




[Index of Archives]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite Forum]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Video 4 Linux]     [Device Mapper]     [Linux Resources]

  Powered by Linux