Bug 9775

Summary:	buildtools results in broken system locale if LANG is not en_US
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Paul Eggleton <bluelightning>
Component:	core	Assignee:	Ross Burton <ross.burton>
Status:	VERIFIED FIXED	QA Contact:
Severity:	major
Priority:	Medium+	CC:	jose.perez.carranza, meta.mr.watcher, meta.watcher, richard.purdie
Version:	2.0
Target Milestone:	2.2 M2
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	Don't know

Description Paul Eggleton 2016-06-15 03:51:53 UTC

I just set up a new CentOS system, and there I have LANG set to en_NZ.utf8. I need buildtools to have all the right dependencies, so I installed the latest snapshot from the autobuilder (20160614-1). However, when I try to run bitbake within the buildtools environment, it tells me "Please use a locale setting which supports utf-8.", and indeed if I run python3 and query sys.getfilesystemencoding() it returns 'ascii'. However if I instead run LANG=en_US.utf8 python3 and try the same thing I get the expected 'utf-8'.

Initially I thought this was something broken in CentOS, but then I tried installing the python 3.4 package from EPEL and running that instead of the python3 from buildtools, and with that I get the correct 'utf-8' value with LANG set to my default "en_NZ.utf8". Do we perhaps need to ship the locale data files for glibc within buildtools as well now?

Comment 1 Ross Burton 2016-07-01 11:27:03 UTC

Can you see what happens if you leave LANG=en_NZ.utf8 but set LOCPATH=/usr/share/locale, so the buildtool glibc should look at the host locales?

Comment 2 Ross Burton 2016-07-04 13:14:25 UTC

Looks like LOCPATH isn't an option.  When LOCPATH isn't set glibc uses the locale archive (/usr/lib/locale/locale-archive) which is a mmappable binary representation of the locale database.  However when LOCPATH is set it won't look for a locale archive but will look for expanded /usr/lib/locale/[locale]/[key] files, which on at least Debian and Fedora don't exist for most locales (Debian only ships those for C.UTF-8).  Instead it is expected that the locale archive is used, and that has a fixed location.

I wonder if it would be possible to patch nativesdk-glibc to use the host's locale path...  the question here is how stable is the binary locale format?

Comment 3 Ross Burton 2016-07-06 14:09:27 UTC

Paul, so poky-contrib:ross/sdki18n has a rather special patch for glibc which will mean nativesdk-glibc will look in /usr/ for the locale archive.

Can you build a fresh buildtools with that patch and see if it solves your problem on the Centos machine?

Comment 4 Paul Eggleton 2016-07-13 05:17:52 UTC

I tested a build with the top commit on ross/sdki18n and that does indeed seem to have fixed the issue - and FWIW I build the buildtools tarball on my Fedora 23 machine and then ran it on CentOS 7, so that may indicate that compatibility isn't going to be too much of an issue either.

Comment 5 Ross Burton 2016-07-14 16:37:54 UTC

Thanks Paul!

Comment 6 Ross Burton 2016-07-20 13:44:29 UTC

Fixed in oe-core 75321b6b0f2c0ac667b9350b387b01a188e195c8.

Comment 7 Jose Perez C 2016-07-29 15:12:13 UTC

Verified that cworks as per Comment 4