Created attachment 2543 [details] patch to fix reconfigure issue in _ctypes There is a regression when building python3-ctypes. It worked fine in daisy but in fido and master it is broken. The symptom is that the library for _ctypes in missing in the rpm and on target. When python3-ctypes is built with the recipes in fido do_configure fails with the printout in do_configure. The build goes on but the _ctypes module is not built. DEBUG: Executing shell function do_configure autoreconf: 'configure.ac' or 'configure.in' is required NOTE: _ctypes failed to autoreconf The cause of this error is an error with a path in the recipe for python3. With the attached patch that is backported from master the configure step works. The next stop for the build is in the compile step. When trying to compile _ctypes the file setup.py does not take care of the configure parameter --with-system-ffi. This is because in setup.py the CONFIG_ARGS is taken from sysconfig lib for the host machine. In some way when building for a x86 target on a x86_64 host the compilation works and the _ctypes library is present in the rpm. But when building for arm on a x86_64 the library for _ctype is not in the rpm. To reproduce this issue in fido: > Setup a build for beaglebone > Apply the attached patch to poky > bitbake python3-ctypes > look in the do_compile log for failure building ctypes extension I am currently using this with Freescla community BSP and my main problem is ctypes. But with this example building for Beaglebone no extension at all seems to be built.
More information collected about this and it is regarding the PYTHONPATH when building. When buildng in Daisy (working) PYTHONPATH is set to: 'PYTHONPATH': '/work/yocto/qep/qep-media-cgt_build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/Python-3.3.3/build/lib.linux-arm-3.3:./Lib:./Lib/plat-linux' When building in fido (not working) PYTHONPATH is empty: 'PYTHONPATH': '' Importing sysconfigdata in fido case will take the host one and in daisy case will take the target one. How should this work?
From tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r1.0/build/Makefile: PYTHON_FOR_BUILD=_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(CROSSPYTHONPATH) python3.3 It seems that CROSSPYTHONPATH shall be passed in as PYTHONPATH but in my case this one is empty when executing in the Makefile.
FWIW, this works in master for me.
(In reply to comment #3) > FWIW, this works in master for me. Which MACHINE are you building against? With 'works', do you mean that you get your _ctype.so file in your RPM?
The recipe python3_3.3.3.bb applies the patch python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch: - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(CROSSPYTHONPATH) '$interp This results in the generated python Makefile being dependent on CROSSPYTHONPATH being defined in order to work properly during crosscompilation. Adding the following CROSSPYTHONPATH export to a .bbappend file makes ctypes build successfully without any warnings. export CROSSPYTHONPATH = "${B}/build/lib.linux-arm-${PYTHON_MAJMIN}:${S}/Lib:${S}/Lib/plat-linux" The CROSSPYTHONPATH is the resolved version of the original "$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR)". Now setup.py loads the correct sysconfig target specific dicitionary _sysconfigdata.py found in "${B}/build/lib.linux-arm-${PYTHON_MAJMIN}". When CROSSPYTHONPATH is missing it uses the native (x86_64) python sysconfig instead and on my platform fails to build ctypes for arm.
I can confirm ctypes fails to compile on arm, hence ctypes*.so is not on the final rpm or target, I will send a patch soon to fix this for all archs on fido and master
Fixed here: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=fbed86a87064783be376179b3627f55612cd6188
Verified on master: 8ef99a00dc75c6eed87aa1bc1528614a2a27eddf