7873 2015-06-10 21:40:17 +0000 python3-ctypes fails to cross compile 2015-06-24 15:25:04 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain 1.8 All arm VERIFIED FIXED Medium+ major 1.9 M2 1 petan679 alejandro.hernandez benjamin.esquivel cristina-danielax.agurida jonas.goransson meta.mr.watcher meta.watcher ross.burton cristina-danielax.agurida --- --- No (bug/feature does not impact docs) oldest_to_newest 51623 0 2543 petan679 2015-06-10 21:40:17 +0000 Created attachment 2543 patch to fix reconfigure issue in _ctypes There is a regression when building python3-ctypes. It worked fine in daisy but in fido and master it is broken. The symptom is that the library for _ctypes in missing in the rpm and on target. When python3-ctypes is built with the recipes in fido do_configure fails with the printout in do_configure. The build goes on but the _ctypes module is not built. DEBUG: Executing shell function do_configure autoreconf: 'configure.ac' or 'configure.in' is required NOTE: _ctypes failed to autoreconf The cause of this error is an error with a path in the recipe for python3. With the attached patch that is backported from master the configure step works. The next stop for the build is in the compile step. When trying to compile _ctypes the file setup.py does not take care of the configure parameter --with-system-ffi. This is because in setup.py the CONFIG_ARGS is taken from sysconfig lib for the host machine. In some way when building for a x86 target on a x86_64 host the compilation works and the _ctypes library is present in the rpm. But when building for arm on a x86_64 the library for _ctype is not in the rpm. To reproduce this issue in fido: > Setup a build for beaglebone > Apply the attached patch to poky > bitbake python3-ctypes > look in the do_compile log for failure building ctypes extension I am currently using this with Freescla community BSP and my main problem is ctypes. But with this example building for Beaglebone no extension at all seems to be built. 51626 1 petan679 2015-06-11 07:54:34 +0000 More information collected about this and it is regarding the PYTHONPATH when building. When buildng in Daisy (working) PYTHONPATH is set to: 'PYTHONPATH': '/work/yocto/qep/qep-media-cgt_build/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r0.0/Python-3.3.3/build/lib.linux-arm-3.3:./Lib:./Lib/plat-linux' When building in fido (not working) PYTHONPATH is empty: 'PYTHONPATH': '' Importing sysconfigdata in fido case will take the host one and in daisy case will take the target one. How should this work? 51630 2 petan679 2015-06-11 08:46:02 +0000 From tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/python3/3.3.3-r1.0/build/Makefile: PYTHON_FOR_BUILD=_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(CROSSPYTHONPATH) python3.3 It seems that CROSSPYTHONPATH shall be passed in as PYTHONPATH but in my case this one is empty when executing in the Makefile. 51643 3 ross.burton 2015-06-11 14:33:37 +0000 FWIW, this works in master for me. 51655 4 petan679 2015-06-11 18:04:08 +0000 (In reply to comment #3) > FWIW, this works in master for me. Which MACHINE are you building against? With 'works', do you mean that you get your _ctype.so file in your RPM? 51706 5 jonas.goransson 2015-06-15 20:14:11 +0000 The recipe python3_3.3.3.bb applies the patch python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch: - PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR) '$interp + PYTHON_FOR_BUILD='_PYTHON_PROJECT_BASE=$(abs_builddir) _PYTHON_HOST_PLATFORM=$(_PYTHON_HOST_PLATFORM) PYTHONPATH=$(CROSSPYTHONPATH) '$interp This results in the generated python Makefile being dependent on CROSSPYTHONPATH being defined in order to work properly during crosscompilation. Adding the following CROSSPYTHONPATH export to a .bbappend file makes ctypes build successfully without any warnings. export CROSSPYTHONPATH = "${B}/build/lib.linux-arm-${PYTHON_MAJMIN}:${S}/Lib:${S}/Lib/plat-linux" The CROSSPYTHONPATH is the resolved version of the original "$(shell test -f pybuilddir.txt && echo $(abs_builddir)/`cat pybuilddir.txt`:)$(srcdir)/Lib:$(srcdir)/Lib/$(PLATDIR)". Now setup.py loads the correct sysconfig target specific dicitionary _sysconfigdata.py found in "${B}/build/lib.linux-arm-${PYTHON_MAJMIN}". When CROSSPYTHONPATH is missing it uses the native (x86_64) python sysconfig instead and on my platform fails to build ctypes for arm. 51742 6 alejandro.hernandez 2015-06-16 21:58:49 +0000 I can confirm ctypes fails to compile on arm, hence ctypes*.so is not on the final rpm or target, I will send a patch soon to fix this for all archs on fido and master 51883 7 alejandro.hernandez 2015-06-23 15:50:35 +0000 Fixed here: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=fbed86a87064783be376179b3627f55612cd6188 51906 8 cristina-danielax.agurida 2015-06-24 15:25:04 +0000 Verified on master: 8ef99a00dc75c6eed87aa1bc1528614a2a27eddf 2543 2015-06-10 21:40:17 +0000 2015-06-10 21:40:17 +0000 patch to fix reconfigure issue in _ctypes python3_ctypes_reconf.patch text/plain 690 petan679 ZGlmZiAtLWdpdCBhL21ldGEvcmVjaXBlcy1kZXZ0b29scy9weXRob24vcHl0aG9uM18zLjMuMy5i YiBiL21ldGEvcmVjaXBlcy1kZXZ0b29scy9weXRob24vcHl0aG9uM18zLjMuMy5iYgppbmRleCA2 MTQ3NDA4Li45MTYyYWQ3IDEwMDY0NAotLS0gYS9tZXRhL3JlY2lwZXMtZGV2dG9vbHMvcHl0aG9u L3B5dGhvbjNfMy4zLjMuYmIKKysrIGIvbWV0YS9yZWNpcGVzLWRldnRvb2xzL3B5dGhvbi9weXRo b24zXzMuMy4zLmJiCkBAIC03NSw5ICs3NSw5IEBAIGV4cG9ydCBfUFlUSE9OX1BST0pFQ1RfU1JD ID0gIiR7U30iCiAjIE5vIGN0eXBlcyBvcHRpb24gZm9yIHB5dGhvbiAzCiBQWVRIT05MU0JPUFRT ID0gIiIKIAotZG9fY29uZmlndXJlX3ByZXBlbmQoKSB7Citkb19jb25maWd1cmVfYXBwZW5kKCkg ewogCXJtIC1mICR7U30vTWFrZWZpbGUub3JpZwotCWF1dG9yZWNvbmYgLVdjcm9zcyAtLXZlcmJv c2UgLS1pbnN0YWxsIC0tZm9yY2UgLS1leGNsdWRlPWF1dG9wb2ludCBNb2R1bGVzL19jdHlwZXMv bGliZmZpIHx8IGJibm90ZSAiX2N0eXBlcyBmYWlsZWQgdG8gYXV0b3JlY29uZiIKKwlhdXRvcmVj b25mIC1XY3Jvc3MgLS12ZXJib3NlIC0taW5zdGFsbCAtLWZvcmNlIC0tZXhjbHVkZT1hdXRvcG9p bnQgLi4vUHl0aG9uLSR7UFZ9L01vZHVsZXMvX2N0eXBlcy9saWJmZmkKIH0KIAogZG9fY29tcGls ZSgpIHsK