10315 2016-09-21 13:10:25 +0000 python3 bytecode files not installed 2017-01-06 20:30:44 +0000 1 1 7 Build System, Metadata & Runtime OE-Core devtools / tool chain unspecified x86 Multiple VERIFIED FIXED Medium+ normal 2.3 M1 1 will.newton joshuagloe jose.perez.carranza joshuagloe meta.mr.watcher meta.watcher raj.khem ross.burton juan.fernandox.ramos.frayle --- --- No (bug/feature does not impact docs) oldest_to_newest 66416 0 will.newton 2016-09-21 13:10:25 +0000 Bytecode (*.pyc) files for python3 standard library packages are not installed which leads to slow startup times for many Python scripts. There seems to be at least two problems. Firstly, the bytecode files don't get built for Python 3 even though they did for Python 2. The patch 020-dont-compile-python-files.patch is potentially the cause of this. Secondly, the manifest files use a simple glob to install some files, e.g.: FILES_${PN}-argparse="${libdir}/python3.5/argparse.* " This worked for Python 2 but will not work as well for Python 3 which puts .pyc files in a __pycache__ subdirectory rather than alongside source files. 66434 1 bluelightning 2016-09-21 19:25:01 +0000 This may be covered by bug 6434. 67009 2 joshuagloe 2016-10-06 16:39:04 +0000 I'm unfamiliar with our Python build and it's not entirely clear to me why 020-dont-compile-python-files.patch was added so I'm adding Khem, the original author of the patch, to CC. If I disable that patch I do get several pyc files written to __pycache__ directories during the build. Is it expected that we'll generate the bytecode during a later step during the build process (and therefore the bug is that we don't)? --- The second part of the issue is that we don't include bytecode files from __pycache__ in packages. __pycache__ was introduced in 3.2: https://docs.python.org/3/whatsnew/3.2.html#pep-3147-pyc-repository-directories I've started working on patches to include __pycache__ files in python packages, starting with the generate-manifest script. 67011 3 raj.khem 2016-10-06 17:03:30 +0000 this patch was done for avoing pycache since they were not working well back in those early days on python3. If its reliable now, then it may be the time to revise it. 67015 4 joshuagloe 2016-10-06 21:07:59 +0000 (In reply to comment #3) > this patch was done for avoing pycache since they were not working well back > in those early days on python3. If its reliable now, then it may be the time > to revise it. Thanks Khem. Sounds like we're right to target changing this in early 2.3 so that we can better assess the current state with a full development cycle ahead of us. 68234 5 joshuagloe 2016-11-11 10:19:00 +0000 Dominic Sacré submitted a patch for this in October: http://lists.openembedded.org/pipermail/openembedded-core/2016-October/127989.html 68262 6 raj.khem 2016-11-11 18:20:00 +0000 this patch looks fine to me. 68716 7 joshuagloe 2016-11-30 13:33:36 +0000 Still in review and currently queued in ross/mut 68729 8 joshuagloe 2016-11-30 16:21:29 +0000 This change has merged: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=3f2ab3f7cc1bf1646f2acfcace9b9cf546cc3a5d 69593 9 juan.fernandox.ramos.frayle 2017-01-06 20:30:44 +0000 (In reply to comment #8) > This change has merged: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/ > ?id=3f2ab3f7cc1bf1646f2acfcace9b9cf546cc3a5d I tested this with core-image-sato-sdk-qemux86 image downloaded from: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M1.rc1/machines/qemu/qemux86/ Verified