| Summary: | python3 bytecode files not installed | ||
|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Will Newton <will.newton> |
| Component: | devtools / tool chain | Assignee: | Joshua Lock <joshuagloe> |
| Status: | VERIFIED FIXED | QA Contact: | Juan Ramos <juan.fernandox.ramos.frayle> |
| Severity: | normal | ||
| Priority: | Medium+ | CC: | jose.perez.carranza, joshuagloe, meta.mr.watcher, meta.watcher, raj.khem, ross.burton |
| Version: | unspecified | ||
| Target Milestone: | 2.3 M1 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
I'm unfamiliar with our Python build and it's not entirely clear to me why 020-dont-compile-python-files.patch was added so I'm adding Khem, the original author of the patch, to CC. If I disable that patch I do get several pyc files written to __pycache__ directories during the build. Is it expected that we'll generate the bytecode during a later step during the build process (and therefore the bug is that we don't)? --- The second part of the issue is that we don't include bytecode files from __pycache__ in packages. __pycache__ was introduced in 3.2: https://docs.python.org/3/whatsnew/3.2.html#pep-3147-pyc-repository-directories I've started working on patches to include __pycache__ files in python packages, starting with the generate-manifest script. this patch was done for avoing pycache since they were not working well back in those early days on python3. If its reliable now, then it may be the time to revise it. (In reply to comment #3) > this patch was done for avoing pycache since they were not working well back > in those early days on python3. If its reliable now, then it may be the time > to revise it. Thanks Khem. Sounds like we're right to target changing this in early 2.3 so that we can better assess the current state with a full development cycle ahead of us. Dominic Sacré submitted a patch for this in October: http://lists.openembedded.org/pipermail/openembedded-core/2016-October/127989.html this patch looks fine to me. Still in review and currently queued in ross/mut This change has merged: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=3f2ab3f7cc1bf1646f2acfcace9b9cf546cc3a5d (In reply to comment #8) > This change has merged: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/ > ?id=3f2ab3f7cc1bf1646f2acfcace9b9cf546cc3a5d I tested this with core-image-sato-sdk-qemux86 image downloaded from: http://autobuilder.yoctoproject.org/pub/releases/yocto-2.3_M1.rc1/machines/qemu/qemux86/ Verified |
Bytecode (*.pyc) files for python3 standard library packages are not installed which leads to slow startup times for many Python scripts. There seems to be at least two problems. Firstly, the bytecode files don't get built for Python 3 even though they did for Python 2. The patch 020-dont-compile-python-files.patch is potentially the cause of this. Secondly, the manifest files use a simple glob to install some files, e.g.: FILES_${PN}-argparse="${libdir}/python3.5/argparse.* " This worked for Python 2 but will not work as well for Python 3 which puts .pyc files in a __pycache__ subdirectory rather than alongside source files.