Bug 11510

Summary: Improve the way we handle python's packaging (autopackaging)
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Alejandro Hernandez <alejandro.hernandez>
Component: devtools / tool chainAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: jose.perez.carranza, meta.mr.watcher, meta.watcher
Version: 2.4   
Target Milestone: 2.5 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: LET
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on:    
Bug Blocks: 11694, 11695    

Description Alejandro Hernandez 2017-05-12 18:06:08 UTC 
Improve the way we handle python packaging:

Manifest file is created by a script, on which we specify the "parts" or files from upstream python that should be installed for a certain python package.

Problems:
   - Unorganized (Hard to read)
   - Lots of new bugs
   - Impossible to know if the required files for a certain package have changed (more or less dependencies), or for new packages unless we install and test them all one by one, and even then we wouldn't know if they require less dependencies.

Goals:
   - New format for manifest file (JSON)
   - Make it work on top of “testimage” so its created correctly and automagically
Comment 1 Alejandro Hernandez 2017-06-20 23:16:45 UTC 
This will broken up into three bugs:

This one:
Will create a new manifest file (JSON), on which all packages specify what modules and files they require, this is basically just a replacement of the manifest we use at the moment with a more organized one, since it will include a script to "convert" the new manifest to the old one so bitbake can still parse it the same way.

Second bug/enchancement:
This one will handle the part on which packages are parsed, this should be done via python, by reading the JSON manifest file directly, and modifying bitbake's dictionary on the go.

Up to this point, the manifest replacement will be complete, but other than it being more organized and having got rid of the script to create it (confusing), it will still have the same other problems stated above.

Third bug/enhancement:
This one will create a test which may be run on top of testimage, that will check every python package to see specifically which modules it needs, and will create a manifest file (JSON) automatically which can be used to replace the current file, this can be run either on every release or when adding a new package if one is lazy enough to not check the modules the new package needs to work.
Comment 2 Alejandro Hernandez 2017-08-28 21:45:34 UTC 
Done, but moving to 2.5 per Richard's request.
Comment 3 Ross Burton 2018-01-22 14:16:50 UTC 
Fixed in oe-core 3328211afdef8ffb00dd4dff1143959d5412b075.