Bug 8207

Summary: python backports modules stomp on each others' site-packages/backports/__init__.py
Product: [Build System, Metadata & Runtime] Other YP Layers Reporter: Richard Tollerton <rich.tollerton>
Component: layersAssignee: Alejandro Hernandez <alejandro.hernandez>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: alejandro.delcastillo, bmouring, haris.okanovic, ken.sharp, poky.bs.watcher, poky.watcher, rich.tollerton
Version: unspecified   
Target Milestone: 2.0   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Richard Tollerton 2015-08-25 15:37:12 UTC
When building both python-backports-ssl (meta-python) and python-backports-lzma (meta-virtualization), I commonly get errors of this nature:

---
ERROR: The recipe python-backports-ssl is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
   /ssd/rtollert/nilrt-tmp-glibc/sysroots/x64/usr/lib/python2.7/site-packages/backports/__init__.py
 Matched in manifest-x64-python-backports-lzma.populate_sysroot
 /ssd/rtollert/nilrt-tmp-glibc/sysroots/x64/usr/lib/python2.7/site-packages/backports/__init__.pyc
 Matched in manifest-x64-python-backports-lzma.populate_sysroot
Please verify which recipe should provide the above files.
---

Amazingly, the EXACT SAME PROBLEM occurs when installing archlinux AUR builds of more than one backports package:

---
$ yaourt -S python2-backports.ssl_match_hostname
Packages (1) python2-backports.ssl_match_hostname-3.4.0.2-1
...
error: failed to commit transaction (conflicting files)
python2-backports.ssl_match_hostname: /usr/lib/python2.7/site-packages/backports/__init__.py exists in filesystem
python2-backports.ssl_match_hostname: /usr/lib/python2.7/site-packages/backports/__init__.pyc exists in filesystem
python2-backports.ssl_match_hostname: /usr/lib/python2.7/site-packages/backports/__init__.pyo exists in filesystem
Errors occurred, no packages were upgraded.

$ pacman -Qo /usr/lib/python2.7/site-packages/backports/__init__.py
/usr/lib/python2.7/site-packages/backports/__init__.py is owned by python2-backports.lzma 0.0.3-1
---

In other words, python packages following the backports spec are explicitly designed to install conflicting files. The backports specification explicitly mandates this: https://pypi.python.org/pypi/backports/1.0

This is an abominable situation. Either we need to add some more build infrastructure so that multiple backports modules can be used at once, or people need to stop using backports because they're broken (i.e. move to python3).

I was imagining the build infrastructure would work like
- Create a new python-backports module that installs site-packages/backports/__init__.py
- Create a new backports.bbclass that RDEPENDS+=python-backports, strips out site-packages/backports/__init__.py from FILES, etc.