Bug 12833

Summary: boost_python3 cannot be found after update to boost-1.67
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Lukas Bulwahn <lukas.bulwahn>
Component: oe-core otherAssignee: Ross Burton <ross.burton>
Status: RESOLVED NOTABUG QA Contact:
Severity: normal    
Priority: Medium+ CC: mingli.yu, randy.macleod
Version: unspecified   
Target Milestone: 2.6 M4   
Hardware: All   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Lukas Bulwahn 2018-06-30 10:26:27 UTC
After the boost update to 1.67 (http://git.openembedded.org/openembedded-core/commit/meta/recipes-support/boost?id=358cf46ea4d01b7ad8c355fa103d4a6922cc0a88),
boost_python3 is not found during do_configure:

```
| CMake Error at /home/lukas/repositories/openembedded.org/openembedded-core/build/tmp-glibc/work/i586-oe-linux/camera-calibration-parsers/1.11.13-r0/recipe-sysroot-native/usr/share/cmake-3.10/Modules/FindBoost.cmake:2050 (message):
|   Unable to find the requested Boost libraries.
| 
|   Boost version: 1.67.0
| 
|   Boost include path:
|   /home/lukas/repositories/openembedded.org/openembedded-core/build/tmp-glibc/work/i586-oe-linux/camera-calibration-parsers/1.11.13-r0/recipe-sysroot/usr/include
| 
| 
|   Could not find the following Boost libraries:
| 
|           boost_python3
| 
|   Some (but not all) of the required Boost libraries were found.  You may
|   need to install these additional Boost libraries.  Alternatively, set
|   BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
|   to the location of Boost.
```

This issue was discovered in the meta-ros layer, where the recipes camera-calibration-parsers and cv-bridge fail when using python3 support.

The CMakeLists.txt of those packages just contain:

find_package(PythonLibs)
if(PYTHONLIBS_VERSION_STRING VERSION_LESS 3)
    find_package(Boost REQUIRED python)
else()
    find_package(Boost REQUIRED python3)
endif()

When python3 is used, the configure step fails.
Comment 1 Ross Burton 2018-07-05 14:45:23 UTC
This should have been fixed in master with "cmake: fix build issue with boost 1.67.0" (oe-core 04f367e27e66105466611bee4d63abc78e0a4725).

Lukas, can you verify this?
Comment 2 Lukas Bulwahn 2018-07-05 16:13:01 UTC
My clean build is now running to check that; will come back to you on this tomorrow-ish.

Lukas
Comment 3 Lukas Bulwahn 2018-07-05 17:24:23 UTC
My test build just ran through on this build configuration:

Build Configuration:
BB_VERSION           = "1.37.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-16.04"
TARGET_SYS           = "i586-oe-linux"
MACHINE              = "qemux86"
DISTRO               = "nodistro"
DISTRO_VERSION       = "nodistro.0"
TUNE_FEATURES        = "m32 i586"
TARGET_FPU           = ""
meta                 = "master:416a8c241aff0dca6b8b123e52cf8e2d40c74c8d"
meta-oe              
meta-python          
meta-multimedia      = "master:4517d1c695efd0b175f4e7242e1a7bb4361527d9"
meta-ros             = "master:dd424fbc01c29ea16c0dfc011e52e11750c729f2"


It did fail as pointed out already with not finding boost_python3.

Summary: 2 tasks failed:
  /home/lukas/repositories/github.com/bmwcarit/meta-ros/recipes-ros/image-common/camera-calibration-parsers_1.11.13.bb:do_configure
  /home/lukas/repositories/github.com/bmwcarit/meta-ros/recipes-ros/vision-opencv/cv-bridge_1.11.16.bb:do_configure
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

So that commit you mentioned did not resolve the issue yet.
Comment 4 Stephen K Jolley 2018-07-06 16:17:23 UTC
It appears you question has been answered.
Comment 5 Randy MacLeod 2018-09-13 15:13:01 UTC
Grace, Please check if this is a problem and work with Ross's help if needed.
Comment 6 Mingli Yu 2018-09-19 03:19:25 UTC
There is should be python3 exist after update to boost-1.67.

Compared the previous /usr/lib64/libboost_python3.so.1.66.0, there is /usr/lib64/libboost_python35.so.1.68.0 as the logic in below commit:

https://github.com/boostorg/python/commit/d4d41d94aecc9f8098aabd3587fcb95458451f71
Comment 7 Mingli Yu 2018-09-19 03:27:15 UTC
(In reply to comment #6)
> There is should be python3 exist after update to boost-1.67.
> 
> Compared the previous /usr/lib64/libboost_python3.so.1.66.0, there is
> /usr/lib64/libboost_python35.so.1.68.0 as the logic in below commit:

It should be /usr/lib64/libboost_python35.so.1.67.0 in boost-1.67.

> 
> https://github.com/boostorg/python/commit/
> d4d41d94aecc9f8098aabd3587fcb95458451f71
Comment 8 Mingli Yu 2018-09-19 05:26:11 UTC
(In reply to comment #3)
> My test build just ran through on this build configuration:
> 
> Build Configuration:
> BB_VERSION           = "1.37.0"
> BUILD_SYS            = "x86_64-linux"
> NATIVELSBSTRING      = "ubuntu-16.04"
> TARGET_SYS           = "i586-oe-linux"
> MACHINE              = "qemux86"
> DISTRO               = "nodistro"
> DISTRO_VERSION       = "nodistro.0"
> TUNE_FEATURES        = "m32 i586"
> TARGET_FPU           = ""
> meta                 = "master:416a8c241aff0dca6b8b123e52cf8e2d40c74c8d"
> meta-oe              
> meta-python          
> meta-multimedia      = "master:4517d1c695efd0b175f4e7242e1a7bb4361527d9"
> meta-ros             = "master:dd424fbc01c29ea16c0dfc011e52e11750c729f2"
> 
> 
> It did fail as pointed out already with not finding boost_python3.
> 
> Summary: 2 tasks failed:
>  
> /home/lukas/repositories/github.com/bmwcarit/meta-ros/recipes-ros/image-
> common/camera-calibration-parsers_1.11.13.bb:do_configure
>  
> /home/lukas/repositories/github.com/bmwcarit/meta-ros/recipes-ros/vision-
> opencv/cv-bridge_1.11.16.bb:do_configure
> Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
> 
> So that commit you mentioned did not resolve the issue yet.

I didn't encounter the build issue as you paste.
Could you help to provide your reproduce steps?
Comment 9 Ross Burton 2018-10-01 15:09:52 UTC
Lukas, can you reproduce this with master?
Comment 10 Lukas Bulwahn 2018-10-01 16:17:54 UTC
Hi Mingli, hi Ross,

I have seen your request to try to reproduce, but was really way too busy the last few days to take action and answer. I will try to reproduce it tomorrow morning (CET) and report back if it is resolved or not.

Lukas
Comment 11 Lukas Bulwahn 2018-10-02 07:59:15 UTC
I can still reproduce the issue on the following build when enabling

ROS_USE_PYTHON3 = "yes"

in my local.conf

and this build configuration:

Build Configuration:
BB_VERSION           = "1.39.1"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-16.04"
TARGET_SYS           = "x86_64-oe-linux"
MACHINE              = "qemux86-64"
DISTRO               = "nodistro"
DISTRO_VERSION       = "nodistro.0"
TUNE_FEATURES        = "m64 core2"
TARGET_FPU           = ""
meta                 = "master:8dcd4e6e791c3a8b8bf5e69dca9bb9c887231d69"
meta-oe              
meta-python          
meta-multimedia      = "master:d273070293cf646fb399167dccc1a1b18dc41cb6"
meta-ros             = "master:ad00e80bd424095c0de08590c5d0df87091c36ae"

Then, 

  meta-ros/recipes-ros/vision-opencv/cv-bridge_1.11.16.bb:do_configure
  meta-ros/recipes-ros/image-common/camera-calibration-parsers_1.11.13.bb:do_configure

both fail with:

|   Unable to find the requested Boost libraries.
| 
|   Boost version: 1.68.0
| 
|   Boost include path:
|   /home/lukas/repositories/openembedded.org/openembedded-core/build/tmp-glibc/work/core2-64-oe-linux/camera-calibration-parsers/1.11.13-r0/recipe-sysroot/usr/include
| 
| 
|   Could not find the following Boost libraries:
| 
|           boost_python3
| 
|   Some (but not all) of the required Boost libraries were found.  You may
|   need to install these additional Boost libraries.  Alternatively, set
|   BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT
|   to the location of Boost.
| Call Stack (most recent call first):
|   CMakeLists.txt:10 (find_package)
Comment 12 Stephen K Jolley 2018-10-05 17:17:38 UTC
Ross, it appears that this can be reproduced.  It this satisfactory? If not, please ask the question again and place back in NEEDINFO.
Comment 13 Randy MacLeod 2018-10-12 14:53:23 UTC
Mingli was able to reproduce the error last night using Lukas's instructions.
She can take the defect but won't be able to work on it until late next week or early the following week.
Comment 14 Mingli Yu 2018-10-16 09:04:06 UTC
Patch send out as https://patchwork.openembedded.org/patch/155696/
Comment 15 Ross Burton 2018-10-18 09:20:28 UTC
So boost 1.67 changed the library name from libboost_python[MAJOR] to libboost_python[MAJOR][MINOR].  This is a deliberate change by Boost upstream and other recipes such as opencv whicih still look for just MAJOR will need to be changed.