Bug 1455 - apt-native-0.7.14-r4: do_compile Failed: parallel-make issue?
Summary: apt-native-0.7.14-r4: do_compile Failed: parallel-make issue?
Status: RESOLVED WORKSFORME
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium normal
Target Milestone: 1.1
Assignee: Nitin Kamble
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2011-09-05 22:23 UTC by Dexuan Cui
Modified: 2011-09-08 17:48 UTC (History)
3 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Dexuan Cui 2011-09-05 22:23:21 UTC 
My conf/local is:

BB_NUMBER_THREADS = "16"
PARALLEL_MAKE = "-j 8"
MACHINE ?= "qemuarm"
DL_DIR ?= "/distro/dcui/sources/"
DISTRO ?= "poky"
PACKAGE_CLASSES ?= "package_deb"
SDKMACHINE ?= "i686"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
CONF_VERSION = "1"

The log of "bitbake core-image-sato-sdk" shows:
(When I get the issue, running "bitbake apt-natve -c compile" can succeed. So I think this is a parallel make issue)  

OE Build Configuration:
BB_VERSION        = "1.13.3"
TARGET_ARCH       = "arm"
TARGET_OS         = "linux-gnueabi"
MACHINE           = "qemuarm"
DISTRO            = "poky"
DISTRO_VERSION    = "1.0+snapshot-20110906"
TUNE_FEATURES     = "armv5 dsp thumb arm926ejs"
TARGET_FPU        = "soft"
meta
meta-yocto        = "master:99cb233e0c8058c9a406e18fedae70b0f60e3864"
...
...
ERROR: Logfile of failure stored in: /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/temp/log.do_compile.12636
Log data follows:
| NOTE: make -j 8
...
| Compiling deb/debsystem.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-pkg/debsystem.opic
| Compiling deb/debindexfile.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-pkg/debindexfile.opic
| Compiling deb/debmetaindex.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-pkg/debmetaindex.opic
| Building shared library /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/bin/libapt-pkg-libc6.3-6.so.4.6.0
| Compiling contrib/extracttar.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-inst/extracttar.opic
| Compiling contrib/arfile.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-inst/arfile.opic
| Compiling filelist.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-inst/filelist.opic
| Compiling database.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-inst/database.opic
| Compiling dirstream.cc to /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-inst/dirstream.opic
| database.cc:18: error: 'pkgDataBase' has not been declared
| database.cc:18: error: 'string' was not declared in this scope
ERROR: Function 'do_compile' failed (see /distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/temp/log.do_compile.12636 for further information)
| database.cc:18: error: 'Dir' was not declared in this scope
| database.cc:19: error: expected ',' or ';' before '{' token
| make[2]: *** [/distro/dcui/0905/p2/build/tmp/work/x86_64-linux/apt-native-0.7.14-r4/apt-0.7.14/obj/apt-inst/database.opic] Error 1
| make[2]: *** Waiting for unfinished jobs....
| contrib/extracttar.cc: In member function 'bool ExtractTar::Go(pkgDirStream&)':
| contrib/extracttar.cc:211: warning: array subscript is above array bounds
| contrib/extracttar.cc:218: warning: array subscript is above array bounds
| make[1]: *** [all] Error 2
| make: *** [all] Error 2
| ERROR: oe_runmake failed
Comment 1 Nitin Kamble 2011-09-06 09:12:45 UTC 
is this issue consistently reproducible ?
Comment 2 Dexuan Cui 2011-09-06 09:21:56 UTC 
(In reply to comment #1)
> is this issue consistently reproducible ?

Yes. Following the config in comment #0, I can almost 100% reproduce it.
Comment 3 Nitin Kamble 2011-09-06 09:24:41 UTC 
as it is a native recipe arch, machine, package_class should be irrelevant for reproducing the issue.
Comment 4 Dexuan Cui 2011-09-06 09:36:33 UTC 
(In reply to comment #3)
> as it is a native recipe arch, machine, package_class should be irrelevant for
> reproducing the issue.

Yes, you're right.

I meant there must be some race condition or parallel make issue here.
E.g., if I build from scratch, looks "bitbake apt-native" can't reproduce this bug, but "bitbake core-image-sato-sdk" can almost 100% reproduce this bug.

The config in comment #0 is only a config I used to reproduce the issue. It doesn't mean it's the necessary&sufficient condition. :-)

It reported pkgDataBase is not declared. I had a look at the code, looks a header file does define pkgDataBase and is included in the .c file... Not sure how this compile failure occurs.
Comment 5 Nitin Kamble 2011-09-06 09:38:11 UTC 
ok, thanks for the update. I am trying to reproduce it here.
Comment 6 Nitin Kamble 2011-09-07 06:00:24 UTC 
I tried to reproduce it, and it does not get reproduced.
Comment 7 Dexuan Cui 2011-09-08 17:48:51 UTC 
Nitin,
Sorry, but unluckily now even I can't reproduce the issue, either...

Looks only with my patches(not in poky master yet), it's easier to reproduce -- but I don't think my patch causes the bug.

So let me mark this bug as WorksForMe, and if I meet with it in my ongoing debugging and find a stable way to reproduce it, I'll reopen it.