Bug 3674

Summary: rebuild core-image-sato failed: not specify the order of do_populate_lic and do_rootfs
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Hongxu Jia <hongxu.jia>
Component: coreAssignee: Hongxu Jia <hongxu.jia>
Status: VERIFIED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: hongxu.jia, kai.kang, liang.li2, liezhi.yang, meta.mr.watcher, meta.watcher, Qi.Chen, sgw, yi.zhao, zhenfeng.zhao
Version: 1.4   
Target Milestone: 1.4 M5   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Hongxu Jia 2013-01-04 07:35:40 UTC
*Env
vim conf/local.conf
BB_NUMBER_THREADS = "4"
PARALLEL_MAKE = "-j 4"
PACKAGE_CLASSES ?= "package_deb"

*problem
The first time to build the deb image of core-image-sato is success. But clean
sstate and rebuild image is failed, (bitbake core-image-sato-ccleansstate &&
bitbake core-image-sato), there are some errors in log.do_populate_lic:
>>log<<
...
|DEBUG: Executing shell function sstate_create_package
|shell-init: error retrieving current directory: getcwd: cannot access parent 
 directories: No such file or directory
|chdir: error retrieving current directory: getcwd: cannot access parent 
 directories: No such file or directory
|/home/jiahongxu/yocto/build-20121232-deb2/tmp/work/qemux86-poky-linux/
 core-image-sato/1.0-r0/temp/run.sstate_create_package.3222: line 96: cd:
 /home/jiahongxu/yocto/build-20121232-deb2/tmp/work/qemux86-poky-linux/
 core-image-sato/1.0-r0/core-image-sato-1.0:No such file or directory
|DEBUG: Python function sstate_task_postfunc finished
|ERROR: Function failed: sstate_create_package (see /home/jiahongxu/yocto/
 build-20121232-deb2/tmp/work/qemux86-poky-linux/core-image-sato/1.0-r0/
 temp/log.do_populate_lic.3222for further information)
...

*analysis
1.Vim run.sstate_create_package.3222: line 96,we know the direct reason of
  the error is the dir doesn't exist
...
96 cd /home/jiahongxu/yocto/build-20121232-deb2/tmp/work/qemux86-poky-linux/
   core-image-sato/1.0-r0/core-image-sato-1.0
97 sstate_create_package
...

2.Vim log.task_order, we know there are three tasks while running bitbake 
  core-image-sato-ccleansstate && bitbake core-image-sato
...
  1 do_cleansstate (3164): log.do_cleansstate.3164
  2 do_populate_lic (3222): log.do_populate_lic.3222
  3 do_rootfs (3223): log.do_rootfs.3223
...

3.We use "bitbake -c" to execute three tasks as sequence, everything is ok.
  Especially after executing "do_populate_lic", the dir exists. And after
  "do_rootfs", the dir is removed.

*solution
Explicity specify the order of do_populate_lic and do_rootfs. Add task do_rootfs
after do_populate_lic
Comment 1 Hongxu Jia 2013-01-04 09:01:32 UTC
The task of do_populate_lic use "bb.build.exec_func" to execute shell function
"sstate_create_package", and "bb.build.exec_func" use ${B} as default dir to cd
and execute the shell function. At this place, B = "${WORKDIR}/${BPN}-${PV}/",
and "${WORKDIR}/${BPN}-${PV}/" is removed by task do_rootfs.

When we use more than one bitabke to run tasks, there is a chance that do_rootfs
and do_populate_lic are being executed at the same time. So we should explicity
specify the order of do_populate_lic and do_rootfs to let "${WORKDIR}/${BPN}-
${PV}/"could be correctly used and removed.
Comment 2 Hongxu Jia 2013-01-08 10:02:02 UTC
Patch sent on the mailing list.
Comment 3 Hongxu Jia 2013-01-09 09:07:21 UTC
As Richard suggestion:
Specify dirs in which the shell function "sstate_create_package" and
"sstate_unpack_package" are executed and don't use ${B} as default dirs
to avoid possible race with task do_rootfs at deb image creation time.

Add the following in sstate.bbclass:
sstate_create_package[dirs] = "${SSTATE_BUILDDIR}"
sstate_unpack_package[dirs] = "${SSTATE_INSTDIR}"


Patch sent on the mailing list.
Comment 4 liang li 2013-01-11 08:27:02 UTC
no such error when run "bitbake core-image-sato" (PACKAGE_CLASSES ?= "package_deb" in local.conf),then run " bitbake core-image-sato -ccleansstate ; bitbake core-image-sato" on opensuse 12.2 64bit host:

yocto@yocto-OptiPlex-990:/buildarea/tester/build_202> bitbake core-image-sato -ccleansstate
Loading cache: 100% |###########################################| ETA:  00:00:00
Loaded 1118 entries from dependency cache.

Build Configuration:
BB_VERSION        = "1.17.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "SUSE-LINUX-12.2"
TARGET_SYS        = "arm-poky-linux-gnueabi"
MACHINE           = "qemuarm"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3+snapshot-20130111"
TUNE_FEATURES     = "armv5 thumb dsp"
TARGET_FPU        = "soft"
meta              
meta-yocto        
meta-yocto-bsp    = "master:66f4c8a674d9f8db958b13866af4d63ab8cbcd44"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 2 tasks of which 0 didn't need to be rerun and all succeeded.
yocto@yocto-OptiPlex-990:/buildarea/tester/build_202> bitbake core-image-sato  
Loading cache: 100% |###########################################| ETA:  00:00:00
Loaded 1118 entries from dependency cache.

Build Configuration:
BB_VERSION        = "1.17.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "SUSE-LINUX-12.2"
TARGET_SYS        = "arm-poky-linux-gnueabi"
MACHINE           = "qemuarm"
DISTRO            = "poky"
DISTRO_VERSION    = "1.3+snapshot-20130111"
TUNE_FEATURES     = "armv5 thumb dsp"
TARGET_FPU        = "soft"
meta              
meta-yocto        
meta-yocto-bsp    = "master:66f4c8a674d9f8db958b13866af4d63ab8cbcd44"

NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: /buildarea/poky.ubuntu.new1/poky/LICENSE could not be copied for some reason. It may not exist. WARN for now.
WARNING: /buildarea/poky.ubuntu.new1/poky/meta/COPYING.MIT could not be copied for some reason. It may not exist. WARN for now.
NOTE: Tasks Summary: Attempted 5250 tasks of which 5243 didn't need to be rerun and all succeeded.

Summary: There were 2 WARNING messages shown.

git info :66f4c8a674d9f8db958b13866af4d63ab8cbcd44"  
git branch : poky/master
Comment 5 Hongxu Jia 2013-01-11 09:41:13 UTC
Patch has been merged into your testing branch.
git info :66f4c8a674d9f8db958b13866af4d63ab8cbcd44"  
git branch : poky/master



Merged into master:
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=be78e8190a38c8aff570ed7a3f93a531a296c37d
Comment 6 liang li 2013-01-11 10:05:40 UTC
as Comment 4 said.