Bug 1496 - [multilib] lib32 build against qemux86 failed during do_rootfs
Summary: [multilib] lib32 build against qemux86 failed during do_rootfs
Status: VERIFIED FIXED
Alias: None
Product: Meta-yocto
Classification: Build System, Metadata & Runtime
Component: meta-yocto (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: High normal
Target Milestone: 1.1
Assignee: Richard Purdie
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2011-09-19 06:16 UTC by Jiajun Xu
Modified: 2011-10-08 08:34 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments
Enable images to work with mutlilib - POC (1.20 KB, patch)
2011-09-22 01:32 UTC, Richard Purdie
no flags Details | Diff
Better patch showing how to map variables in image.bbclass case (1.44 KB, patch)
2011-09-22 03:37 UTC, Richard Purdie
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jiajun Xu 2011-09-19 06:16:13 UTC
tree/branch: poky/1.1_M4
commit: 6462c158fc482e495c1171ffc7747e2ad650a292

With 1.1 M4 branch, if I build a lib32 sato image for qemux86 against rpm, it will report failure when do_rootfs for the image.

#########
| Generating solve db for /home/jxu49/osel/poky-rc2/build/tmp/deploy/rpm/all...
|    total:               1      0.000000 MB      0.117477 secs
|    fingerprint:        96      0.003280 MB      0.004312 secs
|    install:            32      0.000000 MB      0.064543 secs
|    dbadd:              32      0.000000 MB      0.061846 secs
|    dbget:             593      0.000000 MB      0.000224 secs
|    dbput:              32      0.263180 MB      0.037912 secs
|    readhdr:           321      0.527296 MB      0.000529 secs
|    hdrload:           160      0.783212 MB      0.001197 secs
|    hdrget:           5283      0.000000 MB      0.002113 secs
| Processing locale-base-en-us...
| Processing locale-base-en-gb...
| Processing lib32-task-core-boot...
| Processing lib32-task-core-apps-console...
| Processing lib32-task-core-ssh-dropbear...
| Unable to find package lib32-task-core-ssh-dropbear (task-core-ssh-dropbear)!
| ERROR: Function 'do_rootfs' failed (see /home/jxu49/osel/poky-rc2/build/tmp/work/qemux86-pokymllib32-linux/lib32-core-image-sato-1.0-r0/temp/log.do_rootfs.11249 for further information)
NOTE: package lib32-core-image-sato-1.0-r0: task do_rootfs: Failed
ERROR: Task 8 (virtual:multilib:lib32:/home/jxu49/osel/poky-rc2/meta/recipes-sato/images/core-image-sato.bb, do_rootfs) failed with exit code '1'
ERROR: 'virtual:multilib:lib32:/home/jxu49/osel/poky-rc2/meta/recipes-sato/images/core-image-sato.bb' failed
#########
Comment 1 Richard Purdie 2011-09-21 17:52:52 UTC
Jiajun: Could you include the local.conf settings you're using for this build and what target you're building please?
Comment 2 Jiajun Xu 2011-09-22 00:45:17 UTC
After discussing with RP and Dongxiao, my previous configuration is wrong. Now I re-test the lib32-core-image-sato build against MACHINE qemux86 and get following failure when do_rootfs. Update the bug title to reflect the re-test result.

Configuration of local.conf:
Target: lib32-core-image-sato
MACHINE: qemux86
###
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
###


#######
NOTE: Resolving any missing task queue dependencies
NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Running task 5507 of 5509 (ID: 8, virtual:multilib:lib32:/home/jxu49/osel/poky-rc2-tmp/meta/recipes-sato/images/core-image-sato.bb, do_rootfs)
NOTE: package lib32-core-image-sato-1.0-r0: task do_rootfs: Started
ERROR: Function 'do_rootfs' failed (see /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/work/qemux86-pokymllib32-linux/lib32-core-image-sato-1.0-r0/temp/log.do_rootfs.8656 for further information)
ERROR: Logfile of failure stored in: /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/work/qemux86-pokymllib32-linux/lib32-core-image-sato-1.0-r0/temp/log.do_rootfs.8656
Log data follows:
| Generating solve db for /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/deploy/rpm/qemux86...
|    total:               1      0.000000 MB      0.956078 secs
|    fingerprint:      1176      0.007038 MB      0.028398 secs
|    install:           392      0.000000 MB      0.515904 secs
|    dbadd:             392      0.000000 MB      0.505608 secs
|    dbget:            2192      0.000000 MB      0.002672 secs
|    dbput:             392      1.833220 MB      0.429663 secs
|    readhdr:          3921      3.631144 MB      0.004358 secs
|    hdrload:          1966      5.392696 MB      0.013285 secs
|    hdrget:          67441      0.000000 MB      0.013922 secs
| Generating solve db for /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/deploy/rpm/all...
|    total:               1      0.000000 MB      0.120617 secs
|    fingerprint:        96      0.003280 MB      0.005061 secs
|    install:            32      0.000000 MB      0.060411 secs
|    dbadd:              32      0.000000 MB      0.057227 secs
|    dbget:             593      0.000000 MB      0.000194 secs
|    dbput:              32      0.263936 MB      0.033075 secs
|    readhdr:           321      0.527296 MB      0.000474 secs
|    hdrload:           160      0.783968 MB      0.001036 secs
|    hdrget:           5283      0.000000 MB      0.001771 secs
| Generating solve db for /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/deploy/rpm/qemux86...
|    total:               1      0.000000 MB      0.919319 secs
|    fingerprint:      1176      0.007038 MB      0.024836 secs
|    install:           392      0.000000 MB      0.513258 secs
|    dbadd:             392      0.000000 MB      0.502754 secs
|    dbget:            2192      0.000000 MB      0.002710 secs
|    dbput:             392      1.833220 MB      0.427266 secs
|    readhdr:          3921      3.631144 MB      0.004343 secs
|    hdrload:          1966      5.392696 MB      0.013321 secs
|    hdrget:          67441      0.000000 MB      0.012641 secs
| Generating solve db for /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/deploy/rpm/all...
|    total:               1      0.000000 MB      0.114690 secs
|    fingerprint:        96      0.003280 MB      0.004677 secs
|    install:            32      0.000000 MB      0.058752 secs
|    dbadd:              32      0.000000 MB      0.055867 secs
|    dbget:             593      0.000000 MB      0.000203 secs
|    dbput:              32      0.263936 MB      0.033427 secs
|    readhdr:           321      0.527296 MB      0.000472 secs
|    hdrload:           160      0.783968 MB      0.001094 secs
|    hdrget:           5283      0.000000 MB      0.001649 secs
| Generating solve db for /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/deploy/rpm/all...
|    total:               1      0.000000 MB      0.114917 secs
|    fingerprint:        96      0.003280 MB      0.004640 secs
|    install:            32      0.000000 MB      0.059053 secs
|    dbadd:              32      0.000000 MB      0.056181 secs
|    dbget:             593      0.000000 MB      0.000219 secs
|    dbput:              32      0.263936 MB      0.033792 secs
|    readhdr:           321      0.527296 MB      0.000473 secs
|    hdrload:           160      0.783968 MB      0.001137 secs
|    hdrget:           5283      0.000000 MB      0.001746 secs
| Processing locale-base-en-us...
| Unable to find package locale-base-en-us (locale-base-en-us)!
| ERROR: Function 'do_rootfs' failed (see /home/jxu49/osel/poky-rc2-tmp/lib32-core-build-qemux86/tmp/work/qemux86-pokymllib32-linux/lib32-core-image-sato-1.0-r0/temp/log.do_rootfs.8656 for further information)
NOTE: package lib32-core-image-sato-1.0-r0: task do_rootfs: Failed
ERROR: Task 8 (virtual:multilib:lib32:/home/jxu49/osel/poky-rc2-tmp/meta/recipes-sato/images/core-image-sato.bb, do_rootfs) failed with exit code '1'
ERROR: 'virtual:multilib:lib32:/home/jxu49/osel/poky-rc2-tmp/meta/recipes-sato/images/core-image-sato.bb' failed
#######
Comment 3 Richard Purdie 2011-09-22 01:32:14 UTC
Ok, now I understand exactly what is happening here. In summary its the "lib32-core-image-sato" that is failing. We'd not considered someone building a multilib image target up to this point.

The image code assumes it starts in normal target package context. multilib.bbclass simply doesn't know how to extend an image class file. I'll attach a patch as a quick attempt at teaching it this and it kind of works (you get an image) but there is a fundamental problem. The code makes the assumption that target packages are in IMAGE_INSTALL and multilib packages are in MULTILIB_IMAGE_INSTALL. I'd previously commented I didn't see why we needed to do this, now we see why its a problem :/.

To make my patch work, we need to just support IMAGE_INSTALL containing both multilib and normal targets. We then need to filter this into sets of multilibs for the code that needs the separate list.

So in summary, we can make this work but there are several sets of changes needed. I'm tempted to say this isn't supported in 1.1 (remove the image targets themselves from the list in multilib.conf) and then make this a post 1.1 task.
Comment 4 Richard Purdie 2011-09-22 01:32:53 UTC
Created attachment 248 [details]
Enable images to work with mutlilib - POC
Comment 5 Dongxiao Xu 2011-09-22 02:34:12 UTC
We already mapped the PACKAGE_INSTALL in multilib.bbclass and I think it contains IMAGE_INSTALL.

when building the lib32-core-image-sato, we defined the DEFAULTTUNE_virtclass-multilib-lib32="x86", that means when doing rootfs, system will trying to search "all", "x86", and "lib32-qemux86" as its installation repos. 

On the other hand when packages generates RPM files, the destination folder is: '${PKGWRITEDIRRPM}/${PACKAGE_ARCH}', where the PACKAGE_ARCH is defined as "${TUNE_PKGARCH}". As we know for qemux86, TUNE_PKGARCH is defined as i586, so packages will be written to deploy/rpm/i586 folder, which is not in the rootfs repo.
Comment 6 Richard Purdie 2011-09-22 03:37:47 UTC
Created attachment 249 [details]
Better patch showing how to map variables in image.bbclass case

The attached patch is a better version of how to map variables for image.bbclass. As noted in the code, we have no mechanism to handle adding multilib recommended/attemponly packages though so this is an issue with the patch as it stands as those items will get ignored.
Comment 7 Richard Purdie 2011-09-22 04:21:34 UTC
Talking further with Dongxiao, there are multiple issues here. The IMAGE_INSTALL/PACKAGE_INSTALL issue does exist and needs addressing since whilst we did remap the variable, we remapped several variables we shouldn't in the image.bbclass case and also, we placed multilib packages into PACKAGE_INSTALL when they need to go into MULTILIB_PACKAGE_INSTALL.

The second issue is that in the configured multilib case in this bug, both the target packages and the multilib "lib32" packages both get placed in the deploy/rpm/i586 directory. I propose the solution is mutlilib packages should be placed into "lib32-i586" instead.
Comment 8 Jiajun Xu 2011-09-26 05:56:58 UTC
With latest edison branch, lib32 build against qemux86 still fails during do_rootfs:


#########
Generating solve db for /home/jxu49/osel/poky/lib32-core-image-qemux86/tmp/deploy/rpm/all...
   total:               1      0.000000 MB      0.124053 secs
   fingerprint:        96      0.003280 MB      0.004623 secs
   install:            32      0.000000 MB      0.068264 secs
   dbadd:              32      0.000000 MB      0.065528 secs
   dbget:             593      0.000000 MB      0.000242 secs
   dbput:              32      0.262388 MB      0.039725 secs
   readhdr:           321      0.524712 MB      0.000552 secs
   hdrload:           160      0.779836 MB      0.001288 secs
   hdrget:           5288      0.000000 MB      0.002535 secs
Processing locale-base-en-us...
Processing locale-base-en-gb...
Manifest: /home/jxu49/osel/poky/lib32-core-image-qemux86/tmp/work/qemux86-poky-linux/lib32-core-image-sato-1.0-r0/rootfs/install/install.manifest
Processing lib32-task-core-boot...
Unable to find package lib32-task-core-boot (task-core-boot)!
ERROR: Function 'do_rootfs' failed (see /home/jxu49/osel/poky/lib32-core-image-qemux86/tmp/work/qemux86-poky-linux/lib32-core-image-sato-1.0-r0/temp/log.do_rootfs.6226 for further information)
#########
Comment 9 Richard Purdie 2011-09-26 06:39:25 UTC
I think Dongxiao had insight into this earlier in that we need to also map the RDEPENDS field for images.
Comment 11 Jiajun Xu 2011-10-08 08:34:09 UTC
Verify the bug as fixed against edison branch.

commit 5ed59ae0f25bf673d514df2371da7e0415b62bb2
Author: Richard Purdie <richard.purdie@linuxfoundation.org>
Date:   Fri Oct 7 00:00:21 2011 +0100

    local.conf.sample: Disable interactive patch resolution for now since
doesn't work well

    Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>