Bug 8667 - populate_sdk task fails when Debian packaging used
Summary: populate_sdk task fails when Debian packaging used
Status: RESOLVED INVALID
Alias: None
Product: Package Management Issues
Classification: Runtime
Component: Debian Packaging - DEB (show other bugs)
Version: 1.8.2
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 4.99
Assignee: Richard Purdie
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-11-09 13:16 UTC by Martin Townsend
Modified: 2020-06-01 23:19 UTC (History)
2 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
dpkg supported archs (5.64 KB, text/plain)
2015-11-09 22:40 UTC, Aníbal Limón
no flags Details
repro_bug.patch (2.27 KB, application/octet-stream)
2018-05-29 22:41 UTC, bhs
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Townsend 2015-11-09 13:16:11 UTC
When building with
PACKAGE_CLASSES ?= "package_deb"
DPKG_ARCH ?= "armv7ahf-vfp-neon"
in my local.conf, populate_sdk fails with the following message:
ERROR: Unable to install packages. Command '/home/martin/ws_poweroasis/build/am43-devboard-aquila/bia-tmp-glibc/sysroots/x86_64-linux/usr/bin/apt-get  install --force-yes --allow-unauthenticated nativesdk-packagegroup-sdk-host packagegroup-cross-canadian-am43-devboard-aquila' returned 100:
Reading package lists...
Building dependency tree...
Reading state information...
W: Unable to read /home/martin/ws_poweroasis/build/am43-devboard-aquila/bia-tmp-glibc/work/am43_devboard_aquila-oe-linux-gnueabi/bia-image/1.0-r0/apt-sdk/preferences.d/ - DirectoryExists (2: No such file or directory)
E: Unable to locate package nativesdk-packagegroup-sdk-host

Looking in the work directory I could see that all the native related packages were called
xxxx-armv7ahf-vfp-neon.deb

After debugging I found that by putting the following in package_deb.bbclass
python () {
    if d.getVar('PACKAGES', True) != '':
        deps = ' dpkg-native:do_populate_sysroot virtual/fakeroot-native:do_populate_sysroot'
        d.appendVarFlag('do_package_write_deb', 'depends', deps)
        d.setVarFlag('do_package_write_deb', 'fakeroot', "1")

    # Map TARGET_ARCH to Debian's ideas about architectures
    darch = d.getVar('PACKAGE_ARCH', True)
    if darch in ["x86", "i486", "i586", "i686", "i686-nativesdk", "pentium"]:
         d.setVar('DPKG_ARCH', 'i386')
    elif darch in ["x86_64", "x86_64-nativesdk"]:
         d.setVar('DPKG_ARCH', 'amd64')
    elif darch == "arm":
         d.setVar('DPKG_ARCH', 'armel')

It now works fine.
Comment 1 Aníbal Limón 2015-11-09 22:10:19 UTC
(In reply to comment #0)
> When building with
> PACKAGE_CLASSES ?= "package_deb"
> DPKG_ARCH ?= "armv7ahf-vfp-neon"
> in my local.conf, populate_sdk fails with the following message:
> ERROR: Unable to install packages. Command
> '/home/martin/ws_poweroasis/build/am43-devboard-aquila/bia-tmp-glibc/
> sysroots/x86_64-linux/usr/bin/apt-get  install --force-yes
> --allow-unauthenticated nativesdk-packagegroup-sdk-host
> packagegroup-cross-canadian-am43-devboard-aquila' returned 100:
> Reading package lists...
> Building dependency tree...
> Reading state information...
> W: Unable to read
> /home/martin/ws_poweroasis/build/am43-devboard-aquila/bia-tmp-glibc/work/
> am43_devboard_aquila-oe-linux-gnueabi/bia-image/1.0-r0/apt-sdk/preferences.d/
> - DirectoryExists (2: No such file or directory)
> E: Unable to locate package nativesdk-packagegroup-sdk-host
> 
> Looking in the work directory I could see that all the native related
> packages were called
> xxxx-armv7ahf-vfp-neon.deb
> 
> After debugging I found that by putting the following in package_deb.bbclass
> python () {
>     if d.getVar('PACKAGES', True) != '':
>         deps = ' dpkg-native:do_populate_sysroot
> virtual/fakeroot-native:do_populate_sysroot'
>         d.appendVarFlag('do_package_write_deb', 'depends', deps)
>         d.setVarFlag('do_package_write_deb', 'fakeroot', "1")
> 
>     # Map TARGET_ARCH to Debian's ideas about architectures
>     darch = d.getVar('PACKAGE_ARCH', True)
>     if darch in ["x86", "i486", "i586", "i686", "i686-nativesdk", "pentium"]:
>          d.setVar('DPKG_ARCH', 'i386')
>     elif darch in ["x86_64", "x86_64-nativesdk"]:
>          d.setVar('DPKG_ARCH', 'amd64')
>     elif darch == "arm":
>          d.setVar('DPKG_ARCH', 'armel')
> 
> It now works fine.

What machine are you using? qemuarm?
Comment 2 Aníbal Limón 2015-11-09 22:40:00 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > When building with
> > PACKAGE_CLASSES ?= "package_deb"
> > DPKG_ARCH ?= "armv7ahf-vfp-neon"
> > in my local.conf, populate_sdk fails with the following message:
> > ERROR: Unable to install packages. Command
> > '/home/martin/ws_poweroasis/build/am43-devboard-aquila/bia-tmp-glibc/
> > sysroots/x86_64-linux/usr/bin/apt-get  install --force-yes
> > --allow-unauthenticated nativesdk-packagegroup-sdk-host
> > packagegroup-cross-canadian-am43-devboard-aquila' returned 100:
> > Reading package lists...
> > Building dependency tree...
> > Reading state information...
> > W: Unable to read
> > /home/martin/ws_poweroasis/build/am43-devboard-aquila/bia-tmp-glibc/work/
> > am43_devboard_aquila-oe-linux-gnueabi/bia-image/1.0-r0/apt-sdk/preferences.d/
> > - DirectoryExists (2: No such file or directory)
> > E: Unable to locate package nativesdk-packagegroup-sdk-host
> > 
> > Looking in the work directory I could see that all the native related
> > packages were called
> > xxxx-armv7ahf-vfp-neon.deb
> > 
> > After debugging I found that by putting the following in package_deb.bbclass
> > python () {
> >     if d.getVar('PACKAGES', True) != '':
> >         deps = ' dpkg-native:do_populate_sysroot
> > virtual/fakeroot-native:do_populate_sysroot'
> >         d.appendVarFlag('do_package_write_deb', 'depends', deps)
> >         d.setVarFlag('do_package_write_deb', 'fakeroot', "1")
> > 
> >     # Map TARGET_ARCH to Debian's ideas about architectures
> >     darch = d.getVar('PACKAGE_ARCH', True)
> >     if darch in ["x86", "i486", "i586", "i686", "i686-nativesdk", "pentium"]:
> >          d.setVar('DPKG_ARCH', 'i386')
> >     elif darch in ["x86_64", "x86_64-nativesdk"]:
> >          d.setVar('DPKG_ARCH', 'amd64')
> >     elif darch == "arm":
> >          d.setVar('DPKG_ARCH', 'armel')
> > 
> > It now works fine.
> 
> What machine are you using? qemuarm?

Running dpkg-architecture (-L) to list known archs armv7ahf-vfp-neon isn't recorgnized.

Dpkg version is 1.17.25.

See attachment.
Comment 3 Aníbal Limón 2015-11-09 22:40:40 UTC
Created attachment 2846 [details]
dpkg supported archs
Comment 4 Martin Townsend 2015-11-09 22:50:31 UTC
We are using a custom machine based on the variscite am43-devboard which has an ARM cortex A9.
We add armv7ahf-vfp-neon as a foreign architecture for the moment in a script that runs on first boot.  It's on my list of todo's to augment the dpkg recipe to make DPKG_ARCH the primary architecture when creating this component.  It wasn't obvious at first glance.
Comment 5 Aníbal Limón 2015-11-09 22:56:56 UTC
(In reply to comment #4)
> We are using a custom machine based on the variscite am43-devboard which has
> an ARM cortex A9.
> We add armv7ahf-vfp-neon as a foreign architecture for the moment in a
> script that runs on first boot.  It's on my list of todo's to augment the
> dpkg recipe to make DPKG_ARCH the primary architecture when creating this
> component.  It wasn't obvious at first glance.

Also your code that posted seems to you only add pentium in line,
 
- if darch in ["x86", "i486", "i586", "i686", "i686-nativesdk", "pentium"]:

Can you add the diff instead of whole code please?
Comment 6 Martin Townsend 2015-11-09 23:02:34 UTC
It adds nativesdk and instead of using DPKG_ARCH it uses PACKAGE_ARCH, here's the diff.  This was only a hack to get it working for me.  I don't really know the code well enough to come up with the real fix.

     # Map TARGET_ARCH to Debian's ideas about architectures
-    darch = d.getVar('DPKG_ARCH', True)
-    if darch in ["x86", "i486", "i586", "i686", "pentium"]:
+    darch = d.getVar('PACKAGE_ARCH', True)
+    if darch in ["x86", "i486", "i586", "i686", "i686-nativesdk", "pentium"]:
          d.setVar('DPKG_ARCH', 'i386')
-    elif darch == "x86_64":
+    elif darch in ["x86_64", "x86_64-nativesdk"]:
          d.setVar('DPKG_ARCH', 'amd64')
     elif darch == "arm":
          d.setVar('DPKG_ARCH', 'armel')
Comment 7 Aníbal Limón 2015-11-11 22:13:48 UTC
(In reply to comment #6)
> It adds nativesdk and instead of using DPKG_ARCH it uses PACKAGE_ARCH,
> here's the diff.  This was only a hack to get it working for me.  I don't
> really know the code well enough to come up with the real fix.
> 
>      # Map TARGET_ARCH to Debian's ideas about architectures
> -    darch = d.getVar('DPKG_ARCH', True)
> -    if darch in ["x86", "i486", "i586", "i686", "pentium"]:
> +    darch = d.getVar('PACKAGE_ARCH', True)
> +    if darch in ["x86", "i486", "i586", "i686", "i686-nativesdk",
> "pentium"]:
>           d.setVar('DPKG_ARCH', 'i386')
> -    elif darch == "x86_64":
> +    elif darch in ["x86_64", "x86_64-nativesdk"]:
>           d.setVar('DPKG_ARCH', 'amd64')
>      elif darch == "arm":
>           d.setVar('DPKG_ARCH', 'armel')

The PACKAGE_ARCH is the OE one to name arch'es the DPKG_ARCH maps to Debian arch'es.

These issue seems to be a specific case of your board since you are adding that arch into dpkg.

Also another variable exist for SDK build that is named SDK_ARCH (OE) and DEB_SDK_ARCH (Debian) this is analogue to the {PACKAGE, DPKG}_ARCH. This
variable is set in rootfs_deb.bbclass.

Can you modify that file instead of package_deb.bbclass for review if works?
Comment 8 Martin Townsend 2015-11-12 22:44:18 UTC
I will try DPKG_ARCH = armhf which is what I think should be the architecture from what I've read, with the reverted package_deb.bbclass. Then I'll try SDK_ARCH (OE) and DEB_SDK_ARCH and report back with findings.
Comment 9 Martin Townsend 2015-11-23 15:36:09 UTC
Tried armhf still same problem.
Tried the following in local.conf
SDK_ARCH = "x86_64"
DEB_SDK_ARCH = "x86_64"
still same problem
Comment 10 Stephen K Jolley 2015-11-23 18:56:43 UTC
NEEDINFO answered.
Comment 11 Aníbal Limón 2017-04-19 15:17:38 UTC
This bug is an enhancement because it tries to add a new architecture to apt/dpkg and then adds the support into out package management code. Moving to 2.4.
Comment 12 bhs 2018-05-29 22:41:32 UTC
Created attachment 4310 [details]
repro_bug.patch
Comment 13 bhs 2018-05-29 22:42:48 UTC
Sorry, please ignore repro_bug.patch (delete if possible). It's meant for #12761
Comment 14 Richard Purdie 2020-05-28 07:24:15 UTC
We don't really support adding in custom package architectures to debian. We made a choice just to map our names over into the debian namespace so this issue is really outside of the scope of what we support.