<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE bugzilla SYSTEM "https://bugzilla.yoctoproject.org/page.cgi?id=bugzilla.dtd">

<bugzilla version="5.0.6"
          urlbase="https://bugzilla.yoctoproject.org/"
          
          maintainer="it-coreprojects-helpdesk@linuxfoundation.org"
>

    <bug>
          <bug_id>13053</bug_id>
          
          <creation_ts>2018-11-30 20:44:10 +0000</creation_ts>
          <short_desc>oe-core OCI container compatibility / deployment and conversion</short_desc>
          <delta_ts>2026-03-19 16:14:44 +0000</delta_ts>
          <reporter_accessible>1</reporter_accessible>
          <cclist_accessible>1</cclist_accessible>
          <classification_id>7</classification_id>
          <classification>Build System, Metadata &amp; Runtime</classification>
          <product>OE-Core</product>
          <component>core</component>
          <version>2.7</version>
          <rep_platform>All</rep_platform>
          <op_sys>Multiple</op_sys>
          <bug_status>RESOLVED</bug_status>
          <resolution>FIXED</resolution>
          
          
          <bug_file_loc></bug_file_loc>
          <status_whiteboard></status_whiteboard>
          <keywords></keywords>
          <priority>Medium+</priority>
          <bug_severity>enhancement</bug_severity>
          <target_milestone>6.0 M2</target_milestone>
          
          
          <everconfirmed>1</everconfirmed>
          <reporter name="Bruce Ashfield">bruce.ashfield</reporter>
          <assigned_to name="Bruce Ashfield">bruce.ashfield</assigned_to>
          <cc>kouheiaoki</cc>
    
    <cc>meta.mr.watcher</cc>
    
    <cc>meta.watcher</cc>
    
    <cc>pokylinux</cc>
    
    <cc>randy.macleod</cc>
    
    <cc>richard.purdie</cc>
          
          
          <cf_os>---</cf_os>
          <cf_regression_type>---</cf_regression_type>
          
          <cf_docchange>Yes (doc changes required)</cf_docchange>

      

      

      

          <comment_sort_order>oldest_to_newest</comment_sort_order>  
          <long_desc isprivate="0" >
    <commentid>82375</commentid>
    <comment_count>0</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2018-11-30 20:44:10 +0000</bug_when>
    <thetext>The prerequisite for this feature is bugzilla: 13051

An OCI container generated via oe-core should be capable of being copied (pushed) to a container registry, to be pull/executed on a different execution environment.

This means that the OCI container should support both the OCI imagespec v1 and the docker image format. Sufficient metadata from the build will be part of the image such 
that it can be executed on another host/container engine.

It should also be possible to install/nest the generated container &quot;image&quot; such that it can be included in a rootfs/image and run on target.

This forms the foundation of a CI/CD loop, but actual integration of the outputs into a CI environment is not part of this feature.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>84758</commentid>
    <comment_count>1</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2019-08-27 15:30:33 +0000</bug_when>
    <thetext>This will be in meta-virt for 3.0, moving to 3.1 for the larger oe-core discussion about container support in core.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85210</commentid>
    <comment_count>2</comment_count>
    <who name="Robert Berger">pokylinux</who>
    <bug_when>2019-10-04 08:48:41 +0000</bug_when>
    <thetext>@Bruce: 
Do I understand this correctly? 
All the container stuff is being moved from 3.0 to 3.1? 

and it will show up here?
http://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization

In case you have something preliminary to test please let me know.

At the moment I create a rootfs suitable for a container and out of this a docker image. First preliminary experiments to use this with kubernetes are also being made.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85216</commentid>
    <comment_count>3</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2019-10-04 12:34:25 +0000</bug_when>
    <thetext>(In reply to comment #2)
&gt; @Bruce: 
&gt; Do I understand this correctly? 
&gt; All the container stuff is being moved from 3.0 to 3.1? 
&gt; 
&gt; and it will show up here?
&gt; http://git.yoctoproject.org/cgit/cgit.cgi/meta-virtualization
&gt; 
&gt; In case you have something preliminary to test please let me know.
&gt; 
&gt; At the moment I create a rootfs suitable for a container and out of this a
&gt; docker image. First preliminary experiments to use this with kubernetes are
&gt; also being made.

Correct. The prequisite support (but 13051) is already in meta-virt and has been used for some time to create directly deployable containers (and we now have docker, runc, podman and k8s available in meta-virt to run them on target).

The &quot;on host&quot; or runquemu-like work is nearly done and that will be part of the 3.1 effort. Patches will show up in meta-virt first for that.

That being said, the containers that are being generated now could easily be deployed and run with many different container frameworks, it just needs a bit of glue for whatever you are using (that assumes that you are running on a matching architecture, or you&apos;ve arranged for qemu-system* to run things if the architecture isn&apos;t a match).</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85219</commentid>
    <comment_count>4</comment_count>
    <who name="Robert Berger">pokylinux</who>
    <bug_when>2019-10-04 12:59:22 +0000</bug_when>
    <thetext>(In reply to comment #3)
&gt; 
&gt; Correct. The prequisite support (but 13051) is already in meta-virt and has
&gt; been used for some time to create directly deployable containers (and we now
&gt; have docker, runc, podman and k8s available in meta-virt to run them on
&gt; target).

I am very interested in this. Will you be at ELCE?

&gt; 
&gt; The &quot;on host&quot; or runquemu-like work is nearly done and that will be part of
&gt; the 3.1 effort. Patches will show up in meta-virt first for that.

Not quite sure what you mean there.

&gt; 
&gt; That being said, the containers that are being generated now could easily be
&gt; deployed and run with many different container frameworks, it just needs a
&gt; bit of glue for whatever you are using (that assumes that you are running on
&gt; a matching architecture, or you&apos;ve arranged for qemu-system* to run things
&gt; if the architecture isn&apos;t a match).

At the moment I build a rootfs without kernel and so on for x86[1] and ARM and manually run some script, which uses this dockerfile[2] which just adds the rootfs which I generated before via bitbake (ADD app-container-image-mosquitto-container-x86-64.tar.bz2 /)

[1] https://gitlab.com/meta-layers/meta-resy/blob/master/conf/machine/container-x86-64.conf
[2] https://gitlab.com/app-container/app-container-mosquitto/blob/master/dockerfile/Dockerfile

Basically what I want is containers which can be deployed/managed via kubernetes where I can put pods on Embedded ARM boards while other server components run on x86 servers. So some containers will need to be x86 and ARM and the appropriate one should be picked per node.

We definitely need to talk ;)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85261</commentid>
    <comment_count>5</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2019-10-08 15:11:50 +0000</bug_when>
    <thetext>(In reply to comment #4)
&gt; (In reply to comment #3)
&gt; &gt; 
&gt; &gt; Correct. The prequisite support (but 13051) is already in meta-virt and has
&gt; &gt; been used for some time to create directly deployable containers (and we now
&gt; &gt; have docker, runc, podman and k8s available in meta-virt to run them on
&gt; &gt; target).
&gt; 
&gt; I am very interested in this. Will you be at ELCE?
&gt; 
&gt; &gt; 
&gt; &gt; The &quot;on host&quot; or runquemu-like work is nearly done and that will be part of
&gt; &gt; the 3.1 effort. Patches will show up in meta-virt first for that.
&gt; 
&gt; Not quite sure what you mean there.

In a similar way that runqemu can test our images on your dev host, we can
build containers and execute them directly on the dev-host the same way. It
isn&apos;t overly complex, but the glue and integration is nearly done.

&gt; 
&gt; &gt; 
&gt; &gt; That being said, the containers that are being generated now could easily be
&gt; &gt; deployed and run with many different container frameworks, it just needs a
&gt; &gt; bit of glue for whatever you are using (that assumes that you are running on
&gt; &gt; a matching architecture, or you&apos;ve arranged for qemu-system* to run things
&gt; &gt; if the architecture isn&apos;t a match).
&gt; 
&gt; At the moment I build a rootfs without kernel and so on for x86[1] and ARM
&gt; and manually run some script, which uses this dockerfile[2] which just adds
&gt; the rootfs which I generated before via bitbake (ADD
&gt; app-container-image-mosquitto-container-x86-64.tar.bz2 /)
&gt; 

We have other people doing similar things with tools like skopeo, and this
is an area that I&apos;ll be looking to drive some convergence.

&gt; [1]
&gt; https://gitlab.com/meta-layers/meta-resy/blob/master/conf/machine/container-
&gt; x86-64.conf
&gt; [2]
&gt; https://gitlab.com/app-container/app-container-mosquitto/blob/master/
&gt; dockerfile/Dockerfile
&gt; 
&gt; Basically what I want is containers which can be deployed/managed via
&gt; kubernetes where I can put pods on Embedded ARM boards while other server
&gt; components run on x86 servers. So some containers will need to be x86 and
&gt; ARM and the appropriate one should be picked per node.

All the components are in place for this, so we are generally ready to go.

I will be at ELCE!

&gt; 
&gt; We definitely need to talk ;)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>85264</commentid>
    <comment_count>6</comment_count>
    <who name="Robert Berger">pokylinux</who>
    <bug_when>2019-10-08 19:41:39 +0000</bug_when>
    <thetext>(In reply to comment #5)
&gt; &gt; 
&gt; &gt; Basically what I want is containers which can be deployed/managed via
&gt; &gt; kubernetes where I can put pods on Embedded ARM boards while other server
&gt; &gt; components run on x86 servers. So some containers will need to be x86 and
&gt; &gt; ARM and the appropriate one should be picked per node.
&gt; 
&gt; All the components are in place for this, so we are generally ready to go.
&gt; 
&gt; I will be at ELCE!
&gt; 
&gt; &gt; 
&gt; &gt; We definitely need to talk ;)

OK please reserve some time for me on ELCE. Dinner is on me ;)</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86767</commentid>
    <comment_count>7</comment_count>
    <who name="Robert Berger">pokylinux</who>
    <bug_when>2020-03-20 17:07:20 +0000</bug_when>
    <thetext>Not sure this is the right bug, but I am wondering how I can do base containers with .oci images.

Think about 2 .oci containers created via BitBake. Both of them contain the Python runtime. One of them hello-world-1.py and another one hello-world-2.py.

At the moment I create 2 .oci containers upload them e.g. to docker hub and pull them on my target.

In order to save download bandwidth (and I guess space on my target as well) I would like to have something like &quot;FROM ubuntu:18.04&quot; in a Dockerfile, create a Python base container and add on top of this.

How can this be done with YP/OE/Bitbake and .oci images?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86779</commentid>
    <comment_count>8</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2020-03-23 16:56:16 +0000</bug_when>
    <thetext>(In reply to comment #7)
&gt; Not sure this is the right bug, but I am wondering how I can do base
&gt; containers with .oci images.
&gt; 
&gt; Think about 2 .oci containers created via BitBake. Both of them contain the
&gt; Python runtime. One of them hello-world-1.py and another one
&gt; hello-world-2.py.
&gt; 
&gt; At the moment I create 2 .oci containers upload them e.g. to docker hub and
&gt; pull them on my target.
&gt; 
&gt; In order to save download bandwidth (and I guess space on my target as well)
&gt; I would like to have something like &quot;FROM ubuntu:18.04&quot; in a Dockerfile,
&gt; create a Python base container and add on top of this.
&gt; 
&gt; How can this be done with YP/OE/Bitbake and .oci images?

Can you clarify a couple of things ?

Do you want to create those base containers on the artifact repo (i.e. dockerhub) and then construct new containers using dockerfiles based on those uploads ?

I don&apos;t think it is that case, since there&apos;s nothing preventing you from doing that now.

Are you talking about doing this at build time, and potentially mixing yocto and non-yocto artifacts ? If so, There&apos;s no way to do this right now, since that fetching would have to be done at build time, and you&apos;d be bringing in untested/unlicenced/unknown code and then layering your build on top of it. This has been talked about on the meta-virt list, but there&apos;s no viable way to do it now.

Or are you talking about creating a layered OCI image directly from a single bitbake build ? If so, that also isn&apos;t possible, but it is something that we can look into adding. Right now, you need two separate build and pushes to a registry to achieve this.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>86781</commentid>
    <comment_count>9</comment_count>
    <who name="Robert Berger">pokylinux</who>
    <bug_when>2020-03-23 21:18:00 +0000</bug_when>
    <thetext>(In reply to comment #8)
&gt; 
&gt; Can you clarify a couple of things ?
&gt; 
&gt; Do you want to create those base containers on the artifact repo (i.e.
&gt; dockerhub) and then construct new containers using dockerfiles based on
&gt; those uploads ?
&gt; 
&gt; I don&apos;t think it is that case, since there&apos;s nothing preventing you from
&gt; doing that now.
&gt; 
&gt; Are you talking about doing this at build time, and potentially mixing yocto
&gt; and non-yocto artifacts ? If so, There&apos;s no way to do this right now, since
&gt; that fetching would have to be done at build time, and you&apos;d be bringing in
&gt; untested/unlicenced/unknown code and then layering your build on top of it.
&gt; This has been talked about on the meta-virt list, but there&apos;s no viable way
&gt; to do it now.
&gt; 
&gt; Or are you talking about creating a layered OCI image directly from a single
&gt; bitbake build ? If so, that also isn&apos;t possible, but it is something that we
&gt; can look into adding. Right now, you need two separate build and pushes to a
&gt; registry to achieve this.

I build with meta-virtualisation .oci images. I don&apos;t want to mix yocto and non-yocto artifacts.

I am also happy with 2 builds and pushes at the moment ;)

How will this work?

Let&apos;s assume I have this oci image:

https://gitlab.com/meta-layers/meta-desire/-/blob/master/dynamic-layers/virtualization-layer/recipes-core/images/app-container-image-python3-data-collector-oci.bb

which installs this:

https://gitlab.com/meta-layers/meta-desire/-/blob/master/recipes-core/images/app-container-image-python3-data-collector-common.inc

IMAGE_INSTALL += &quot; \
	busybox \
        python3 \
        python3-localdatacollector \
&quot;

and this one:

https://gitlab.com/meta-layers/meta-desire/-/blob/master/dynamic-layers/virtualization-layer/recipes-core/images/app-container-image-python3-mqttbrokerclient-oci.bb

which installs this:

https://gitlab.com/meta-layers/meta-desire/-/blob/master/recipes-core/images/app-container-image-python3-mqttbrokerclient-common.inc

IMAGE_INSTALL += &quot; \
	busybox \
        python3 \
        python3-mqttbrokerclient \
	python3-mqttbrokerclient-executor \
&quot;

I would be happy to find a way where busybox and python3 are as some &quot;base image&quot; in one build and the rest added on top of it in 2 more builds.

At the moment I build app-container-image-python3-data-collector-oci and push it (with skopeo) to docker hub and app-container-image-python3-mqttbrokerclient-oci as well and push it (with skopeo) to docker hub.

What&apos;s missing (in my brain I guess) is how to split it into base-python3-container + python3-localdatacollector and base-python3-container + python3-mqttbrokerclient and python3-mqttbrokerclient-executor.

Is this how it can be done?

Do I need to manually hack oci meta data to make it work?

What is the workflow you would suggest?</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>90382</commentid>
    <comment_count>10</comment_count>
    <who name="">kouheiaoki</who>
    <bug_when>2021-05-13 20:42:08 +0000</bug_when>
    <thetext>Dear Robert Berger.

At last, it seems some errors on the documents, I list up them below.
And I sent you a merge request on gitlab.
Could you review and merge it?

&gt; https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00-build-docker/docker-build.md

*1) [12] [13] [14] is broken link
*2) Command after &quot;In the container:&quot; in &quot;Build.&quot; section&apos;s needs &quot;bitbake&quot;</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>90384</commentid>
    <comment_count>11</comment_count>
    <who name="">kouheiaoki</who>
    <bug_when>2021-05-13 23:45:33 +0000</bug_when>
    <thetext>(In reply to comment #10)
I&apos;m sorry, This comment is my mistake...
Please delete or ignore this...</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>94092</commentid>
    <comment_count>12</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2022-09-29 15:22:44 +0000</bug_when>
    <thetext>Still valid, moving to 4.2 so it can be coordinated with binary distro efforts.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>101745</commentid>
    <comment_count>13</comment_count>
    <who name="Randy MacLeod">randy.macleod</who>
    <bug_when>2025-04-24 18:54:45 +0000</bug_when>
    <thetext>Bulk move as requesting during YP BB meeting.</thetext>
  </long_desc><long_desc isprivate="0" >
    <commentid>104747</commentid>
    <comment_count>14</comment_count>
    <who name="Bruce Ashfield">bruce.ashfield</who>
    <bug_when>2026-03-19 16:14:44 +0000</bug_when>
    <thetext>meta-virt has this implemented now.

mult-layer oci containers
vdkr/vpdm/vdcontainer

Details in the layer itself.</thetext>
  </long_desc>
      
      

    </bug>

</bugzilla>