I searched manual to find how to produce containers(I'd like to run it with docker), but I coulon't find it. Could you document how to produce containers?
Hi Thanks for asking! Actually, the use of Docker to run the Yocto Project is documented on https://docs.yoctoproject.org/dev-manual/start.html#setting-up-to-use-cross-platforms-crops Marking this issue as "resolved", but don't hesitate to create a more specific issue if you're still missing details. Thanks again, Michael.
Dear Michael Opdenacker. Thank you for replying me, but this isn't one I expect. This explain how to build something WITH Docker, but I'd like to know build DOCKER IMAGES. Could you give me some more advice?
Hi Thanks for the clarification. I found some resources at https://www.google.com/search?q=yocto+IMAGE_FSTYPES+container At least with IMAGE_FSTYPES = “container” you can generate container images. Would this be what you're looking for? Cheers, Michael.
> I found some resources at https://www.google.com/search?q=yocto+IMAGE_FSTYPES+container I had read some resources below https://elinux.org/images/6/62/Building-Container-Images-with-OpenEmbedded-and-the-Yocto-Project-Scott-Murray-Konsulko-Group-1. https://wiki.yoctoproject.org/wiki/images/archive/d/d1/20190820185056%21Yocto_DevDay_Class_San_Diego_2019.pdf But I couldn't understand the point for me(Maybe because these are used at presentation and additional oral explanation should be needed for me). I think it's better to enrich the part of building container in Yocto's official manual for other Yocto users. > IMAGE_FSTYPES = “container” I tried above, but I got an error. I'll report the error details a few days later..
Adding some comments from IRC, in favor of adding documentation on this topic indeed.. 15:39 < LetoThe2nd> michaelo: i would say it is a common use case these days, but usually requires some form of custom distro/machine combination. i've presented it like that: https://youtu.be/jPbcQEffzJo 15:41 < ndec> michaelo: looks like a good thing to add to our documentation.. assuming we can find a way to document it properly :-) Re-opening the bug
Another useful link: https://github.com/LetoThe2nd/meta-containerization
Thank you for agreeing with me and sharing useful video! I'd like to watch it. > > IMAGE_FSTYPES = “container” > I tried above, but I got an error. > I'll report the error details a few days later.. I got error like below ERROR: /xx/xx/image-container.bb: No IMAGE_CMD defined for IMAGE_FSTYPES entry 'container' - possibly invalid type name or missing support class I replaced recipe's path to other not to including technical information. I can't share the recipe because it is our company's resource, sorry... But I can answer some question of recipe not identifying company's information. If you are interested in my recipe, please give me questions.
meta-virtualization is more active for containers that meta-containerization.
Hello, Which layers and branches are you using to do this? If you add the following to local.conf, it should work: IMAGE_FSTYPES = " tar.gz container" PREFERRED_PROVIDER_virtual/kernel = "linux-dummy" You can then run e.g. "bitbake core-image-minimal", which will generate a .tar.gz rootfs image that can be built into a container using tools such as Docker or Kaniko. The Dockerfile used for turning the rootfs tarball into the image can be as simple as this: FROM scratch ADD core-image-minimal-qemux86-64.tar.gz /
Robert Berger, it would be nice to have your input from the OCI container tooling side.
We need first to figure out where the doc should go about e.g. how to create docker and oci containers. I guess I could cook up some simple examples how to create them and also how to test a few things. I started here[1] with a few tests mainly for docker, docker-compose, and a bit of podman and podman-compose. But certainly, I could add also how to create e.g. for x86-64 docker and oci containers, so people would have an easy start. On top of that, I can also create multi-arch containers.
This[1] is my first take on how to build a root file system suitable for inclusion in a Docker container with poky and how to add it to the Dockerfile. As soon as I find some time I'll add also how to do that with meta-virtualization and oci images. It will be kind of an add-on to what I describe here[1] but allows also build images for e.g. arm. At the moment I'm thinking about just another branch in the same repo, so people who already know the first step can concentrate on the difference. [1] https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00-build-docker/docker-build.md
This[1] is my first take on building oci containers. [1] https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md
kouheiaoki@gmail.com, timothy.t.orling@intel.com would you please have a look at my instructions and see if they work for you? Comments are welcome!
Robert Berger Thank you very much! I'll have a look! Which one should I check? https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00-build-docker/docker-build.md https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md
(In reply to comment #15) > Robert Berger > Thank you very much! > I'll have a look! > > Which one should I check? > > https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00- > build-docker/docker-build.md > > https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01- > build-oci/oci-build.md Depends on what you want ;) If you want only x86-64 and include a rootfs built by poky into a Dockerfile it's: https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00-build-docker/docker-build.md If you want *) to build for various architectures like x86-64, arm32, arm64 (those I tried) *) to get rid of docker in your build process *) to be able to use except for docker also other oci compatible engines e.g. podman I suggest you use the second approach: https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md Since both approaches have quite some overlap I typically use both. The first one to get a container up and running and tested on my PC and the second one to get it on an embedded system. For simplicity, I didn't document: 1) how to add the container to a root file system as a package, so you could get rid of the registry. 2) how to create multi-arch containes.
OK, thanks a lot! I'll check both instructions. I'll do it a few days later.. I'm in vacance, excuse me for the delay to checking them.
Dear Robert Berger I tried to build container according to https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md first, but I'm stuck. There are 5 point bothering me. Can you check these? 1) 2nd get meta-openembedded will not be necessary(I guess it's mistake by copy & paste). 2) bblayers.conf[17], conf-notes.txt[18], local.conf.sample[19] is out of link. 3) ln -sf sources/manifests/resy-cooker.sh res-cooker.sh Iguess this isn't "res-cooker.sh" but "resy-cooker.sh"("y" after "res" added). 4) After below, cd /workdir ./resy-poky-container.sh container-x86-64-ex-compact-oci app-container-image-lighttpd-oci didn't work and output error message below. bash: app-container-image-lighttpd-oci: command not found I guess below command is needed. source /workdir/resy-cooker.sh 5) After 4), I got an error like below bash: resy-cooker.sh: line 2288: syntax error near unexpected token `(' bash: resy-cooker.sh: line 2288: ` #echo "+ (1) bitbake $var ;[ $? -ne 0 ] && echo "ERRORS foound" && exit 1"' it's very strange because line 2288 is comment line, so this line shouldn't have any error. I can't find "fi" corresponding "if" on line 781. Does this have any effect the error?
>1) >2nd get meta-openembedded will not be necessary(I guess it's mistake by copy & > >paste). It is needed by meta-virtualization. No copy/paste error? >2) >bblayers.conf[17], conf-notes.txt[18], local.conf.sample[19] is out of link. What do you mean by "out of link"? Those are the templates on which bblayers.conf, local.conf are created. >3) >ln -sf sources/manifests/resy-cooker.sh res-cooker.sh >Iguess this isn't "res-cooker.sh" but "resy-cooker.sh"("y" after "res" added). Ah yes. Thanks. >4) >After below, >cd /workdir >./resy-poky-container.sh container-x86-64-ex-compact-oci >app-container-image-lighttpd-oci didn't work and output error message below. >bash: app-container-image-lighttpd-oci: command not found 4.1) can you search in my layer for app-container-image-lighttpd-oci? It should be here: https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/dynamic-layers/virtualization-layer/recipes-core/images/app-container-image-lighttpd-oci.bb If it's not you got the wrong branch >4.2) if it is you have the wrong bblayers.conf > >Which is possible due to your comment above with "out of link" > >I guess below command is needed. >source /workdir/resy-cooker.sh Not actually I would say, since you already called "./resy-poky-container.sh container-x86-64-ex-compact-oci" above. So you should end up in a container where already everything is set up for you. My ususal workflow is to call it. I call ./resy-poky-container.sh to start the container and get a shell prompt inside it and then inside the container I execute "source /workdir/resy-cooker.sh container-x86-64-ex-compact-oci". This should not be necessary if you follow the instructions. >5) >After 4), I got an error like below > >bash: resy-cooker.sh: line 2288: syntax error near unexpected token `(' >bash: resy-cooker.sh: line 2288: ` #echo "+ (1) bitbake $var ;[ $? -ne >0 ] && echo "ERRORS foound" && exit 1"' >it's very strange because line 2288 is comment line, so this line shouldn't have >any error. >I can't find "fi" corresponding "if" on line 781. >Does this have any effect the error? Who calls resy-cooker.sh and from where? Indeed 2288 is commented out and seems to create an error in your setup. No idea how this can happen ;) I believe the most efficient way to get you set up would be to join the yocto chat and PM me - rber@freenode. Ideally we can then also work out some kind of ssh access for me to your box so I can have a look what's going on.
(In reply to comment #19) > >1) > >2nd get meta-openembedded will not be necessary(I guess it's mistake by copy & > >paste). > > It is needed by meta-virtualization. No copy/paste error? Sorry, Can I add something? There are two "get meta-openembedded" sections, one is after "get meta-virtualization", the other is after "get meta-container-examples". "2nd" means 'after "get meta-container-examples"' one. > >2) > >bblayers.conf[17], conf-notes.txt[18], local.conf.sample[19] is out of link. > > What do you mean by "out of link"? > > Those are the templates on which bblayers.conf, local.conf are created. Sorry for my poor English, I'd like to say "There are some broken link". > >4) > >After below, > >cd /workdir > >./resy-poky-container.sh container-x86-64-ex-compact-oci > > >app-container-image-lighttpd-oci didn't work and output error message below. > >bash: app-container-image-lighttpd-oci: command not found > > 4.1) can you search in my layer for app-container-image-lighttpd-oci? > > It should be here: > > https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/dynamic- > layers/virtualization-layer/recipes-core/images/app-container-image-lighttpd- > oci.bb Do you mean 'execute "bitbake app-container-image-lighttpd-oci"'? Sorry, I couldn't read it from only "app-container-image-lighttpd-oci". Your instruction may lack "bitbake". > >4.2) if it is you have the wrong bblayers.conf > > > >Which is possible due to your comment above with "out of link" > > > >I guess below command is needed. > >source /workdir/resy-cooker.sh > > Not actually I would say, since you already called "./resy-poky-container.sh > container-x86-64-ex-compact-oci" above. > > So you should end up in a container where already everything is set up for > you. > > My ususal workflow is to call it. > > I call ./resy-poky-container.sh to start the container and get a shell > prompt inside it and then inside the container I execute "source > /workdir/resy-cooker.sh container-x86-64-ex-compact-oci". > > This should not be necessary if you follow the instructions. > > >5) > >After 4), I got an error like below > > > >bash: resy-cooker.sh: line 2288: syntax error near unexpected token `(' > >bash: resy-cooker.sh: line 2288: ` #echo "+ (1) bitbake $var ;[ $? -ne >0 ] && echo "ERRORS foound" && exit 1"' > > >it's very strange because line 2288 is comment line, so this line shouldn't have >any error. > > >I can't find "fi" corresponding "if" on line 781. > >Does this have any effect the error? > > Who calls resy-cooker.sh and from where? pokyuser called in the container. > Indeed 2288 is commented out and seems to create an error in your setup. > No idea how this can happen ;) OK, I'll check my setup > I believe the most efficient way to get you set up would be to join the > yocto chat and PM me - rber@freenode. Thank you for your kindness. I don't have freenode account, but I'll get one soon. > Ideally we can then also work out some kind of ssh access for me to your box > so I can have a look what's going on. Soory, this is not my private PC, so ssh access isn't allowed other than me. But I provide information as much as I can.
Hi, (In reply to comment #20) > (In reply to comment #19) > > >1) > > >2nd get meta-openembedded will not be necessary(I guess it's mistake by copy & > >paste). > > > > It is needed by meta-virtualization. No copy/paste error? > Sorry, Can I add something? > There are two "get meta-openembedded" sections, > one is after "get meta-virtualization", > the other is after "get meta-container-examples". > "2nd" means 'after "get meta-container-examples"' one. Ah I see. That's a copy paste error of the heading only ;) This is another layer you need and if you don't have it it would explain why the repice can not be found. > > > >2) > > >bblayers.conf[17], conf-notes.txt[18], local.conf.sample[19] is out of link. > > > > What do you mean by "out of link"? > > > > Those are the templates on which bblayers.conf, local.conf are created. > Sorry for my poor English, I'd like to say "There are some broken link". Which? > > > >4) > > >After below, > > >cd /workdir > > >./resy-poky-container.sh container-x86-64-ex-compact-oci > > > > >app-container-image-lighttpd-oci didn't work and output error message below. > > >bash: app-container-image-lighttpd-oci: command not found > > > > 4.1) can you search in my layer for app-container-image-lighttpd-oci? > > > > It should be here: > > > > https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/dynamic- > > layers/virtualization-layer/recipes-core/images/app-container-image-lighttpd- > > oci.bb > Do you mean 'execute "bitbake app-container-image-lighttpd-oci"'? > Sorry, I couldn't read it from only "app-container-image-lighttpd-oci". > Your instruction may lack "bitbake". find | grep app-container-image-lighttpd-oci > > > >4.2) if it is you have the wrong bblayers.conf > > > > > >Which is possible due to your comment above with "out of link" > > > > > >I guess below command is needed. > > >source /workdir/resy-cooker.sh > > > > Not actually I would say, since you already called "./resy-poky-container.sh > > container-x86-64-ex-compact-oci" above. > > > > So you should end up in a container where already everything is set up for > > you. > > > > My ususal workflow is to call it. > > > > I call ./resy-poky-container.sh to start the container and get a shell > > prompt inside it and then inside the container I execute "source > > /workdir/resy-cooker.sh container-x86-64-ex-compact-oci". > > > > This should not be necessary if you follow the instructions. > > > > >5) > > >After 4), I got an error like below > > > > > >bash: resy-cooker.sh: line 2288: syntax error near unexpected token `(' > > >bash: resy-cooker.sh: line 2288: ` #echo "+ (1) bitbake $var ;[ $? -ne >0 ] && echo "ERRORS foound" && exit 1"' > > > > >it's very strange because line 2288 is comment line, so this line shouldn't have >any error. > > > > >I can't find "fi" corresponding "if" on line 781. > > >Does this have any effect the error? > > > > Who calls resy-cooker.sh and from where? > pokyuser called in the container. > > > Indeed 2288 is commented out and seems to create an error in your setup. > > No idea how this can happen ;) > OK, I'll check my setup > > > I believe the most efficient way to get you set up would be to join the > > yocto chat and PM me - rber@freenode. > Thank you for your kindness. > I don't have freenode account, but I'll get one soon. > > > Ideally we can then also work out some kind of ssh access for me to your box > > so I can have a look what's going on. > Soory, this is not my private PC, so ssh access isn't allowed other than me. > But I provide information as much as I can. Hehe so let's see how much time it will take. I can give you access to one of my machines so you can see how it is supposed to look like if you think that helps.
At first, I'm sorry. I found my mistake in my setup. I forgot cloning meta-virtualization... I cloned it just now, and "bitbake" working now! Now bitbaking, I'll check result after a few hours(Maybe 2 or 3 hours). (In reply to comment #21) > Hi, > > (In reply to comment #20) > > (In reply to comment #19) > > > >1) > > > >2nd get meta-openembedded will not be necessary(I guess it's mistake by copy & > >paste). > > > > > > It is needed by meta-virtualization. No copy/paste error? > > Sorry, Can I add something? > > There are two "get meta-openembedded" sections, > > one is after "get meta-virtualization", > > the other is after "get meta-container-examples". > > "2nd" means 'after "get meta-container-examples"' one. > > Ah I see. > > That's a copy paste error of the heading only ;) > > This is another layer you need and if you don't have it it would explain why > the repice can not be found. OK, this is only the heading error :) I cloned meta-container-ex-compact-oci. > > > > > >2) > > > >bblayers.conf[17], conf-notes.txt[18], local.conf.sample[19] is out of link. > > > > > > What do you mean by "out of link"? > > > > > > Those are the templates on which bblayers.conf, local.conf are created. > > Sorry for my poor English, I'd like to say "There are some broken link". > > Which? Below. *) [17] https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/template-container-x86-64-ex-compact-master/bblayers.conf.sample in section https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md#bblayersconf *) [18] https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/template-container-x86-64-ex-compact-master/conf-notes.txt in section https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md#conf-notestxt *) [19] https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/template-container-x86-64-ex-compact-master/local.conf.sample in section https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md#siteconf > > > >4) > > > >After below, > > > >cd /workdir > > > >./resy-poky-container.sh container-x86-64-ex-compact-oci > > > > > > >app-container-image-lighttpd-oci didn't work and output error message below. > > > >bash: app-container-image-lighttpd-oci: command not found > > > > > > 4.1) can you search in my layer for app-container-image-lighttpd-oci? > > > > > > It should be here: > > > > > > https://gitlab.com/meta-layers/meta-container-ex-compact/-/blob/oci/dynamic- > > > layers/virtualization-layer/recipes-core/images/app-container-image-lighttpd- > > > oci.bb > > Do you mean 'execute "bitbake app-container-image-lighttpd-oci"'? > > Sorry, I couldn't read it from only "app-container-image-lighttpd-oci". > > Your instruction may lack "bitbake". > > find | grep app-container-image-lighttpd-oci Sorry, I meant I found the recipe but you instruction in https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md#build-container-and-setup after "In the container:" looks NO "bitbake". > > > >4.2) if it is you have the wrong bblayers.conf > > > > > > > >Which is possible due to your comment above with "out of link" > > > > > > > >I guess below command is needed. > > > >source /workdir/resy-cooker.sh > > > > > > Not actually I would say, since you already called "./resy-poky-container.sh > > > container-x86-64-ex-compact-oci" above. > > > > > > So you should end up in a container where already everything is set up for > > > you. > > > > > > My ususal workflow is to call it. > > > > > > I call ./resy-poky-container.sh to start the container and get a shell > > > prompt inside it and then inside the container I execute "source > > > /workdir/resy-cooker.sh container-x86-64-ex-compact-oci". > > > > > > This should not be necessary if you follow the instructions. > > > > > > >5) > > > >After 4), I got an error like below > > > > > > > >bash: resy-cooker.sh: line 2288: syntax error near unexpected token `(' > > > >bash: resy-cooker.sh: line 2288: ` #echo "+ (1) bitbake $var ;[ $? -ne >0 ] && echo "ERRORS foound" && exit 1"' > > > > > > >it's very strange because line 2288 is comment line, so this line shouldn't have >any error. > > > > > > >I can't find "fi" corresponding "if" on line 781. > > > >Does this have any effect the error? > > > > > > Who calls resy-cooker.sh and from where? > > pokyuser called in the container. > > > > > Indeed 2288 is commented out and seems to create an error in your setup. > > > No idea how this can happen ;) > > OK, I'll check my setup 4.2) and 5) will be caused by what I forgot cloning meta-virtualization. The error doesn't occur now.
(In reply to comment #22) > At first, I'm sorry. > I found my mistake in my setup. > I forgot cloning meta-virtualization... > I cloned it just now, and "bitbake" working now! > Now bitbaking, I'll check result after a few hours(Maybe 2 or 3 hours). > OK great! I also updated (hopefully) the things you reported in my doc. Please have a look again and check if something is missing. The party goes on after your build is ready - skopeo!
Dear Robert Berger. Thanks for big help & sorry to be late for response to you! I'm sorry, I stop trying skopeo because I'd like to use container without dockerhub. > https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00-build-docker/docker-build.md > > https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md I understand now first approach is the way running container on a local PC, and second one is the way running container on an embedded system. I prefer first one because I'd love to test containers on local PC if images by bitbake is built for embedded system. > The first one to get a container up and running and tested on my PC and the second one to get it on an embedded system. I misunderstood above means "second one is the way producing container including image built for embedded system like arm". Today I can't try skopeo because of lack of time, but I remember second approach when running container on an embedded system.
(In reply to comment #24) > Dear Robert Berger. > > Thanks for big help & sorry to be late for response to you! > > I'm sorry, I stop trying skopeo because I'd like to use container without > dockerhub. Do you want some workflow where you create the rootfs, include it in your Dockerfile, build a container and spin it up without docker hub for testing on your PC? Or do you want some container to end up on your Embedded device (also e.g. arm) without a docker registry? > > > https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.00-build-docker/docker-build.md > > > > https://gitlab.com/meta-layers/meta-virtualization-tests/-/blob/master/01.01-build-oci/oci-build.md > > I understand now first approach is the way running container on a local PC, > and second one is the way running container on an embedded system. At the moment I perfer to use oci images to run on the embedded system, but you could also run them on your PC if built for x86-64. > I prefer first one because I'd love to test containers on local PC if images > by bitbake is built for embedded system. OK. There might be even more options ;) Like oci, skopeo and podman without container registry, maybe even with docker. Did you try. > > > > The first one to get a container up and running and tested on my PC and the second one to get it on an embedded system. > I misunderstood above means "second one is the way producing container > including image built for embedded system like arm". OK > > > Today I can't try skopeo because of lack of time, > but I remember second approach when running container on an embedded system. Yep. Also there is a way to embed the container in the embedded system without a registry and without skopeo. Stay tuned, since Bruce is currently working on something, which should simplify some steps.
(In reply to comment #25) > Do you want some workflow where you create the rootfs, include it in your > Dockerfile, build a container and spin it up without docker hub for testing > on your PC? > > Or do you want some container to end up on your Embedded device (also e.g. > arm) without a docker registry? I'd love first one. > Like oci, skopeo and podman without container registry, maybe even with > docker. > Did you try. I haven't try it yet. > > Today I can't try skopeo because of lack of time, > > but I remember second approach when running container on an embedded system. > > Yep. Also there is a way to embed the container in the embedded system > without a registry and without skopeo. > > Stay tuned, since Bruce is currently working on something, which should > simplify some steps. I'm looking for that! I hope that the way to produce containers is more simple, like only "bitbake ${some_recipe}"
Any updates on this Robert?
Trevor proposed to help with testing (thanks) Another idea is to consolidate the relevant information here and post it on the docs@ mailing list.
(In reply to comment #27) > Any updates on this Robert? I posted a full walkthrough for docker and oci containers. My understanding is, that the OP was able to step through and even pushed some corrections to my typos. What else would you like to see?
@Michael: I am not quite sure what else is needed here, but I am happy to help out with the doc (once I know what you are after).
Bulk move of 5.1 M4 to 5.2 M2 as approved by AlexB.
Bulk move of bugs I am assigned to from Milestone 5.2M4 to 5.3M1.
I've posted documentation for the image-container class here: https://lists.yoctoproject.org/g/docs/topic/patch_v2/116809656 I will close this bug once this patch merges, or if there are any additional suggestions commented here. Antonin
Closing as the patch was merged: https://git.yoctoproject.org/yocto-docs/commit/?id=6ce00e5875eb3469fefd55cc22acaaeaf620053a Antonin