Bug 11277

(In reply to comment #0)

> 
>     Your MACHINE must be either qemux86" or "qemux86-64".

Do you know why's this a dependency?

(In reply to comment #1)
> (In reply to comment #0)
> 
> > 
> >     Your MACHINE must be either qemux86" or "qemux86-64".
> 
> Do you know why's this a dependency?

I suspect the documentation is out-dated or over-simplifies the problem. Right now, those MACHINEs still work better under qemu (network configured correctly thanks to the ip boot parameter and some extra packages which apply that), but we recently also made it possible to run meta-intel MACHINEs.

title is confusing. "enable the use of QEMU" is already
satisfied as CI jobs have completed qemu sessions.
Inside text of Description however there is KVM mentioned.
KVM is one operating mode of QEMU.
So what is actually asked is "enable KVM mode of QEMU"?

Then  about KVM.
In current setup of pre- and post-build jobs,
these all run in docker.
Thats why it seems "no kvm support", although the hosts have KVM support.
But it does not automatically propagate into docker session without
adding some set-up for this,
- we need to create device inside docker
- we need to run docker in --privileged mode

That's probably achievable, but then lets ask, is it necessary
to pile two levels of virtualization for this to work?
Can we run QEMU_KVM session in host, leaving docker out?

This thread starts to touch another topic which came out when we looked
how to run tests in parallel. And making tests running in parallel is harder
if the pos-tests are technically implemented as build steps inside docker.

From architecture POV, test steps should be independent from build steps,
with possibility to run them on some other worker node, in parallel.
But a post-test that runs in same docker session with builder,
is not detachable to be parallel.

(In reply to comment #3)
> So what is actually asked is "enable KVM mode of QEMU"?

Yes.

> Then  about KVM.
> In current setup of pre- and post-build jobs,
> these all run in docker.
> Thats why it seems "no kvm support", although the hosts have KVM support.
> But it does not automatically propagate into docker session without
> adding some set-up for this,
> - we need to create device inside docker
> - we need to run docker in --privileged mode
> 
> That's probably achievable, but then lets ask, is it necessary
> to pile two levels of virtualization for this to work?

Selftests need the ability to invoke qemu. That's because they need to control how the virtual machine is configured, before booting it up.

> Can we run QEMU_KVM session in host, leaving docker out?

No, that won't work for the selftests.

> This thread starts to touch another topic which came out when we looked
> how to run tests in parallel. And making tests running in parallel is harder
> if the pos-tests are technically implemented as build steps inside docker.
> 
> From architecture POV, test steps should be independent from build steps,
> with possibility to run them on some other worker node, in parallel.
> But a post-test that runs in same docker session with builder,
> is not detachable to be parallel.

True in theory, but in practice I expect the performance to be better when sharing the same build directory. That's because the post-selftest benefit from the existing build artifacts, which are exactly where the tests needs them. Doing it on another node and in another environment is both more complicated (need to share at least the sstate) and slower (need to rebuild directories and images from sstate).

Image testing is different because it has more-or-less well-defined input (but even there we have to hack around with non-upstreameable classes to make all required files available), whereas the selftest doesn't.

currently these steps are performed in CI builder, in docker session:
pre-build,build,store-images,post-build
I think it's good idea to open privileges as little as possible,
so can this work:
1. we run 3 steps in regular docker session, as present;
2. we add 2nd docker session in privileged mode, which can start qemu-kvm,
where we will start 4th step: selftests
Note that stopping of 1st docker session will lose runtime context of build session and keep everything which is in file system. Is this a problem?

If above plan is not doable, we have to run the one
docker session with wide privileges.

(In reply to comment #5)
> currently these steps are performed in CI builder, in docker session:
> pre-build,build,store-images,post-build
> I think it's good idea to open privileges as little as possible,
> so can this work:
> 1. we run 3 steps in regular docker session, as present;
> 2. we add 2nd docker session in privileged mode, which can start qemu-kvm,
> where we will start 4th step: selftests
> Note that stopping of 1st docker session will lose runtime context of build
> session and keep everything which is in file system. Is this a problem?

Shouldn't be a problem. However, I wonder why this is so hard. I don't know what --privileged does, so let me ask: is this required? Normally, /dev/kvm is group-owned by kvm. When the user starting docker is in that group, what special privileges does docker need? It doesn't need to grant any additional privileges.

Granting the build job access to KVM doesn't worry me.

I may have been reading some older examples (about --privileged mode required).
Seems it is possible to pass rights to one device only (--device option).
I will try is this enough to get kvm mode going.

refkit PR#111 adds passing of /dev/kvm to docker session,
and I verified with qemu calling code added 
(which I temporarily borrowed from PR#94),
that -enable-kvm is supported.

In addition to that, builder hosts have /dev/kvm set as group+rw
and belong to jenkins group, i.e. the device is usable with
same credentials as are running the processes in docker session.

This was implemented in PR111, but that small commit was then also added to PR#97 which has now been merged, thus marking this as RESOLVED