Bug 9793

Summary: Toaster container solution should be extended to run on windows (and mac)
Product: [Build System, Metadata & Runtime] Toaster Reporter: brian avery <brian.avery>
Component: toasterAssignee: Randy Witt <randy.e.witt>
Status: RESOLVED FIXED QA Contact: Libertad <libertad.gonzalez.de.la.cruz>
Severity: normal    
Priority: Medium+ CC: belen.barros.pena, jessica.zhang, joshuagloe, randy.e.witt, stanciux.mihail, stephano
Version: 2.2   
Target Milestone: 2.2 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description brian avery 2016-06-16 21:56:10 UTC
the toaster container made for Bug 9555 works on linux but not on windows/mac.

Issues:
1) the 9155 container does it's work/build/tmp in the shared folder intot he container from the host.   Windows and Mac filesystems do not support some features bitbake needs to run like FIFOS, and hard links
2) On Windows, but not the mac, I found that keeping the sstate-cache on the shared host folder resulted in bitbake hanging.

We should put the work/tmp dir on the linux partition and the artifacts/downloads and ideally sstate-cache on the shared host side so they are persistent.

Issue 2 was noted with a std docker toolbox install and the default vm on virtualbox.
Comment 1 brian avery 2016-07-11 17:27:12 UTC
The usersetup has issues if the container has stopped and you need to restart it. This happens when you need to restart docker after a reboot, for instance.

The docker log:


subprocess.CalledProcessError: Command '['sudo', 'restrict_useradd.sh', '1000', 'toasteruser']' returned non-zero exit status 9
useradd: user 'toasteruser' already exists
Traceback (most recent call last):
  File "/usr/bin/usersetup.py", line 51, in <module>
    subprocess.check_call(cmd.split(), stdout=sys.stdout, stderr=sys.stderr)
  File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', 'restrict_useradd.sh', '1000', 'toasteruser']' returned non-zero exit status 9
Comment 2 brian avery 2016-08-11 17:25:39 UTC
you can solve this with a linked persistent data container and a docker compose file.  I've successfully run a complete toaster build on the Mac with this approach.
Comment 3 Joshua Lock 2016-08-12 11:05:54 UTC
Ah-ha, a persistent data container. I'd not heard of it but was surprised that there didn't seem to be something like it.

FYI, for whomever is implementing the solution, docs are: 

https://docs.docker.com/engine/tutorials/dockervolumes/

Do we want to keep each of TMPDIR, SSTATE_DIR and DL_DIR persistent? I think it might make sense to have more than one persistent data store, one for the shareable artefacts (DL_DIR and SSTATE_DIR) and one for the temporary files and build output (TMP_DIR).

If we choose not to persist TMPDIR it may make sense to keep DEPLOY_DIR on a persistent data container.
Comment 4 Randy Witt 2016-08-12 19:00:54 UTC
(In reply to comment #3)
> Ah-ha, a persistent data container. I'd not heard of it but was surprised
> that there didn't seem to be something like it.
> 
> FYI, for whomever is implementing the solution, docs are: 
> 
> https://docs.docker.com/engine/tutorials/dockervolumes/
> 
> Do we want to keep each of TMPDIR, SSTATE_DIR and DL_DIR persistent? I think
> it might make sense to have more than one persistent data store, one for the
> shareable artefacts (DL_DIR and SSTATE_DIR) and one for the temporary files
> and build output (TMP_DIR).
> 
> If we choose not to persist TMPDIR it may make sense to keep DEPLOY_DIR on a
> persistent data container.

Initially the idea is that *everything* is going to be in the same persistent data container. This is so that it can follow the same model of the current container https://github.com/crops/toaster-container.

The idea is to start with the simplicity of everything being in the same data container, and we can extend it later if necessary.
Comment 5 Randy Witt 2016-09-26 22:30:59 UTC
With the instructions at https://github.com/crops/docker-win-mac-docs/wiki this can be considered finished.