The 'build appliance' virtual machine portion of a release should have access to an sstate cache feed URI, by default, for building for any of the supported hardware platforms for the release. This will allow the end user to focus on things such as playing / learning the system / target hardware and not spend a lot of time building the software (which is even more expense inside of a typical end-user VM environment).
I am marking this as a low since we already have SSTATE_MIRROR support available, adding sstate-cache would explode the size of the image to beyond reason. The decision to keep it small and portable was a design point. A vendor could create the disk image (or a second disk image) containing what ever sub-set of downloads, sstate-cache and core metadata.
Note that I'm not saying the sstate-cache needs to be included within the initial download. I'm saying a feed should be published that matches the release, and the feed location and how to configure / use should be a supported and documented part of the release.
This is distro / release specific and something that can be addressed directly by the vendor, I think that documenting how to do this should be sufficent as all the bits are available. Rough outline: Setup SSTATE_MIRROR server at vendor/distro maintainer website Build a custom self-hosted image that sets the SSTATE_MIRROR in a bbappend file
Note that my intent here is not that this be a vendor-provided thing but part of the yocto out of box experience.
Product/Component don't seem right for this one. Moving this to Documentation.
Based on Saul's comment yes, Docs. Based on my intent, no. But documenting is better than nothing at least :)
Saul, I need more explanation on your proposed outline for documenting this. Is this just a matter of putting some MIRROR statements in the local.conf file and then building the self-hosted-image? Here is your comment about the rough outline.... Rough outline: Setup SSTATE_MIRROR server at vendor/distro maintainer website Build a custom self-hosted image that sets the SSTATE_MIRROR in a bbappend file
Hi, Is there any update regarding this?
Seems like all we need to do here is publish some URI stuff in the Build Appliance documentation page and instruct the user to apply it if they want to use the SSTATE mirrors. Is that right? If so, all I need is the URI and I can appropriately include it in the Build Appliance documentation page. Scott
My initial argument was that in addition to documenting how to enable this, populating and providing a mirror. This will allow new users (that are in an artifically slow environment) to be able to grab a yocto release and experiment more easily.
Saul, Can we address this in the Build Appliance page of the website? The documentation part at least? If we agree that an sstate cache feed is needed then that should be a different bug right? Scott
Saul to provide Scott with info...
As for documentation. The Build Appliance can use a Shared State mirror if there is one available either in the local environment or provided by a Distribution Vendor. To set up a Shared State mirror, one would got Settings and select the Shared state tab, from there a mirror site can be added by selecting the URL of the Standard mirror and adding the URL. At this time the Yocto Project does not provide any any Shared State Mirrors.
Scott, let me know if you need more info that what's in this but, at this time the Yocto Projec is not likely going to provide that sstate cache mirror that Tom is looking for, once documented, this can be closed.
Changed to ACCEPTED and pushed the date to 1.6.
Right now we have the BA documentation here: https://www.yoctoproject.org/documentation/build-appliance-manual. If you look at it, you will see a bulleted list near the top under the heading "Understanding and Using the Build Appliance." Directly below that list is a paragraph that provides a bit on performance in general when using the BA. I would propose that we add a new paragraph directly beneath that one that would contain Saul's suggested text from his Comment 13. With a bit of re-writing, it would appear as follows: "....realize that the fastest build times occur when running the Yocto project on a system natively running Linux. However, because of the steps required to install a particular version of Linux and any required packages, the Build Appliance offers a quicker way for starting your first successful build. Although at this time, the Yocto Project does not provide any Shared State Mirrors, the Build Appliance can use a Shared State Mirror if one is available either in the local environment or is provided by a Distribution Vendor. To set up a Shared State Mirror, open the "Setting" tab in the Build Appliance and select "Shared State" tab. From there, select the URL of the Standard Mirror and add the URL to add a mirror site." Note that I am not 100% sure on the exact procedure here and am doing a bit of guessing using Saul's text. So, someone needs to verify this. I am moving the bug into IN PROGRESS DESIGN for now. Once we are happy with the solution, we can implement it on the website and move it into a review phase. Thanks, Scott
Enhancement - moving to 1.7
Hi, I never really got anyone to say it was okay to implement my solution for this on the BA web page that has the documentation. So, I added it. You can see it here: https://www.yoctoproject.org/documentation/build-appliance-manual. It is after the initial list right before the "Setting Up and Running the Build Appliance for VMware" section. I am changing the state to IN PROGRESS REVIEW. If there are issues then let me know. Thanks, Scott
The instructions you've added look good, yes. But the real point of this bug is to provide the sstate cache so that first impression / out of box experiences are much quicker and to avoid the "I tried things out on my old test system and it took 3 days, is this right?" problem that always pops up at BoFs (and so must happen a lot more often in places we don't see).
Hi, Since this bug is against the documentation, and it seems like that has settled out for this. Maybe a new bug can be filed to address the other point. I would like to close this for the docs as it seems there is no more work for that part. Scott
(In reply to comment #20) > Hi, > > Since this bug is against the documentation, and it seems like that has > settled out for this. Maybe a new bug can be filed to address the other > point. I would like to close this for the docs as it seems there is no more > work for that part. > > Scott I originally filed this for the "lets provide the binaries" side of the issue. If someone else wants to file a new issue that's clearly just for the "provide the binaries" part, OK.
(In reply to comment #18) > Hi, > > I never really got anyone to say it was okay to implement my solution for > this on the BA web page that has the documentation. So, I added it. You > can see it here: > https://www.yoctoproject.org/documentation/build-appliance-manual. It is > after the initial list right before the "Setting Up and Running the Build > Appliance for VMware" section. > > I am changing the state to IN PROGRESS REVIEW. > > If there are issues then let me know. > > Thanks, > Scott Hi, I did some tests with BA. The sstate mirrors works fine in it. Here are detailed steps: 1. Create a sstate repo via http in host machine. I use the following way: a). Build an image in BA. b). Copy the sstate-cache directory to the host. c). Create the sstate repo via http. Make sure you can access it in browser. For example: http://128.224.162.103/sstate/ 2. Startup a new BA. Add the sstate mirror. a). Click Settings -> Shared state b). Add the URL "http://128.224.162.103/sstate/PATH;downloadfilename=PATH" in shared state mirrors. c). Click Save button (There is an minor issue with adding SSTATE_MIRRORS setting in hob. I already filed a bug 6600 to report it) 3. Build the image in BA. The bitbake log shows the sstate mirror works. And it really can save lots of building time. ############################# NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Running setscene task 2 of 721 (/home/builder/poky/meta/recipes-core/busybox/busybox_1.22.1.bb, do_populate_sysroot_setscene) NOTE: Running setscene task 3 of 721 (/home/builder/poky/meta/recipes-core/busybox/busybox_1.22.1.bb, do_populate_lic_setscene) NOTE: recipe busybox-1.22.1-r32: task do_populate_lic_setscene: Succeeded NOTE: Running setscene task 4 of 721 (/home/builder/poky/meta/recipes-core/busybox/busybox_1.22.1.bb, do_package_qa_setscene) NOTE: recipe busybox-1.22.1-r32: task do_populate_sysroot_setscene: Succeeded NOTE: Running setscene task 5 of 721 (/home/builder/poky/meta/recipes-core/busybox/busybox_1.22.1.bb, do_package_write_ipk_setscene) NOTE: recipe busybox-1.22.1-r32: task do_package_qa_setscene: Succeeded NOTE: Running setscene task 6 of 721 (/home/builder/poky/meta/recipes-core/sysvinit/sysvinit_2.88dsf.bb, do_populate_sysroot_setscene) NOTE: recipe sysvinit-2.88dsf-r14: task do_populate_sysroot_setscene: Succeeded NOTE: Running setscene task 7 of 721 (/home/builder/poky/meta/recipes-core/sysvinit/sysvinit_2.88dsf.bb, do_populate_lic_setscene) NOTE: recipe busybox-1.22.1-r32: task do_package_write_ipk_setscene: Succeeded ########################
Hi, I expanded the discussion on using sstate mirrors based on comment 22. Can you review this new paragraph and be sure that it is technically correct? One thing I don't know how to do here is remove the hyperlinks that are automatically being generated for the example URLs used in the description. If someone knows how to do that let me know and I will get them out. Right now if you did click on one, the browser just spins for a while and returns back to the page (Firefox anyway). Thanks, Scott
Hi Scott, The instructions for using sstate-cache mirrors in Build Appliance are correct and we must take into account the fact that this is one example on how to use it. I am referring here mainly on how to initially produce the sstate cache. More advanced users of YP will be able to come up with their own way of initial population of a sstate cache mirror. Only a typo found in the webpage: "Start up a new Build Appliance and and the Shared State Mirror." notice "and and" repetition. Otherwise, I do agree with your documentation changes as being appropriate. In regard to the un-needed hyperlinks, maybe the yoctoproject.org website admin can help you get rid of. /Cristian
Given the last comment from Tom Rini: https://bugzilla.yoctoproject.org/show_bug.cgi?id=2112#c21 I think we can close this bug as resolved with resolution being fixed. /Cristian
Hi, I fixed the typo as noted in Comment 25. I am marking this bug RESOLVED now and setting the documentation flag to "Done." Thanks, Scott