Bug 9596 - Build "minimal" extensible SDK installers instead of "full" ones
Summary: Build "minimal" extensible SDK installers instead of "full" ones
Status: RESOLVED FIXED
Alias: None
Product: AutoBuilder
Classification: Infrastructure
Component: autobuilder (show other bugs)
Version: 2.1
Hardware: x86 Multiple
: Medium+ enhancement
Target Milestone: 2.3 M3
Assignee: Joshua Lock
QA Contact: Francisco Pedraza
URL:
Whiteboard:
Depends on: 9727 10794
Blocks:
  Show dependency tree
 
Reported: 2016-05-05 20:09 UTC by Henry Bruce
Modified: 2017-09-06 23:26 UTC (History)
9 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Henry Bruce 2016-05-05 20:09:45 UTC
Currently "full" eSDk installers are built as can be found at http://downloads.yoctoproject.org/releases/yocto/yocto-2.1/toolchain/x86_64/

These installers do not allow the "extensible" part of eSDK to be exercised so we should build "minimal" installers instead (i.e. SDK_EXT_TYPE = "minimal"). This means we also need to publish matching sstate.
Comment 1 Richard Purdie 2016-05-12 14:35:43 UTC
I'm conscious that if we do this, we need to reference the autobuilder sstate feed to make the eSDK useful. It has implications for the automated QA of the eSDK too.

Paul/Randy: Do we want to do this? Or do we need to build both? Interested in your feedback on what priority this should be.
Comment 2 Randy Witt 2016-05-12 17:17:40 UTC
Jessica was actually talking to me about this yesterday. Since we do have the esdk container now it makes sense to have "something" published, which we do have and I wasn't even aware of that.

I would really like to see the minimal ones published as well. It's a much better user experience from the initial go. And we would also install the packagedata.

But as you said, this requires adding a usable "stable" sstate feed that won't 403/404 if lots of users try to use it at the same time.

Building both would be great, but I would want to ask Michael about infrastructure impacts beforehand.
Comment 3 Joshua Lock 2016-09-15 20:03:14 UTC
Moving to same milestone and owner as the Depends on: 9727
Comment 4 Henry Bruce 2016-09-17 01:22:28 UTC
Please don't let perfect be the enemy of good. Let's get an esdk installer that uses the yoctoproject.org sstate mirror in place and then handle demands on infrastructure resources.

If developers cannot quickly assess the benefits of a correctly configured esdk installer, then they will not transition away from the traditional sdk and our investment in esdk will not see the rewards it deserves.
Comment 5 Joshua Lock 2016-11-22 13:33:22 UTC
This is just a case of setting SDK_EXT_TYPE[1], as Henry indicated, and ensuring the SDK's built by the autobuilder are pointed at the sstate mirrors in #9727.

1. http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#var-SDK_EXT_TYPE
Comment 6 Joshua Lock 2016-11-23 12:33:45 UTC
I need to work with Paul to figure out the correct mechanism to set an SSTATE_MIRRORS variable for use by the eSDK that won't force the variable to be used during the build.
Comment 7 Joshua Lock 2016-11-23 21:06:19 UTC
We can create an sdk-extra.conf file with the variables we only want to be set in the SDK, i.e. SSTATE_MIRRORS
Comment 8 Joshua Lock 2016-11-24 17:36:08 UTC
I have a patch in testing now.
Comment 10 Joshua Lock 2016-12-12 21:35:41 UTC
Re-opening.

The minimal eSDK results in sanity test failures. The tests need updating to install required components. Until those updates are made, some time after M1, we need to disable the minimal eSDK generation on the autobuilder.
Comment 11 Henry Bruce 2016-12-12 21:46:10 UTC
Can you share the URL of the eSDK install so I can also test?
Comment 12 Joshua Lock 2016-12-13 10:42:12 UTC
(In reply to comment #11)
> Can you share the URL of the eSDK install so I can also test?

Sure:

http://autobuilder.yoctoproject.org/pub/nightly/20161211-1/toolchain/x86_64/

This is a transient URL, so don't be surprised if/when it disappears.

The minimal eSDK installers are significantly smaller (i.e. ~25M vs ~2.8G for poky-glibc-x86_64-core-image-sato-core2-64-toolchain-ext-2.2.sh) but don't include a toolchain, we have a change queued to enable inclusion of the toolchain when we re-enable minimal eSDK installers:

http://git.yoctoproject.org/clean/cgit.cgi/yocto-autobuilder/commit/?id=0d2ff70f0747ffb7fb9e006cb2ad41fd6cc5390c
Comment 13 Joshua Lock 2017-01-26 11:35:55 UTC
Dependency bugs haven't been resolved and M2 rc1 has been built, moving this to M3.
Comment 14 Francisco Pedraza 2017-02-01 02:14:42 UTC
(In reply to comment #13)
> Dependency bugs haven't been resolved and M2 rc1 has been built, moving this
> to M3.

Hi Joshua.
The fix was already commited: http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=aca3afcb592ef4ff070760ab2aafba03152a57fb

also https://bugzilla.yoctoproject.org/show_bug.cgi?id=10794 is marked as resolved now.
Regards
Comment 15 Joshua Lock 2017-02-17 15:52:40 UTC
We're all ready to go on this, we just need to be able to detect the change in selftest behaviour that enabled using it with a minimal eSDK.

I've proposed a layer version bump for that https://patchwork.openembedded.org/patch/137291/

Once that's merged I can push the patch for yocto-autobuilder and we can start building minimal eSDK.
Comment 17 Henry Bruce 2017-09-06 23:26:47 UTC
I just took a look at http://downloads.yoctoproject.org/releases/yocto/yocto-2.3.1/toolchain/x86_64/ and don't see any minimal eSDK installers. Unless minimal install size is now over 200MB.