Bug 12128

Summary: Rework the Quickstart and intro to the manuals
Product: [Documentation] Quick Start Reporter: Richard Purdie <richard.purdie>
Component: quick-startAssignee: Stephano Cetola <stephano>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: jefro, kristi, poky.doc.watcher, poky.watcher, robin.l.jordan, sgw, srifenbark, stephano, tracey.m.erway
Version: unspecified   
Target Milestone: 2.5 M3   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Richard Purdie 2017-09-26 15:25:48 UTC
Our quickstart as our lead into the documentation has served the project well but I'd like to propose a step back and for us to consider what we really need here now.

The main issue is that not everyone wants to build an image and once they have, they're kind of hitting a wall, not knowing what to do next.

What we therefore need is something like:

a) Description of umbrella nature of YP and the fact it has many components. One is a build system, another is a reference/test distro. Also tools to support these.

b) Intro to layer model - stack of layers to support their hardware, software and allow them to add their own customisations

c) Reference build as an example (current quickstart)

d) "Cross Roads Pointer" guide

Three pieces. Firstly a "what resources are there?":

Quick pointers to Wiki, Bugzill, Mailing lists, manuals

Secondly, now you understand the basics, point at several howto/worked examples:

 i) Adding a layer for hardware support
 ii) Adding a layer for softare
 iii) Writing your own recipe
 iv) Creating your own layer for your customisations
 v) Adding your own custom kernel
 vi) CHanging the default kernel defconfig (for linux-yocto and otherwise)
 vii) Submitting changes

Thirdly, "Beyond Builds":

 Layer Index
 Yocto Autobuilder
 Yocto Project Compatibility
 Auto upgrade tools
 Patchtest/Patchwork
 Pseudo
 Opkg
 [add other YP components here]


I'm trying to sketch a rough outline here which we can then build into something Scott can pull together for the rework...
Comment 1 Scott Rifenbark 2017-09-27 23:25:27 UTC
Just to be clear here... you mean that the "cross-roads pointer guide" part consists of the three pieces mentioned in your comment.
Comment 2 Stephen K Jolley 2017-09-28 14:45:07 UTC
Provide Scott with the information he needs.
Comment 3 Saul Wold 2017-11-14 16:58:13 UTC
Scott, correct, (In reply to comment #1)
> Just to be clear here... you mean that the "cross-roads pointer guide" part
> consists of the three pieces mentioned in your comment.

Right, the cross-roads pointer guide has the Pointer, Basics and Beyond Basics.

As RP also points out there is some work initial ground work which is laid out in his a,b,c,d points before getting into the cross-roads pointer guide.
Comment 4 Scott Rifenbark 2017-11-21 22:39:45 UTC
Here is an update on the doc effort here.  Right now, the QS really needs to be concerned with a brief introduction to YP and what the QS is about.  Then, it simply needs to concern itself with setting up a build host and providing a quick experience in building out a couple of images (one for QEMU and one for hardware).  Finally, we can have that "signposts" section that Richard alludes to in the original bugs comment. 

The information on YP as an umbrella and the layer stuff is going to be in a new document that is going to be covering YP concepts.  See Bug 12370 for the bug on the YP Concepts Manual.
Comment 5 Scott Rifenbark 2017-11-28 15:36:58 UTC
Hi, 

I have placed a reviewable QS at http://www.yoctoproject.org/docs/scott_temp/yocto-project-qs/yocto-project-qs.html.

I need some more information on this and some reviews.  The structure is now as suggested by Comment 1 sans the software over stuff as that is going to be in a new manual of YP concepts (see Bug 12370).

Some areas in the reviewable QS are in red... I need information for those.

Thanks, 
Scott
Comment 6 Robin Jordan 2017-11-28 17:40:26 UTC
I’ve reviewed the QS guide.  I am unable to help with the red areas; however, I do have one suggestion.  I like how you refer users to the Yocto Project Overview Manual in the Welcome section:

“For more introductory and conceptual information on the Yocto Project, see the Yocto Project Overview Manual.”

Some users may stop there and not read on once they realize the QS might not be where they want to start.  Would it also make sense to refer to “Section 1.3 Where to Go Next?” in the Welcome section to assure a user makes it all the way down to it?  It’s got good information, but many users might not explore all the way down.  I’m just thinking a sentence to let the user know there are next steps after getting set up.
Comment 7 Scott Rifenbark 2017-11-28 18:09:25 UTC
Thanks Robin.. see http://www.yoctoproject.org/docs/scott_temp/yocto-project-qs/yocto-project-qs.html for an update.

Scott
Comment 8 Scott Rifenbark 2017-12-06 17:50:12 UTC
Hi, 

I needs information for the following sections:

 - http://www.yoctoproject.org/docs/scott_temp/yocto-project-qs/yocto-project-qs.html#qs-guided-examples

 - http://www.yoctoproject.org/docs/scott_temp/yocto-project-qs/yocto-project-qs.html#going-beyond-builds


The areas I need help in use red text. 

Scott
Comment 9 Stephano Cetola 2018-01-10 18:15:56 UTC
I'm going to be working on these changes and will send my rough draft to Richard for review.
Comment 10 Richard Purdie 2018-01-10 22:41:30 UTC
Scott: Its not "Pah-kee", its pronounced more like Pock in "Pocket", or Pocky (Pock-ie).
Comment 11 Richard Purdie 2018-01-10 22:46:39 UTC
Looking at the document Scott, I'll be honest, it doesn't do what I think we need to do at all. Its basically a rehash of the same old quickstart guide. We *need* to get away from that and radically change the approach.

I appreciate the new website has information and there isn't any point in duplicating it. I do strongly believe we need some summary level of the a) and b) sections I mention up front though and links pointing at the new website info so people can understand the terms.

As for c) itself, its far too full of "do this, except if" type conditions with far too many paths. We somehow need to streamline it and push some pieces to different sections. For example "real hardware/minnowboard" should be its own section and building on a basic "build and boot a virtual machine" which the main quick start should focus on.
Comment 12 Scott Rifenbark 2018-01-10 22:55:20 UTC
(In reply to comment #10)
> Scott: Its not "Pah-kee", its pronounced more like Pock in "Pocket", or
> Pocky (Pock-ie).

Ok - I did not realize that distinction. I will say "Pock-ee" since "ie" is phonetically vague.
Comment 13 Scott Rifenbark 2018-01-10 23:16:23 UTC
(In reply to comment #11)
> Looking at the document Scott, I'll be honest, it doesn't do what I think we
> need to do at all. Its basically a rehash of the same old quickstart guide.
> We *need* to get away from that and radically change the approach.
> 
> I appreciate the new website has information and there isn't any point in
> duplicating it. I do strongly believe we need some summary level of the a)
> and b) sections I mention up front though and links pointing at the new
> website info so people can understand the terms.
> 
> As for c) itself, its far too full of "do this, except if" type conditions
> with far too many paths. We somehow need to streamline it and push some
> pieces to different sections. For example "real hardware/minnowboard" should
> be its own section and building on a basic "build and boot a virtual
> machine" which the main quick start should focus on.

Tracey and Jefro have indicated that the information on the new website that covers the software overview and layer stuff will be finished and then moved to the new Yocto Project Overview Manual.  That is why I am referencing that manual here - http://www.yoctoproject.org/docs/2.5/yocto-project-qs/yocto-project-qs.html#welcome.  Once that information is done, I can summarize it and put it in the beginning of the Quick Start.  

Regarding the c), it now consists of setting up the build system and building out two types of images: a QEMU one and an actual one.  I was told that I needed to address a build host that was set up with CROPS and one that was not (standard Linux build host).  That has caused the "do this, except if" stuff you are referring to.  I would much rather have this simply use a standard build Linux build host and put the whole CROPS discussion elsewhere.  

I think the sections on building the images is pretty straightforward.  It has clear steps for the QEMU image and then a section with steps for the image to run on a target.  Should we just present a single image build example?  Say, just the QEMU one?

Isn't http://www.yoctoproject.org/docs/2.5/yoocto-project-qs/yocto-project-qs.html#building-an-image-for-hardware its own section for "real hardware/minnowboard"?

I guess I don't know what you mean by "build and boot a virtual machine"?  Do you mean the image for emulation?
Comment 14 Jeff Osier-Mixon 2018-01-10 23:41:04 UTC
My 2 pence

I'm afraid I don't agree that (a) and (b) need to be represented in this guide, except possibly as a single sentence and a link to the new website and reference the new concepts guide. I definitely appreciate the goal to get users to understand the project, but the answer is not to put them into a technical quick start where the goal is to use the tools for the first time. We can create a larger "getting started" section in the programmer's guide covering all of the aspects of the decision tree.

I believe the quick start should be quick - it should not have separate sections at all. It should literally only cover one path, as its goal is to show the user a YP build by example. The layer model and the umbrella nature of the project will be covered in the concept guide and also on the new website in great detail.

I think the logical path to show is using CROPS to build for QEMU. An argument could be made for doing this as a native build rather than CROPS, although that leaves out users who are stuck on windows or mac.

Otherwise, we eventually lead back to what we have now - a "quick" start that is over 12 pages long and which people use as an incomplete reference manual rather than a guide.
Comment 15 Richard Purdie 2018-01-11 17:22:20 UTC
(In reply to comment #14)
> My 2 pence
> 
> I'm afraid I don't agree that (a) and (b) need to be represented in this
> guide, except possibly as a single sentence and a link to the new website
> and reference the new concepts guide.

You need to think more carefully about this. What you're saying is that:

* the overall components of the project aren't important.

* the way the project is structured (layers) doesn't matter.

I 100% disagree with this. I think if our users have an idea of the components and the importance of layers, their experience of the project is massively improved.

As an example, if from day one, someone creates a layer for any of their customisations as part of the quick start, it encourages them to use the concept. This will then save them a ton of pain down the line.

So I do strongly believe the quick start as it stands is not fit for purpose and we need to move on and do something different and it includes at least explaining the concept of a layer to new users.

I believe all the above *directly* corresponds to the new user feedback that we keep collecting too.

> I definitely appreciate the goal to
> get users to understand the project, but the answer is not to put them into
> a technical quick start where the goal is to use the tools for the first
> time. We can create a larger "getting started" section in the programmer's
> guide covering all of the aspects of the decision tree.

I think a quick start needs to mention key concepts and give people pointers to places to learn more about various topics and this is what I really want us to aim for.

We need a step back, think about the feedback we get from users, then improve the material to incorporate that feedback.

> I believe the quick start should be quick - it should not have separate
> sections at all. It should literally only cover one path, as its goal is to
> show the user a YP build by example.

I do agree on this and its why I feel there are too many divergent paths from it as it stands right now. I also realise users want to go in many different directions with what they do with the project. Two things though, they do need a working understanding of the project components/terminology and they do need to know about layers.

My proposal of the "pointers" at the end is to try and keep the size under control but stop users "falling off a cliff" and allow us to further cover many of the workflows people may have (Where do I put my image customisations, how do I pull in a BSP layer etc).

> The layer model and the umbrella nature
> of the project will be covered in the concept guide and also on the new
> website in great detail.

Which is good and we can point to those for the details but I think mention here is necessary too.
 
> I think the logical path to show is using CROPS to build for QEMU. An
> argument could be made for doing this as a native build rather than CROPS,
> although that leaves out users who are stuck on windows or mac.

I'm still broadly in favour of native build as not showing that suggests there is something "hard" about it we're hiding. We could offer two paths in the quick start. A windows/mac/VM crops one and a native one. Cover the same process in the two environments. I'm leaning in favour of that.

Its less about creating a simple document and more about one that creates a document which fulfils the users needs.

> Otherwise, we eventually lead back to what we have now - a "quick" start
> that is over 12 pages long and which people use as an incomplete reference
> manual rather than a guide.

I would still rather a 12 page guide than 6 pages which leaves users unable to know where to go next. See above, I think we slim down the "core" quick start, pushing some material to follow on "quick" guides to specific tasks (e.g. adding my BSP). If there are two paths in the guide and that lengthens, I think its ok as long as the overall experience improves and is good.
Comment 16 Scott Rifenbark 2018-01-11 18:10:21 UTC
(In reply to comment #15)
> (In reply to comment #14)
> > My 2 pence
> > 
> > I'm afraid I don't agree that (a) and (b) need to be represented in this
> > guide, except possibly as a single sentence and a link to the new website
> > and reference the new concepts guide.
> 
> You need to think more carefully about this. What you're saying is that:
> 
> * the overall components of the project aren't important.
> 
> * the way the project is structured (layers) doesn't matter.
> 
> I 100% disagree with this. I think if our users have an idea of the
> components and the importance of layers, their experience of the project is
> massively improved.
> 
> As an example, if from day one, someone creates a layer for any of their
> customisations as part of the quick start, it encourages them to use the
> concept. This will then save them a ton of pain down the line.
> 
> So I do strongly believe the quick start as it stands is not fit for purpose
> and we need to move on and do something different and it includes at least
> explaining the concept of a layer to new users.
> 
> I believe all the above *directly* corresponds to the new user feedback that
> we keep collecting too.
> 
> > I definitely appreciate the goal to
> > get users to understand the project, but the answer is not to put them into
> > a technical quick start where the goal is to use the tools for the first
> > time. We can create a larger "getting started" section in the programmer's
> > guide covering all of the aspects of the decision tree.
> 
> I think a quick start needs to mention key concepts and give people pointers
> to places to learn more about various topics and this is what I really want
> us to aim for.
> 
> We need a step back, think about the feedback we get from users, then
> improve the material to incorporate that feedback.
> 
> > I believe the quick start should be quick - it should not have separate
> > sections at all. It should literally only cover one path, as its goal is to
> > show the user a YP build by example.
> 
> I do agree on this and its why I feel there are too many divergent paths
> from it as it stands right now. I also realise users want to go in many
> different directions with what they do with the project. Two things though,
> they do need a working understanding of the project components/terminology
> and they do need to know about layers.
> 
> My proposal of the "pointers" at the end is to try and keep the size under
> control but stop users "falling off a cliff" and allow us to further cover
> many of the workflows people may have (Where do I put my image
> customisations, how do I pull in a BSP layer etc).
> 
> > The layer model and the umbrella nature
> > of the project will be covered in the concept guide and also on the new
> > website in great detail.
> 
> Which is good and we can point to those for the details but I think mention
> here is necessary too.
>  
> > I think the logical path to show is using CROPS to build for QEMU. An
> > argument could be made for doing this as a native build rather than CROPS,
> > although that leaves out users who are stuck on windows or mac.
> 
> I'm still broadly in favour of native build as not showing that suggests
> there is something "hard" about it we're hiding. We could offer two paths in
> the quick start. A windows/mac/VM crops one and a native one. Cover the same
> process in the two environments. I'm leaning in favour of that.

Right now, I have a set up section that tells the user how to set up CROPS.  Once CROPS is set up, you are left with a shell terminal that still needs the poky repo cloned.  From that point, it is like you are on a native machine.  I do believe we are covering both scenarios currently. 

> 
> Its less about creating a simple document and more about one that creates a
> document which fulfils the users needs.
> 
> > Otherwise, we eventually lead back to what we have now - a "quick" start
> > that is over 12 pages long and which people use as an incomplete reference
> > manual rather than a guide.
> 
> I would still rather a 12 page guide than 6 pages which leaves users unable
> to know where to go next. See above, I think we slim down the "core" quick
> start, pushing some material to follow on "quick" guides to specific tasks
> (e.g. adding my BSP). If there are two paths in the guide and that
> lengthens, I think its ok as long as the overall experience improves and is
> good.
Comment 17 Richard Purdie 2018-01-21 09:31:38 UTC
Just to complete the loop here, there have been various discussions between the people in question here.

I believe we now all agree that in high level terms:

* The Quickstart should become a "Getting Started" which may be slightly longer
* We should tweak the quickstart flow to include setting up and adding your own local layer for customisations
* Some reference to terminology are important for the new user experience as terminology continues to be something which confuses people.
* The Getting Started Guide would end with a set of "cross road" pointers into the specific task based doucmentation about how users may do various things.
* There may be two variants of the 'quickstart' piece, one for a container based CROPS and one for a native Linux version.
* We need to move lots of the incidental notes from the quickstart to improve the flow. (e.g. real hardware like minnow becomes its own section pointed to by the cross roads pointer, the lists of packages for specific distros should be in the reference manual in sections, we can point to that from the quickstart).

The key thing to keep in mind that whilst the quickstart has served us well for 7-8 years, its time to take a fresh look at it and adapt it to changes in the project like layers.
Comment 18 Scott Rifenbark 2018-01-24 19:15:05 UTC
(In reply to comment #17)
> Just to complete the loop here, there have been various discussions between
> the people in question here.
> 
> I believe we now all agree that in high level terms:
> 
> * The Quickstart should become a "Getting Started" which may be slightly
> longer

How about "Getting Started With Yocto Project" as a new title?

> * We should tweak the quickstart flow to include setting up and adding your
> own local layer for customisations

Get me this information and I will get it in the manual.  Keep in mind we do have a section in the dev-manual called "Understanding and Creating Layers" (http://www.yoctoproject.org/docs/2.5/dev-manual/dev-manual.html#understanding-and-creating-layers).  Is the information different than what is in this section?

> * Some reference to terminology are important for the new user experience as
> terminology continues to be something which confuses people.

I can add a reference-like terminology section at the end of the QS.  Bear in mind we have a large section of terminology definitions in the current ref-manual (http://www.yoctoproject.org/docs/2.5/ref-manual/ref-manual.html#ref-terms).  I see no problems with some key terms being duplicated here if the need for them to be local out weighs the inconvenience of double-documenting them.  Once practice that is done in the YP doc set is to link to a term's definition when it is introduced.

> * The Getting Started Guide would end with a set of "cross road" pointers
> into the specific task based doucmentation about how users may do various
> things.

This section is in place but is missing some of the information we want to link to.  I need people to actually look and see if the list of "other places" is adequate.  I formed the list from what was suggested in Comment 1.  Here are the current sections... http://www.yoctoproject.org/docs/2.5/yocto-project-qs/yocto-project-qs.html#qs-guided-examples and http://www.yoctoproject.org/docs/2.5/yocto-project-qs/yocto-project-qs.html#going-beyond-builds

> * There may be two variants of the 'quickstart' piece, one for a container
> based CROPS and one for a native Linux version.

I need clarification on this... My understanding is that for a container-based set up the only difference is getting CROPS in there.  Presumably, once you have that running, you basically have a native-looking and behaving shell.  Once the shell is available, things become exactly like preparing a native Linux machine.  Are my assumptions wrong here?

> * We need to move lots of the incidental notes from the quickstart to
> improve the flow. (e.g. real hardware like minnow becomes its own section
> pointed to by the cross roads pointer, the lists of packages for specific
> distros should be in the reference manual in sections, we can point to that
> from the quickstart).

If we move the example that builds an image for minnow out of the QS, then are you saying we leave just the example that builds for QEMU?

Also, BTW - we do point to the ref-manual for proper packages needed to set up the host.  The ref-manual has always been the definitive location for that information.  Right now, we duplicate the "Essential" packages in the QS as part of the "setup" section.  So, we are saying here that we don't want that duplication?  It is okay to send the user to another manual for that part of the setup?  But, we can duplicate terms?

> 
> The key thing to keep in mind that whilst the quickstart has served us well
> for 7-8 years, its time to take a fresh look at it and adapt it to changes
> in the project like layers.
Comment 19 Stephano Cetola 2018-04-19 08:12:54 UTC
Documentation has been updated and approved.
Comment 20 Scott Rifenbark 2018-04-19 08:33:57 UTC
Pretty sure we still need to get the following done in the brief-yoctoprojectqs manual:

Scott/Stephano:  Add 3 additional steps to First Build doc
 * Creating layer (bitbake-layers create-layer --help)
 * BSP (adding a vendor BSP, e.g. meta-freescale)
 * Customizing your own image.

During our last meeting, we decided to add these three items to the http://yoctoproject.org/docs/2.5/brief-yoctoprojectqs/brief-yoctoprojectqs.html document.

Scott