Bug 10596

At OEDEM, Jan-Simon Möller <dl9pf@gmx.de> volunteered to work on the base script that sstate tests would then hook into. The agreement was that I would help find someone to write the sstate tests. I therefore put this as M2 to give Jan-Simon time to do that. This work needs to build off what he does.

I started with the script and the basic requirements listed in the document 
https://github.com/dl9pf/BSP-LAYER_GUIDE

the script is bash right now (bsp-layer-checker).

I'll work on it if I get to it tomorrow to clean it up.

> I'll work on it if I get to it tomorrow to clean it up.

Jan, have you already done the clean-up? Anibal is about to jump into this work so just wondering if your pending commits are now at upstream.

Didn't get to it between the years. Give me the weekend. Otherwise just go ahead, we can collaborate over on github for this.

(In reply to comment #4)
> Didn't get to it between the years. Give me the weekend. Otherwise just go
> ahead, we can collaborate over on github for this.

Hi Jan,

I will start to code DISTRO layer tests on Python, and the plan is to port your shell script for BSP layer tests to Python, the reason is there are some functionality already written in python for make sstate checks.

Thanks for your help,
Anibal

Ok, then go ahead.

I'm working in this branch, currently only contains the detection of layer(s) and it's types.

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=alimon/yp_compatible

Thanks Anibal for copying me on this bug.  I will monitor and wait for some solution and then we can figure where (if needed) to update the documentation.

Scott

*** Bug 5427 has been marked as a duplicate of this bug. ***

A RFC was sent to the ML,

https://lists.yoctoproject.org/pipermail/yocto/2017-February/034489.html

I expect comments to add more tests.

The main script and the basic tests was implemented, merged at,

https://bugzilla.yoctoproject.org/show_bug.cgi?id=11131

A new bug was load to add more tests #11131.

Hi, 

Right now this bug has been set to RESOLVED.  Can someone tell me if there is a documentation component still?  The doc flag is set to "yes" but sometimes when people resolve a bug they forget about that flag.  Do we need to have some documentation changes associated with this bug?

Soctt

(In reply to comment #12)
> Hi, 
> 
> Right now this bug has been set to RESOLVED.  Can someone tell me if there
> is a documentation component still?  The doc flag is set to "yes" but
> sometimes when people resolve a bug they forget about that flag.  Do we need
> to have some documentation changes associated with this bug?
> 
> Soctt

The current functionality is very basic to test if a layer is compatible you need to run:

$ source oe-init-build-env
$ yocto-compat-layer.py LAYER_DIR

The layer is tested and generates a result log in the form of 
yocto-compat-layer-TIMESTAMP.log.

The current implemented tests are in the commit description [1]. 

Richard what you think about add a manual section about this script?

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/scripts/yocto-compat-layer.py?id=93633edcf84dbf5d97a43cb8d1ebff790a6ab334

(In reply to comment #13)
> The current functionality is very basic to test if a layer is compatible you
> need to run:
> 
> $ source oe-init-build-env
> $ yocto-compat-layer.py LAYER_DIR
> 
> The layer is tested and generates a result log in the form of 
> yocto-compat-layer-TIMESTAMP.log.
> 
> The current implemented tests are in the commit description [1]. 
> 
> Richard what you think about add a manual section about this script?
> 
> [1]
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/scripts/yocto-compat-
> layer.py?id=93633edcf84dbf5d97a43cb8d1ebff790a6ab334

YP Compatible is important and I believe we need to document:

a) What it is
b) What being YP Compatible requires
c) How you can use the script to test this

so you cover c) above but there some other pieces of this we also need to document.

Anibal, 

Can you address points "a" and "b" from Richard's Comment 14?

Thanks

(In reply to comment #15)
> Anibal, 
> 
> Can you address points "a" and "b" from Richard's Comment 14?
> 
> Thanks

a) point we need to explain that the tool helps the users to validate the alignment with Yocto project compatible programee [1], i don't know if we have documented this into the manual.

b) The requirements to be Yocto Project compatible also are described in the registration form [2] like have a README file, the tool now has some tests implemented in the tool here a description in the commit message [2], the tests are divided into COMMON, BSD and DISTRO. For example if a layer is a DISTRO one needs to pass COMMON + DISTRO tests, if it is a BSP layer needs to pass COMMON + BSP otherwise only COMMON.

Feel free to ask more questions,

Anibal

[1] https://www.yoctoproject.org/webform/yocto-project-compatible-registration
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/scripts/yocto-compat-layer.py?id=93633edcf84dbf5d97a43cb8d1ebff790a6ab334

Anibal, 

The compliance worksheet (https://www.yoctoproject.org/webform/yocto-project-compatible-registration)... is that implemented?  If not, where exactly will it reside in the yoctoproject website?  What are the menu selections I use to get to it?

Anibal, 

this issue also transcends BSP layers correct?  It can be for any layer a person wants to use with YP correct?

Scott

Moving target to 2.3 M4 for the documentation component.

Richard and Anibal, 

I am thinking of adding a new section right above this one - http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#enabling-your-layer - in the dev-manual to address this layer compatibility issue.  I would title it something like "Making sure your layers are compatible".  I would introduce the concept, the reasoning, the script, and provide an example.

Scott

(In reply to comment #18)
> Anibal, 
> 
> this issue also transcends BSP layers correct?  It can be for any layer a
> person wants to use with YP correct?

That's right,

Anibal

> 
> Scott

(In reply to comment #17)
> Anibal, 
> 
> The compliance worksheet
> (https://www.yoctoproject.org/webform/yocto-project-compatible-registration).
> .. is that implemented?  If not, where exactly will it reside in the
> yoctoproject website?  What are the menu selections I use to get to it?

Let me review the registration form to make a mapping in the tests and the points on the form.

Anibal

(In reply to comment #22)
> (In reply to comment #17)
> > Anibal, 
> > 
> > The compliance worksheet
> > (https://www.yoctoproject.org/webform/yocto-project-compatible-registration).
> > .. is that implemented?  If not, where exactly will it reside in the
> > yoctoproject website?  What are the menu selections I use to get to it?
> 
> Let me review the registration form to make a mapping in the tests and the
> points on the form.

Isn't the form out-dated, i.e. it needs to be updated to work for "Yocto Compatibility 2.0"?

I also need to know how to find the form itself from the YP website.  Currently, I don't know how to find it.

(In reply to comment #24)
> I also need to know how to find the form itself from the YP website. 
> Currently, I don't know how to find it.

Yes there is a problem in the navigation in the website :/.

(In reply to comment #23)
> (In reply to comment #22)
> > (In reply to comment #17)
> > > Anibal, 
> > > 
> > > The compliance worksheet
> > > (https://www.yoctoproject.org/webform/yocto-project-compatible-registration).
> > > .. is that implemented?  If not, where exactly will it reside in the
> > > yoctoproject website?  What are the menu selections I use to get to it?
> > 
> > Let me review the registration form to make a mapping in the tests and the
> > points on the form.
> 
> Isn't the form out-dated, i.e. it needs to be updated to work for "Yocto
> Compatibility 2.0"?

As i know the form is on usage but yes it needs to be updated to YP Compatible 2.0, anyway the basic test cases were based on that form.

(In reply to comment #26)
> (In reply to comment #23)
> > (In reply to comment #22)
> > > (In reply to comment #17)
> > > > Anibal, 
> > > > 
> > > > The compliance worksheet
> > > > (https://www.yoctoproject.org/webform/yocto-project-compatible-registration).
> > > > .. is that implemented?  If not, where exactly will it reside in the
> > > > yoctoproject website?  What are the menu selections I use to get to it?
> > > 
> > > Let me review the registration form to make a mapping in the tests and the
> > > points on the form.
> > 
> > Isn't the form out-dated, i.e. it needs to be updated to work for "Yocto
> > Compatibility 2.0"?
> 
> As i know the form is on usage but yes it needs to be updated to YP
> Compatible 2.0, anyway the basic test cases were based on that form.

The current tests covers next points partially see after :,

- Do all layers contain a README file which details the origin of the layer, its maintainer, where to submit changes, and any dependencies or version requirements?: ONLY search if README exists.

- Do all layers build without errors against OpenEmbedded-Core with only the dependencies/requirements listed in their README file?: The bitbake environment (bitbake -e) and parsing (bitbake -p world) is executed but not a full build.

The next points are covered fully:

- Are hardware support, configuration (distro) policy, and recipe metadata separated into different layers which do not depend on each other?: 

  -- That's implemented when discover if a layer is BSP, DISTRO or SOFTWARE, isn't  allow to be more than one profile.
  -- The other implemented functionality is to check if a BSP or a DISTRO layer
     doesn't change signatures of software recipes because isn't allowed to do 
     that only adding the layer.

Anibal, 

So Comment 27 relates to the form on the website correct?  

We will need a real link that I can put in the docs when that form is accessible and correct.  For now, I am going to use the link you have in the comments here for this bug.  

I am moving forward with the proposed new section as I suggested in Comment 20.

Scott

We have a small issue in that the new website will cover Yocto Project Compatible v2 and we don't have the new website yet. We do have an idea what YP Compat 2 consists of though and we're preparing the script now. Therefore there isn't a link, yet.

Scott, I propose something like the following section in the location you mention:

Yocto Project Layer Compatibility Programme
===========================================

The Yocto project runs a programme which encourages layers to conform to certain standards and best practises. You can put anything in a layer but there are some best practises which if followed, maximise on the chances that your layer will interact well with other layers.

The first version, 1.0 of the programme has been running for a number of years. There is a form on the Yocto Project website which you can fill in to request permission to use the Yocto Project Compatible logo for your layer and application is open to Yocto Project members and non-commercial open source projects. You would need to be able to answer "Yes" to the questions, or have a good explanation for why you couldn't answer yes.

The second version, 2.0 of the programme is also in development and will include some tighter constraints based on experience of how layers have worked in the real world and where pitfalls have been found. A change is that starting with 2.0, a script is available to test some components of layer compatibility. As part of the submission process, people will be required to confirm that they have run the script against their layer and that it passed the tests. Whilst YP Compat 2.0 has not officially launched yet, the script is available in OE-Core and can be used to assess the status of layers in advance of this.

Hi, 

I have created some stuff in the dev-manual to deal with this new script and the evolving YP Compatibility Program situation.  See these areas and provide comments and feedback.

 * A new step 5 in this list that generalizes how to create a layer.  See http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#creating-your-own-layer.

 * The first bullet in this list of other recommendations.  See http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#other-recommendations

 * The new section on YP Compatibility.  See http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#making-sure-your-layer-is-compatible-with-yocto-project.  Note that for the bsp.test_bsp_no_set_machine and distro.test_distro_no_set_distro tests the description as written in the commit did not make sense to me.  I could use some clarification on them both.

Thanks, 
Scott