Bug 10100

Summary: Suggested documentation for bb.utils.contains(), bb.utils.mkdirhier(), and bb.data.inherits_class()
Product: [Documentation] BitBake User Manual Reporter: ulfalizer
Component: bitbake-manualAssignee: Scott Rifenbark <srifenbark>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: bluelightning, joshuagloe, sgw, srifenbark
Version: 2.2   
Target Milestone: 2.2 M4   
Hardware: x86   
OS: Multiple   
Whiteboard: 07 Sept 2016: RESOLVED
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)

Description ulfalizer 2016-08-08 23:04:17 UTC
Hello,

Some commonly-used functions like bb.utils.mkdirhier() are not documented at
the moment. I suggest replacing '3.10. Accessing Datastore Variables Using
Python' with the following:

  3.10. Functions That Can Be Called from Python
  
  This section lists some commonly used functions that BitBake makes available to
  Python functions.
  
  3.10.1. Functions for Accessing Datastore Variables
  
  *Same as before*
  
  3.10.2. Other Functions
  
  Below is a list of some other commonly used functions:
  
  bb.utils.mkdirhier(dir):
    Creates all non-existing directories in the path 'dir', just like 'mkdir -p'.
    Unlike 'os.mkdir()', this function does not complain if 'dir' already exists.
  
  bb.utils.contains(variable, checkvalues, truevalue, falsevalue, d):
    Returns 'truevalue' if 'variable' contains all the values in 'checkvalues',
    and 'falsevalue' otherwise. The value of 'variable' will be split on
    whitespace. 'checkvalues' can be either a string, which will be split on
    whitespace, or some value that can be converted into a set(). 'd' is the
    datastore to use.
  
    For example, if FOO has the value "x y z", then the following call returns
    True:
  
      bb.utils.contains("FOO", "z y u x", True, False, d)
  
  bb.data.inherits_class(klass, d):
    Returns True if the recipe inherits the class 'klass'.


The link at the bottom of '3.4.4. Anonymous Python Functions' will need to be
updated to point to '3.10. Functions That Can Be Called from Python' as well.

Cheers,
Ulf
Comment 1 Paul Eggleton 2016-08-09 21:43:18 UTC
To be honest I worry about manually documenting all of these python functions in the bitbake manual. Where do we stop? Should we be running some kind of automated API documentation tool (e.g. pydoc) that uses documentation comments as input instead?
Comment 2 Scott Rifenbark 2016-08-09 22:03:04 UTC
Adding myself to the CC list.  Setting some work parameters.  Unsure of the Person-Days as it would be a huge task.  I set to "future" as well.

Scott
Comment 3 ulfalizer 2016-08-10 02:22:38 UTC
(In reply to comment #1)
> To be honest I worry about manually documenting all of these python
> functions in the bitbake manual. Where do we stop? Should we be running some
> kind of automated API documentation tool (e.g. pydoc) that uses
> documentation comments as input instead?

I think we could document some common ones without committing to documenting all of them.
Comment 4 ulfalizer 2016-08-10 02:46:31 UTC
(In reply to comment #1)
> To be honest I worry about manually documenting all of these python
> functions in the bitbake manual. Where do we stop? Should we be running some
> kind of automated API documentation tool (e.g. pydoc) that uses
> documentation comments as input instead?

(The reply got submitted prematurely when I edited the title. :)

I think we could document some common ones without committing to documenting all of them. I don't think we need to add all the documentation at once either. These functions are the only ones I wanted to document for now, so don't worry about an incoming flood of documentation patches. :)

For the undocumented functions, we could just point people in the right direction ("...see the source files in bitbake/lib/bb. Many of the functions have docstrings."). Putting generated pydoc for lots of functions directly into the manual might be too spammy.

A good thing about documenting common functions directly in the manual is that it makes it clear that they're part of the "official" API and safe to use. Sometimes it can be difficult to know which parts of Yocto are considered internal and could change at any time. I don't know what the status of all the functions in utils.py is for example.

Cheers,
Ulf
Comment 5 ulfalizer 2016-08-10 02:57:45 UTC
Re. the suggestion above, we could then rewrite

  Below is a list of some other commonly used functions:

to

  Below is a list of some other commonly used functions. You can
  find more functions by looking at the source code of the 'bb'
  module, in bitbake/lib/bb.


Cheers,
Ulf
Comment 6 ulfalizer 2016-08-12 09:58:27 UTC
I think bb.utils.contains() irks me more than bb.utils.mkdirhier() (though I think both are used often enough to warrant documentation in the manual), because bb.utils.contains() shows up all over the place. A quick Google search on "bb.utils.contains" leads to a SO question about where to find documentation, so that suggests some frustration too.

One nit if this is added: bb.utils.contains() ought to go before bb.utils.mkdirhier() to make it alphabetical.

Cheers,
Ulf
Comment 7 ulfalizer 2016-09-02 11:48:03 UTC
Hello,

What do you think of this version? It just points people in the right
direction, without committing to documenting any functions separately in the
manual.

  3.10. Functions That Can Be Called from Python

    BitBake provides many functions that can be called from Python functions. This
    section lists the most commonly used ones, and mentions where to find others.
  
  3.10.1. Functions for Accessing Datastore Variables
  
    *Same as before*
  
  3.10.2. Other Functions

    You can find many other functions that can be called from Python by looking
    at the source code of the 'bb' module, in bitbake/lib/bb. For example,
    bitbake/lib/bb/utils.py includes the commonly used functions
    bb.utils.contains() and bb.utils.mkdirhier(), which come with docstrings.


Cheers,
Ulf
Comment 8 ulfalizer 2016-09-02 11:51:12 UTC
Urr... that should have been 3.11. Stuff has moved down since the bug was submitted.

(The link at the bottom of '3.5.4. Anonymous Python Functions' will need to be updated to point to '3.11. Functions That Can Be Called from Python' too.)
Comment 9 Scott Rifenbark 2016-09-07 14:58:02 UTC
Hi, 

Look at the new 3.11 section.  I think I captured the gist of it - http://www.yoctoproject.org/docs/2.2/bitbake-user-manual/bitbake-user-manual.html#functions-you-can-call-from-within-python.

thanks, 
Scott
Comment 10 ulfalizer 2016-09-07 17:57:46 UTC
Hello,

Looks good to me. Thanks!

Cheers,
Ulf
Comment 11 Scott Rifenbark 2016-09-07 21:05:11 UTC
Thanks Ulf, 

Setting to RESOLVED and putting the doc flag to "done."

Scott