Bug 9612

Summary: functions used in recipes and recipe examples (like bb.utils.contains) not documented.
Product: [Documentation] Mega Manual Reporter: brian avery <brian.avery>
Component: mega-manualAssignee: Antonin Godard <antonin.godard>
Status: IN PROGRESS REVIEW --- QA Contact:
Severity: enhancement    
Priority: Medium CC: bluelightning, mathieu.dubois-briand, michael.opdenacker, mshah, randy.macleod, richard.purdie, sgw, srifenbark, tim.orling
Version: 2.2   
Target Milestone: 6.2 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)
Bug Depends on: 9725    
Bug Blocks:    

Description brian avery 2016-05-09 23:17:13 UTC
Some useful functions like  bb.utils.contains are used in recipes, and appear in the mega-manual in a number of examples in that manual.  Unfortunately the methods for the class and what the args are are not documented afaik.
What we say:
---
26.148. utils.bbclass¶

The utils class provides some useful Python functions that are typically used in inline Python expressions (e.g. ${@...}). One example use is for bb.utils.contains().

This class is enabled by default because it is inherited by the base class.
---
Seems like these should be documented especially if we are using them as examples in the manual.
Comment 1 Paul Eggleton 2016-05-12 04:01:37 UTC
Looks like what we do have is a bit confused anyway. bb.utils is not the same thing as utils.bbclass (though there is some functional overlap).
Comment 3 brian avery 2016-05-20 17:21:39 UTC
We should probably document all the functions used outside of the utils file including arguments and return values.  For ones commonly used in (contains, for instance) meta/bbclass files we should collocate the example with the definition of the function and arguments.
Comment 4 Scott Rifenbark 2016-06-06 21:55:50 UTC
So we want to document all functions in the class?  I am not sure what you mean by document "all functions outside of the utils file".
Comment 5 brian avery 2016-06-07 19:51:04 UTC
What I meant by "used outside" of the utils file is that some of the functions in the utils file are class functions not meant to be used in recipes/classes.  The other functions, which amount to the utils API, are.  The API functions used in the outside recipes or bbclass files should be documented.

Functions like:
contains
lockfile
unlockfile
prunedir
explode_dep_versions2
join_deps
explode_deps
mkdirheir
copyfile

The above were from a quick grep/scan script and are probably not complete...
Comment 6 Paul Eggleton 2016-06-07 20:57:08 UTC
If any of the functions we want documented don't have docstrings in the source then we need to fix that before sending over to Scott, just so he has something to work with.
Comment 7 Joshua Lock 2016-06-07 21:01:57 UTC
Of Brian's list the only one missing a docstring is prunedir, though that does have a non-docstring comment explaining what it does.
Comment 8 Joshua Lock 2016-06-07 21:06:59 UTC
(In reply to comment #7)
> Of Brian's list the only one missing a docstring is prunedir, though that
> does have a non-docstring comment explaining what it does.

That's not true, explode_versions and explode_dep_versions don't have docstrings or comments.

I shall write a patch for bitbake/lib/bb/utils.py
Comment 9 brian avery 2016-06-07 21:28:17 UTC
see bug 9725 for the docstring bug. i put a list of fxns in it and made it block this bug.
Comment 10 Scott Rifenbark 2016-06-09 16:39:21 UTC
So for documenting these functions, are we suggesting an entire new section in the ref-manual similar to how we document classes and tasks?  Right now we have long lists of these at http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#ref-classes and http://www.yoctoproject.org/docs/2.2/ref-manual/ref-manual.html#ref-tasks, respectively.  

If so, I imagine that this would create a pretty big list of functions to document.  Am I right in understanding the bigger picture here for this documentation bug?

Scott
Comment 11 brian avery 2016-06-09 18:19:05 UTC
from my perspective, yes. I'd add a new section Library funxtions (utils are in bitbake/lib/... ) and document each function that is intended for external use.
Paul, concur?
Comment 12 Paul Eggleton 2016-06-09 21:25:40 UTC
Yes, that would make sense to me also.
Comment 13 Scott Rifenbark 2016-06-14 20:37:18 UTC
Okay... who can give me the list of functions we need?  Or, can I generate that list somehow?

Thanks,
Scott
Comment 14 Joshua Lock 2016-06-15 13:29:55 UTC
Hi Scott,

(In reply to comment #13)
> Okay... who can give me the list of functions we need?  Or, can I generate
> that list somehow?

I have the AR to go and document the functions in the code, which is being tracked in the "Depends on" bug #9725:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=9725

When that work is done we should have both the list of functions that need documenting and comments in the code which can form the basis of some manual content.
Comment 15 Scott Rifenbark 2017-04-13 20:49:53 UTC
Joshua, 

Can you push this to post 2.3?  We can't get this done in time probably.  I would change it but I don't own it at the moment :)

thanks 
Scott
Comment 16 Randy MacLeod 2020-04-09 15:18:14 UTC
Robert, would you like to work on this? We think it's important to have it documented.
Comment 17 Robert P. J. Day 2020-05-25 15:19:16 UTC
OK, I'll take a look.
Comment 18 mshah 2021-01-11 13:05:46 UTC
Hi All,

I fixed Bug 9725 and one can have a look at the changes from the link mentioned in the comments of Bug 9725.

Is there any work needed for this bug now? If Yes then please let me know.

Thanks,
Milan Shah
Comment 19 Richard Purdie 2021-01-13 23:30:09 UTC
(In reply to comment #18)
> Hi All,
> 
> I fixed Bug 9725 and one can have a look at the changes from the link
> mentioned in the comments of Bug 9725.
> 
> Is there any work needed for this bug now? If Yes then please let me know.
> 
> Thanks,
> Milan Shah

For this bug, the work is to add a new "library functions" section to the manual in the reference manual in the yocto-docs repository and document the utils functions to start with.
Comment 20 mshah 2021-03-09 13:09:39 UTC
(In reply to comment #19)
> For this bug, the work is to add a new "library functions" section to the
> manual in the reference manual in the yocto-docs repository and document the
> utils functions to start with.

Hi Richard,
Does it means that I need to create a new file under ref-manual with library functions.rst name and write function details there?
If yes then can you share a template to follow or show me a sample to start with?
Comment 21 mshah 2021-03-09 13:47:54 UTC
(In reply to comment #19)
> 
> For this bug, the work is to add a new "library functions" section to the
> manual in the reference manual in the yocto-docs repository and document the
> utils functions to start with.

Hi Richard,

I'm making a list of all functions to add to the document. I'm concerned about the ordering of these functions.

Should I order them alphabetically? 

If yes then there are some functions that start with an underscore ( _ ), where should I put them?
Comment 22 mshah 2021-03-12 12:39:24 UTC
Patch is sent for review.

View/Reply Online (#899): https://lists.yoctoproject.org/g/docs/message/899
Comment 23 mshah 2021-03-15 06:05:03 UTC
New Patch is sent
-----------------
View/Reply Online (#903): https://lists.yoctoproject.org/g/docs/message/903
Comment 24 mshah 2021-05-19 05:14:08 UTC
Do we have any update on patch review?

Regards
Milan Shah
Comment 25 Randy MacLeod 2021-05-20 01:14:01 UTC
Pinged the v2 thread and added michael.opdenacker@bootlin.com
Comment 26 Randy MacLeod 2021-11-11 15:59:27 UTC
Michael, can you followup on this please?
Comment 27 Randy MacLeod 2024-10-24 15:03:54 UTC
Bulk move of 5.1 M4 to 5.2 M2 as approved by AlexB.
Comment 28 Randy MacLeod 2025-02-20 16:21:11 UTC
Antonin,
There is a patch on the list as mentioned in the bug or comments.
Comment 29 Antonin Godard 2025-04-22 08:55:20 UTC
Patches sent on bitbake list to auto document patches from the Python docstrings:

https://lists.openembedded.org/g/bitbake-devel/topic/patch_0_4_document_library/112332815

Moving to In Progress Review.
Comment 30 Antonin Godard 2025-06-06 07:45:01 UTC
Bulk move of bugs I am assigned to from Milestone 5.2 to 5.3M1.
Comment 31 Mathieu Dubois-Briand 2026-08-28 08:02:02 UTC
I can see patch merged: https://git.openembedded.org/bitbake/commit/?id=0a711949acc2696f32a61c591ee3ea37041acb91

Antonin, do we need more or should we close this bug entry?
Comment 32 Antonin Godard 2026-08-28 09:01:02 UTC
(In reply to Mathieu Dubois-Briand from comment #31)
> I can see patch merged:
> https://git.openembedded.org/bitbake/commit/
> ?id=0a711949acc2696f32a61c591ee3ea37041acb91
> 
> Antonin, do we need more or should we close this bug entry?

It has been on my mind to try and to the same of `oe.utils.*` function and incorporate that to yocto-docs somehow, which is probably why I haven't closed this bug yet.