Bug 5811

Summary: the variable history should collect and display the variable value for each modification sequence
Product: [Build System, Metadata & Runtime] Toaster Reporter: Belen Barros Pena <belen.barros.pena>
Component: toasterAssignee: Toaster default assignee <toaster>
Status: ACCEPTED --- QA Contact:
Severity: enhancement    
Priority: Low CC: alexandru.damian, andreea.b.proca, belen.barros.pena, bluelightning, cristiana.voicu, david.reyna, randy.macleod, richard.purdie, stanciux.mihail
Version: 1.9   
Target Milestone: Future   
Hardware: All   
OS: Multiple   
Whiteboard: GUI design available; needs data completion test
OS type for building Yocto: --- Type of Regression: New (Never tested)
Verified: Documentation change: No (bug/feature does not impact docs)
Attachments:
Description Flags
variable_history_with_values.png
none
Design - Enhancements to variable history none

Description Belen Barros Pena 2014-02-11 12:38:05 UTC
Created attachment 1765 [details]
variable_history_with_values.png

Very nicely, bitbake -e will show you the variable value at each step, at least for operations like append and set. Like so:

# $LICENSE_PATH [5 operations]
#   append /home/yocto/poky/meta-openembedded/meta-networking/conf/layer.conf:18
#     "${LAYERDIR}/licenses"
#   set data_smart.py:725 [expandVarref]
#     " /home/yocto/poky/meta-openembedded/meta-networking/licenses"
#   append /home/yocto/poky/meta-openembedded/meta-oe/conf/layer.conf:26
#     "${LAYERDIR}/licenses"
#   set data_smart.py:725 [expandVarref]
#     " /home/yocto/poky/meta-openembedded/meta-networking/licenses /home/yocto/poky/meta-openembedded/meta-oe/licenses"
#   set /home/yocto/poky/meta/conf/documentation.conf:204
#     [doc] "Path to additional licenses used during the build."
# computed:
#   " /home/yocto/poky/meta-openembedded/meta-networking/licenses /home/yocto/poky/meta-openembedded/meta-oe/licenses"
LICENSE_PATH=" /home/yocto/poky/meta-openembedded/meta-networking/licenses /home/yocto/poky/meta-openembedded/meta-oe/licenses"

In order to provide a full variable history within Toaster, without forcing users to check the configuration files, we should store those values in the Toaster database.

Then we can display variable history as shown in the attached file (variable_history_with_values.png)
Comment 1 Alexandru Damian 2014-02-20 11:55:34 UTC
VariableHistory.value field added; needs data completion test.

The values are not coming in,
Comment 2 Alexandru Damian 2014-03-11 12:46:56 UTC
Database schema change merged into origin/master.
Comment 3 Belen Barros Pena 2015-05-22 13:01:17 UTC
I am looking at the output of bitbake -e and comparing it with what's shown in Toaster. For BBFILES, Toaster shows as the last operation a set by 

bitbake/lib/bb/cooker.py 

on line 1617. But that operation does not appear in the output of bitbake -e. 

Why would that be? Should that be in the output and the problem is in the command? Or is Toaster picking up things it shouldn't? In principle, the output of the bitbake command and the information shown in Toaster should be exactly the same. 

Looking at cooker.py, this is the code it refers to:

    def collect_bbfiles(self, config, eventdata):
        """Collect all available .bb build files"""
        masked = 0

        collectlog.debug(1, "collecting .bb files")

        files = (config.getVar( "BBFILES", True) or "").split()
        config.setVar("BBFILES", " ".join(files))

        # Sort files by priority
        files.sort( key=lambda fileitem: self.calc_bbfile_priority(fileitem) )

        if not len(files):
            files = self.get_bbfiles()

        if not len(files):
            collectlog.error("no recipe files to build, check your BBPATH and BBFILES?")
            bb.event.fire(CookerExit(), eventdata)
Comment 4 Richard Purdie 2015-05-22 13:23:27 UTC
(In reply to comment #3)
> I am looking at the output of bitbake -e and comparing it with what's shown
> in Toaster. For BBFILES, Toaster shows as the last operation a set by 
> 
> bitbake/lib/bb/cooker.py 
> 
> on line 1617. But that operation does not appear in the output of bitbake
> -e. 
> 
> Why would that be? Should that be in the output and the problem is in the
> command? Or is Toaster picking up things it shouldn't? In principle, the
> output of the bitbake command and the information shown in Toaster should be
> exactly the same. 

The function you're talking about is collect_bbfiles() and you're comparing this with "bitbake -e". "bitbake -e" shows the base configuration, it doesn't go on to parse or include any recipes. This means that when bitbake -e displays, collect_bbfiles() has never been called, nor would it ever need to be.

When a build occurs, certain things happen on top of the base configuration, such as collect_bbfiles(). If you run a command that needs to do this (such as 'bitbake -e bash', you'll see the BBFILES history will now be different and include this call site.

So its not really a bug, you're just looking at the data store at a different point in bitbake's execution and the contents is therefore slightly different.
Comment 5 Belen Barros Pena 2015-05-22 13:27:24 UTC
> So its not really a bug, you're just looking at the data store at a
> different point in bitbake's execution and the contents is therefore
> slightly different.

Does this mean we should keep the information shown in Toaster as is?
Comment 6 Belen Barros Pena 2015-06-01 17:53:57 UTC
Created attachment 2534 [details]
Design - Enhancements to variable history
Comment 7 Richard Purdie 2015-06-02 14:27:58 UTC
(In reply to comment #5)
> > So its not really a bug, you're just looking at the data store at a
> > different point in bitbake's execution and the contents is therefore
> > slightly different.
> 
> Does this mean we should keep the information shown in Toaster as is?

Sorry about the delayed reply, I missed the question. Yes, I believe the information in Toaster is ok as it stands today.
Comment 8 Belen Barros Pena 2016-04-25 16:00:57 UTC
I am going to tentatively move this to 2.2. I've showed the designs to a few people, and they quite liked it. It would be nice to get it in.
Comment 9 David Reyna 2018-10-04 13:19:22 UTC
Reviewed 10/18