Bug 7734

Summary: bitbake-prserv-tool fails to import file
Product: [Build System, Metadata & Runtime] BitBake Reporter: Lucian Musat <georgex.l.musat>
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact: Lucian Musat <georgex.l.musat>
Severity: normal    
Priority: Medium CC: alejandro.hernandez, costin.c.constantin, poky.bs.watcher, poky.watcher, richard.purdie, sgw
Version: 1.7   
Target Milestone: 1.7.3   
Hardware: x86   
OS: x86_64   
Whiteboard:
OS type for building Yocto: --- Type of Regression: Regression (Used to work)
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on: 6752    
Bug Blocks:    

Description Lucian Musat 2015-05-08 14:47:57 UTC
STATUS: FAILED
BUILD: 1.7.2_rc3: 29812e61736a95f1de64b3e9ebbb9c646ebd28dd
ENVIRONMENT: Ubuntu 14.10 x86_64
STEPS TO REPRODUCE:         Test the PR service database import/export feature. Replace the existing database.

        Use the 'm4' recipe for the test. Use the following code to change the recipe's configuration replacing the unique text every time.
"""
do_package_append() {
        bb.build.exec_func('do_test_prserv', d)
}

do_test_prserv() {
       
echo "Unique text here! For example use date and time in seconds."
}
"""

        0) Set up PR service for local use by adding PRSERV_HOST = "localhost:0" in local.conf.

        1) change recipe configuration and build the package
        2) get PR value of package
        3) export the PR database: bitbake-prserv-tool export export.inc
        4) remove the current db
        5) import the PR database" bitbake-prserv-tool import export.inc
        6) change recipe configuration and build the package
        7) get PR value of package
        8) check that PR has increased by 1



EXPECTED OUTCOME: The exported/imported PR database should work as intended and increment the package PR number by 1 compared to the last built package.

ACTUAL OUTCOME: After removing the current db and running "bitbake-prserv-tool import export.inc" the command fails with the error:
ERROR: Only one copy of bitbake should be run against a build directory
Importing from file export.inc failed!

This test failed on the autobuilder (http://yocto-lava.rb.intel.com/dashboard/streams/anonymous/oe-selftest-ubuntu/bundles/aa1bc9f8922ebfd7d34f5e9df928b52bad05f247/22af75ff-b651-47fe-a05c-03d0cb768094/result/38/).
Comment 1 Saul Wold 2015-05-12 14:30:11 UTC
Is this a new test for PR Service? Or an existing test that is now Automated? The failure seems to indicate that bitbake might have still been running, did you verify this failure outside of the lava tools?
Comment 2 Lucian Musat 2015-05-12 15:40:40 UTC
Hi Saul,

This is an old test that was automated some time ago. I tested this manually too. Right now I can't even pass the export step because of that error. I know this is usually because another bitbake is running in the background, but in this case I can't see anything:

lucian@lucian-OptiPlex-9010:~/poky/build$ bitbake-prserv-tool export export.inc
ERROR: Only one copy of bitbake should be run against a build directory
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ ps ax | grep python
 3173 pts/1    S+     0:00 grep --color=auto python
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ ps ax | grep bitbake
 3177 pts/1    S+     0:00 grep --color=auto bitbake
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ ps aux | grep gcc
lucian    3321  0.0  0.0  10964  2304 pts/1    S+   18:31   0:00 grep --color=auto gcc
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ 
lucian@lucian-OptiPlex-9010:~/poky/build$ bitbake m4
NOTE: Started PRServer with DBfile: /home/lucian/poky/build/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 36896, PID: 3339
WARNING: Host distribution "Ubuntu-15.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
Parsing recipes: 100% |##########################################################################################################| Time: 00:00:11
Parsing of 893 .bb files complete (0 cached, 893 parsed). 1289 targets, 40 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION        = "1.24.0"
BUILD_SYS         = "x86_64-linux"
NATIVELSBSTRING   = "Ubuntu-15.04"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "qemux86"
DISTRO            = "poky"
DISTRO_VERSION    = "1.7.2"
TUNE_FEATURES     = "m32 i586"
TARGET_FPU        = ""
meta              
meta-yocto        
meta-yocto-bsp    = "(detachedfrom29812e6):29812e61736a95f1de64b3e9ebbb9c646ebd28dd"

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 382 tasks of which 382 didn't need to be rerun and all succeeded.

Summary: There was 1 WARNING message shown.
lucian@lucian-OptiPlex-9010:~/poky/build$
Comment 3 Saul Wold 2015-05-12 19:15:40 UTC
Did this pass for the RC2 build?
Comment 4 Costin 2015-05-13 13:23:36 UTC
Hello,

For the RC2 build it did pass. I tested it. However, it seems that this bug is not consistent in terms of reproducibility. There are cases when both import/export of database work fine with no issue and there are times like comment#2 when it alternates. This bug is similar to 6752. 

Costin
Comment 5 Richard Purdie 2015-05-13 14:34:14 UTC
The message "Only one copy of bitbake should be run against a build directory" means what it says, there are multiple copies of bitbake running. Is this something transient, e.g. does waiting a short while make the problem go away, or do you have to kill processes to allow things to work again?

If its a transient process, its much less of an issue that if bitbake is hanging/crashing.
Comment 6 Lucian Musat 2015-05-14 08:46:53 UTC
Hi Richard,

As you can see I looked with ps and could not find any other bitbake process running. Furthermore bitbake-prserv-tool complained but the bitbake command ran just fine. If there was a bitbake process in the background bitbake itself would have complained about that, but it didn't.
Comment 7 Alejandro Hernandez 2015-06-25 17:12:49 UTC
I am not able to reproduce this locally, the process for me goes smoothly, importing the db successfully, and after rebuilding the PR is increased by 1 as it should've, could this be Autobuilder specific?, I will test on the local AB just in case
Comment 8 Alejandro Hernandez 2015-06-25 19:45:35 UTC
I need more info since I am not able to reproduce this
Comment 9 Richard Purdie 2015-07-10 11:44:00 UTC
I believe this to be the same issues as Leonardo's changes from http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b5b74b430f0c62bcb4bbbcce02b18f2cb12d336 (and preceeding two patches) fix andrelated to bug #7941. I believe this is resolved now.
Comment 10 Richard Purdie 2015-07-10 11:44:09 UTC
Resolve.