Bug 4872 - Do smart install and remove more times will occur Segmentation fault
Summary: Do smart install and remove more times will occur Segmentation fault
Status: VERIFIED FIXED
Alias: None
Product: Package Management Issues
Classification: Runtime
Component: RPM (show other bugs)
Version: 1.5
Hardware: Beagleboard arm
: Medium normal
Target Milestone: 1.9
Assignee: Mark Hatle
QA Contact: Yi Zhao
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2013-07-19 08:53 UTC by Ning Zhang
Modified: 2015-09-08 07:04 UTC (History)
6 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: Don't know


Attachments
test script (213 bytes, application/x-shellscript)
2013-07-19 08:53 UTC, Ning Zhang
no flags Details
test log (16.68 KB, text/x-log)
2013-07-19 08:54 UTC, Ning Zhang
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ning Zhang 2013-07-19 08:53:45 UTC
Created attachment 1337 [details]
test script

Poky: master:3dee534f1e25109e0bdb681de0746c336f4b8840

When i run the smart automated test on beagleboard target, it always has some Segmentation fault, and each time it occurs with different place. This matter first happened in yocto 1.5 M2.

So i create a short smart test case, it do install and remove groff 10 tiems, run it, "Segmentation fault" happens 2 times.

This issue just exists in 1.5M3 weekly test.

Detail please see attachments.

===================test code====================
#!/bin/sh

for i in $(seq 1 10)
do
echo start $i test
smart install  http://128.224.162.102/rpmrepo/beagleboard/armv7a_vfp_neon/groff-1.22.2-r0.armv7a_vfp_neon.rpm -y
sleep 1s
smart remove groff -y
sleep 1s
done


===================some log=====================
start 2 test                                                                
                                                                               ^MFetching packages...    
                                                                               ^M-> http://128.224.162.102/rpmrepo/.../groff-1.22.2-r0.armv7a_vfp_neon.rpm
groff-1.22.2-r0.armv7a_vfp_ne.. ######################################## [100%]                                                                                                                                 
Loading cache...                                                                
Updating cache...               ######################################## [100%]                                                                                                                                    
Computing transaction...                                                                               ^M
Committing transaction...                                                                            
Preparing...                    ######################################## [  0%]                                                                           
./test.sh: line 3:  1422 Segmentation fault      smart install http://128.224.162.102/rpmrepo/beagleboard/armv7a_vfp_neon/groff-1.22.2-r0.armv7a_vfp_neon.rpm -y
Loading cache...BDB2053 Freeing read locks for locker 0x1fe1: 1422/3069939712  
BDB2053 Freeing read locks for locker 0x1fe2: 1422/3069939712                                                                                                   
BDB2053 Freeing read locks for locker 0x1fe5: 1422/3069939712                  
BDB2053 Freeing read locks for locker 0x1ff7: 1422/3069939712                                        
BDB2053 Freeing read locks for locker 0x1ff8: 1422/3069939712                                                                                             
BDB2053 Freeing read locks for locker 0x1ffa: 1422/3069939712                  
BDB2053 Freeing read locks for locker 0x1fff: 1422/3069939712                  
BDB2053 Freeing read locks for locker 0x2001: 1422/3069939712                                                                                                   
BDB2053 Freeing read locks for locker 0x2004: 1422/3069939712                  
BDB4503 Aborting txn 0x8000189c: 1422/3069939712                                                     
BDB1502 Freeing log information for process: 1422/0, (ref 1)                                                                                              
BDB1502 Freeing log information for process: 1422/0, (ref 1)                                                                      
BDB1502 Freeing log information for process: 1422/0, (ref 1)                   
BDB1502 Freeing log information for process: 1422/0, (ref 1)                                                                                                    
BDB1502 Freeing log information for process: 1422/0, (ref 1)                   
BDB1502 Freeing log information for process: 1422/0, (ref 1)                                         
BDB2017 Freeing mutex for process: 1422/0                                                                                                                 
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                                                                                                       
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                                            
BDB2017 Freeing mutex for process: 1422/0                                                                                                                 
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                                                                                                       
BDB2017 Freeing mutex for process: 1422/0                                                                                                                 
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                                                
BDB2017 Freeing mutex for process: 1422/0                    
BDB2017 Freeing mutex for process: 1422/0                                      
BDB2017 Freeing mutex for process: 1422/0                                                                                                                       
BDB2017 Freeing mutex for process: 1422/0                                                                
BDB2017 Freeing mutex for process: 1422/0                                                            
BDB2017 Freeing mutex for process: 1422/0                                                                                                                 
BDB2017 Freeing mutex for process: 1422/0
Comment 1 Ning Zhang 2013-07-19 08:54:24 UTC
Created attachment 1338 [details]
test log
Comment 2 Ning Zhang 2013-08-09 09:48:39 UTC
I've done this test with below scenarios:

1. test 20130731-1 images, kernel is 3.4.52, poky is 9de0ad47462c13ac4a2d73e12e92be6c1b4e0415, test failed.
2. test 20130618-2 images, kernel is 3.4.36, poky is 590010a6525b0e1bc1de73e794764e23404591df, test pass.
3. Use scenario 1 image, downgrade kernel version with scenario 2(cp uImage, MLO files and /lib/modules/ files), which means use 3.4.36 kernel and 3.4.52 rootfs, test pass. 
4. Use scenario 2 image, upgrade kernel version with scenario 1(cp uImage, MLO files and /lib/modules/ files), which means use 3.4.52 kernel and 3.4.36 rootfs, test failed.

I tested all above on a same beagleboard xm target.

So we guess this is not a hardware issue, it is the 3.4.52 kernel issue.
Comment 3 Mark Hatle 2013-09-25 15:48:08 UTC
A few notes.

I have seen this behavior in the past, and it's always been tied to something within BerkleyDB.

Unfortunately, I have no previously been able to reproduce it consistently enough to try to figure out what the cause actually is.

My guess is that it's related to a mismatch in the locking between the host system and target on the initial creation of the database.  In the past reconstructing the RPM DB  (--rebuilddb) and/or simply removing the __db.* files has fixed the issue.

If we can find someone with more in-depth knowledge of BerkleyDB then we might be able to find and fix the bug.
Comment 4 Jeff Johnson 2014-11-24 21:27:07 UTC
Does this problem still exist?

Can you try running under valgrind (moderately slow)
or compiled with -fsanitize:address using gcc-4.8.2
and/or clang?

I'd like to see a stack trace before hazarding a guess.

There are also several issues fixed since Sept 2013 which
might have been the cause.
Comment 5 Mark Hatle 2014-12-01 17:41:20 UTC
I will try to add some instrumentation to the system and see if we can reproduce the issue(s).
Comment 6 Alexandru Georgescu 2015-01-30 12:33:48 UTC
(In reply to comment #5)
> I will try to add some instrumentation to the system and see if we can
> reproduce the issue(s).

Adding Yi to the thread.

Yi, does this happen in Beaglebone in 1.8?
Comment 7 Mark Hatle 2015-04-28 18:16:24 UTC
I've been unable to replicate this with YP 1.7 and YP 1.8 on an IA32 (32-bit or 64-bit) target system.

I'm going to resolve this as I suspect the issue was fixed int he 1.7 time frame as part of the BerkleyDB and other standard upgrades.

If the issue comes back, please reopen.
Comment 8 Yi Zhao 2015-09-08 07:04:33 UTC
We also didn't encounter this issue a long time. Mark it to verified.