Bug 4985

Summary: do smart --help command, sometimes causes a kernel panic
Product: [Build System, Metadata & Runtime] BSPs Reporter: Ning Zhang <ning.zhang>
Component: bsps-meta-yoctoAssignee: Bruce Ashfield <bruce.ashfield>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: Medium CC: ke.zou, sgw, yi.zhao
Version: 1.5   
Target Milestone: 1.5.1   
Hardware: Beagleboard   
OS: arm   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Ning Zhang 2013-08-06 08:50:31 UTC
Poky: master:9de0ad47462c13ac4a2d73e12e92be6c1b4e0415

Steps:
1. Download latest beagleboard from autobuilder.yoctoproject.org/pub/nightly/20130731-1, and use it bootup a beagleboard xm board

2. Create below test script on target, named as test.sh

#! /bin/bash
SMART=smart

COMMANDS="update install reinstall upgrade remove check fix download clean \
        search query newer info stats \
        config channel priority mirror flag"

for i in $COMMANDS
do
        sleep 1s
        $SMART $i --help
        if [ $? -ne 0 ]; then
                echo "$SMART $i --help run failed"
                $SMART $i --help 2>&1
                RET=1
        fi
done

3. run this script 10 times. 

for i in $(seq 1 10);do echo $i;./test.sh > aaa$i.log; done



============My test log======================= 
root@beagleboard:~# for i in $(seq 1 10);do echo $i;./test.sh > aaa$i.log; done
1
2
./test.sh: line 8:   611 Segmentation fault      $SMART $i --help
3
Traceback (most recent call last):
  File "/usr/bin/smart", line 35, in <module>
    import pyexpat
ImportError: /usr/lib/python2.7/lib-dynload/pyexpat.so: file too short
4
5
6
7
./test.sh: line 8:   830 Segmentation fault      $SMART $i --help
8
./test.sh: line 8:   872 Segmentation fault      $SMART $i --help
9
10


2.
Comment 1 Ning Zhang 2013-08-06 10:02:07 UTC
I retest this on a beagleboard c4 target, it has the same issue.

See test log:

onet@beagleboard:~# for i in $(seq 1 10); do echo $i; ./test.sh > test$i.log ; do
1
2
3
Unable to handle kernel paging request at virtual address ffffffff
pgd = cd984000
[ffffffff] *pgd=8f0fe821, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#2] ARM
Modules linked in: xt_connmark iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle iptable_filter ip_tables x_tables nfnetlink
CPU: 0    Tainted: G      D       (3.4.52-yocto-standard #1)
PC is at __d_lookup_rcu+0x114/0x164
LR is at do_lookup+0x50/0x2d8
pc : [<c011eac0>]    lr : [<c0114b28>]    psr: 600f0013
sp : cd8f5d2c  ip : 00000062  fp : cd8f5d54
r10: ce4ca005  r9 : 00000003  r8 : cd8f5dc4
r7 : ce9bf580  r6 : 00000002  r5 : 002697fa  r4 : ce9d3388
r3 : ffffffff  r2 : c0114b28  r1 : ce9d33a6  r0 : c011eab0
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: 8d984019  DAC: 00000015
Process python (pid: 1699, stack limit = 0xcd8f42f0)
Stack: (0xcd8f5d2c to 0xcd8f6000)
5d20:                            cd8f5db4 cd8f5dbc ce4ca008 ce9bf580 cd8f5db4
5d40: ce49c010 cd8f5e80 cd8f5d94 cd8f5d58 c0114b28 c011e9b8 00000007 cd8f5dc4
5d60: cd8f5d7c 00000002 c02c751c cd8f5dbc 0000006c ce4ca008 cd8f4000 cd8f5db4
5d80: 00000000 cd8f5e80 cd8f5dfc cd8f5d98 c0114f04 c0114ae4 cec03e00 c0c5e570
5da0: c0b195c0 00000001 ce4ca009 b6f4a000 cd8f5de4 ce49c010 ce9bf580 ce9ce258
5dc0: cd8f5dfc 002697fa 00000003 ce4ca005 00000000 00000041 ce4ca000 ce4ca000
5de0: ffffff9c cd8f4000 cd8f5e80 00000000 cd8f5e5c cd8f5e00 c01156f4 c0114dbc
5e00: cd8f5e18 cd8f5e10 c00d24e0 c00d2084 c0b195c0 00000001 00000000 c09194a4
5e20: 00000001 c01129a4 cd8f5e84 c00d20a8 c08ab7d0 00000001 cd8f5e80 ce4ca000
5e40: ffffff9c ffffff9c cd8f4000 00000000 cd8f5e7c cd8f5e60 c0115dc8 c011569c
5e60: ce4ca000 00000001 cd8f5e80 cd8f5f20 cd8f5f04 cd8f5e80 c0117554 c0115da8
5e80: ce49c010 ce9bf580 c00d22b4 c00d2230 cd8f5eac ce49c010 ce9b1f00 ce9ce258
5ea0: 00000051 00000002 00000001 00000000 cd8f5ebc 00000000 b6f4a000 00000001
5ec0: cd88dc00 00000000 cd8e13c8 bf000000 00000000 00000000 00000400 cd877000
5ee0: cd88dc00 bea2b020 cd8f5f40 44acc1a8 000000c3 c000e468 cd8f5f1c cd8f5f08
5f00: c01175ac c01174fc 00000000 cd8f5f18 cd8f5f3c cd8f5f20 c010d43c c0117594
5f20: cd8f5f6c cd8f5f30 bea2b020 000d8530 cd8f5fa4 cd8f5f40 c010d9c8 c010d408
5f40: b6f4a000 c000e468 cd8f4000 00000000 cd8f5f6c cd8f5f60 c005b680 c02fee88
5f60: cd8f5f8c cd8f5f70 c00ee4a8 c005b674 00001000 b6f4a000 00000000 0000005b
5f80: cd8f5fa4 cd8f5f90 c00ef818 c00ee45c 000fbe28 00000000 00000000 cd8f5fa8
5fa0: c000e200 c010d9a0 00000000 000d8530 bea2d12c bea2b020 bea2b020 00000002
5fc0: 00000000 000d8530 44acc1a8 000000c3 bea2d12c bea2b020 00000030 00000001
5fe0: 44acd100 bea2afd4 44a60940 45d503ac a00f0010 bea2d12c 00000000 00000000
[<c011eac0>] (__d_lookup_rcu+0x114/0x164) from [<c0114b28>] (do_lookup+0x50/0x2d8)
[<c0114b28>] (do_lookup+0x50/0x2d8) from [<c0114f04>] (link_path_walk+0x154/0x818)
[<c0114f04>] (link_path_walk+0x154/0x818) from [<c01156f4>] (path_lookupat+0x64/0x70c)
[<c01156f4>] (path_lookupat+0x64/0x70c) from [<c0115dc8>] (do_path_lookup+0x2c/0x6c)
[<c0115dc8>] (do_path_lookup+0x2c/0x6c) from [<c0117554>] (user_path_at_empty+0x64/0x98)
[<c0117554>] (user_path_at_empty+0x64/0x98) from [<c01175ac>] (user_path_at+0x24/0x2c)
[<c01175ac>] (user_path_at+0x24/0x2c) from [<c010d43c>] (vfs_fstatat+0x40/0x78)
[<c010d43c>] (vfs_fstatat+0x40/0x78) from [<c010d9c8>] (sys_stat64+0x34/0x50)
[<c010d9c8>] (sys_stat64+0x34/0x50) from [<c000e200>] (ret_fast_syscall+0x0/0x30)
Code: e51b203c e51b0038 e5836000 e51b3030 (e5832000) 
---[ end trace afa6248f98ef0d40 ]---
./test.sh: line 8:  1699 Segmentation fault      $SMART $i --help
4
5
6
7
Unable to handle kernel paging request at virtual address ffffffff
pgd = cd984000
[ffffffff] *pgd=8f0fe821, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#3] ARM
Modules linked in: xt_connmark iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle iptable_filter ip_tables x_tables nfnetlink
CPU: 0    Tainted: G      D       (3.4.52-yocto-standard #1)
PC is at __d_lookup_rcu+0x114/0x164
LR is at do_lookup+0x50/0x2d8
pc : [<c011eac0>]    lr : [<c0114b28>]    psr: 600f0013
sp : cd989d2c  ip : 00000037  fp : cd989d54
r10: ce4cf009  r9 : 00000009  r8 : cd989dc4
r7 : ce9d3380  r6 : 00000002  r5 : d8122fa3  r4 : cc859888
r3 : ffffffff  r2 : cc8527b0  r1 : cc8598ac  r0 : cc859880
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: 8d984019  DAC: 00000015
Process python (pid: 1852, stack limit = 0xcd9882f0)
Stack: (0xcd989d2c to 0xcd98a000)
9d20:                            cd989db4 cd989dbc ce4cf012 ce9d3380 cd989db4
9d40: ce49c010 cd989e80 cd989d94 cd989d58 c0114b28 c011e9b8 00000007 cd989dc4
9d60: cd989d7c 00000002 c02c751c cd989dbc 00000070 ce4cf012 cd988000 cd989db4
9d80: 00000000 cd989e80 cd989dfc cd989d98 c0114f04 c0114ae4 cec03e00 c0c5f328
9da0: c0b1cca0 00000001 ce4cf013 b6fb8000 cd989de4 ce49c010 ce9d3380 ce9c9980
9dc0: cd989dfc d8122fa3 00000009 ce4cf009 00000000 00000041 ce4cf000 ce4cf000
9de0: ffffff9c cd988000 cd989e80 00000000 cd989e5c cd989e00 c01156f4 c0114dbc
9e00: cd989e18 cd989e10 c00d24e0 c00d2084 c0b1cca0 00000001 00000000 c09194a4
9e20: 00000001 c01129a4 cd989e84 c00d20a8 c08ab7d0 00000001 cd989e80 ce4cf000
9e40: ffffff9c ffffff9c cd988000 00000000 cd989e7c cd989e60 c0115dc8 c011569c
9e60: ce4cf000 00000001 cd989e80 cd989f20 cd989f04 cd989e80 c0117554 c0115da8
9e80: ce49c010 ce9d3380 c00d22b4 c00d2230 cd989eac ce49c010 ce9b1f00 ce9c9980
9ea0: 00000051 00000002 00000001 00000000 cd989ebc 00000000 b6fb8000 c010a5b0
9ec0: cd88d900 00000000 cd804738 bf000000 00000000 00000000 00000400 cd8f2000
9ee0: cd88d900 bed9f7a8 cd989f40 44acc1a8 000000c3 c000e468 cd989f1c cd989f08
9f00: c01175ac c01174fc 00000000 ce49c010 cd989f3c cd989f20 c010d43c c0117594
9f20: cd989f64 cd989f30 bed9f7a8 000d8530 cd989fa4 cd989f40 c010d9c8 c010d408
9f40: cd821c00 00000000 00000006 c000e468 cd988000 00000000 cd989f84 cd989f68
9f60: c0107544 c010a3d4 00000004 0000000f 00000000 00000003 cd989fa4 cd989f88
9f80: c0107620 c01074dc 45cfc1e4 0002da80 b6fb6000 00000000 00000000 cd989fa8
9fa0: c000e200 c010d9a0 00000000 000d8530 beda18b4 bed9f7a8 bed9f7a8 00000005
9fc0: 00000000 000d8530 44acc1a8 000000c3 beda18b4 bed9f7a8 0000002d 00000001
9fe0: 44acd100 bed9f75c 44a60940 45d503ac a00f0010 beda18b4 00000000 00000000
[<c011eac0>] (__d_lookup_rcu+0x114/0x164) from [<c0114b28>] (do_lookup+0x50/0x2d8)
[<c0114b28>] (do_lookup+0x50/0x2d8) from [<c0114f04>] (link_path_walk+0x154/0x818)
[<c0114f04>] (link_path_walk+0x154/0x818) from [<c01156f4>] (path_lookupat+0x64/0x70c)
[<c01156f4>] (path_lookupat+0x64/0x70c) from [<c0115dc8>] (do_path_lookup+0x2c/0x6c)
[<c0115dc8>] (do_path_lookup+0x2c/0x6c) from [<c0117554>] (user_path_at_empty+0x64/0x98)
[<c0117554>] (user_path_at_empty+0x64/0x98) from [<c01175ac>] (user_path_at+0x24/0x2c)
[<c01175ac>] (user_path_at+0x24/0x2c) from [<c010d43c>] (vfs_fstatat+0x40/0x78)
[<c010d43c>] (vfs_fstatat+0x40/0x78) from [<c010d9c8>] (sys_stat64+0x34/0x50)
[<c010d9c8>] (sys_stat64+0x34/0x50) from [<c000e200>] (ret_fast_syscall+0x0/0x30)
Code: e51b203c e51b0038 e5836000 e51b3030 (e5832000) 
---[ end trace afa6248f98ef0d41 ]---
./test.sh: line 8:  1852 Segmentation fault      $SMART $i --help
Unable to handle kernel paging request at virtual address ffffffff
pgd = cd984000
[ffffffff] *pgd=8f0fe821, *pte=00000000, *ppte=00000000
Internal error: Oops: 817 [#4] ARM
Modules linked in: xt_connmark iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle iptable_filter ip_tables x_tables nfnetlink
CPU: 0    Tainted: G      D       (3.4.52-yocto-standard #1)
PC is at __d_lookup_rcu+0x114/0x164
LR is at do_lookup+0x50/0x2d8
pc : [<c011eac0>]    lr : [<c0114b28>]    psr: 600f0013
sp : ce407da4  ip : 00000037  fp : ce407dcc
r10: ce4cb009  r9 : 00000009  r8 : ce407e3c
r7 : ce9d3380  r6 : 00000002  r5 : d8122fa3  r4 : cc859888
r3 : ffffffff  r2 : cc8527b0  r1 : cc8598ac  r0 : cc859880
Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 10c5387d  Table: 8d984019  DAC: 00000015
Process python (pid: 1869, stack limit = 0xce4062f0)
Stack: (0xce407da4 to 0xce408000)
7da0:          ce407e2c ce407e34 ce4cb012 ce9d3380 ce407e2c ce49c010 ce407ed8
7dc0: ce407e0c ce407dd0 c0114b28 c011e9b8 00000007 ce407e3c ce407df4 00000002
7de0: c02c751c ce407e34 00000070 ce4cb012 ce406000 ce407e2c 00000000 ce407ed8
7e00: ce407e74 ce407e10 c0114f04 c0114ae4 c0114ac0 c0122d4c cc8c2100 c00ff990
7e20: ce4cb013 ce407e30 c08f1000 ce49c010 ce9d3380 ce9c9980 ce407e74 d8122fa3
7e40: 00000009 ce4cb009 c010a6f4 ce407f60 00000000 ce4cb000 ffffff9c 00000001
7e60: ce406000 ce407ed8 ce407ecc ce407e78 c01169e0 c0114dbc ce407e8c c00ff9e4
7e80: ce49c010 cc8c0f80 d22a33a8 00000000 ce49c010 cc8c2100 ce9b1f00 cc886b40
7ea0: 00000001 ce407f60 00000001 ce4cb000 ffffff9c 00000001 ce406000 00000000
7ec0: ce407f54 ce407ed0 c01175f0 c0116940 00000041 00000817 ce49c010 ce9d3380
7ee0: b17d354a 00000009 ce4cb027 ce49c010 ce9b1f00 ce9c9980 00000051 00000002
7f00: 00000001 00000000 00000000 00000005 ce6db300 cd8f93c0 00000000 00020000
7f20: ce407f54 ce407f30 c012274c 00020000 00000000 ce6d9480 00020000 ce4cb000
7f40: 00000005 ffffff9c ce407f94 ce407f58 c01085e8 c01175c0 ce407f84 ce407f68
7f60: 00020000 c0100000 00000024 00000100 00032688 44a976ee 00000008 00000005
7f80: c000e468 ce406000 ce407fa4 ce407f98 c01086a8 c01084f8 00000000 ce407fa8
7fa0: c000e200 c0108688 00032688 44a976ee becd4f3c 00020000 000001b6 000001b6
7fc0: 00032688 44a976ee 00000008 00000005 00000000 00014a4c 44affac8 becd2dc4
7fe0: 00000000 becd2d64 45cfd758 45d50fdc 600f0010 becd4f3c 00000000 00000000
[<c011eac0>] (__d_lookup_rcu+0x114/0x164) from [<c0114b28>] (do_lookup+0x50/0x2d8)
[<c0114b28>] (do_lookup+0x50/0x2d8) from [<c0114f04>] (link_path_walk+0x154/0x818)
[<c0114f04>] (link_path_walk+0x154/0x818) from [<c01169e0>] (path_openat+0xac/0x3ac)
[<c01169e0>] (path_openat+0xac/0x3ac) from [<c01175f0>] (do_filp_open+0x3c/0x90)
[<c01175f0>] (do_filp_open+0x3c/0x90) from [<c01085e8>] (do_sys_open+0xfc/0x190)
[<c01085e8>] (do_sys_open+0xfc/0x190) from [<c01086a8>] (sys_open+0x2c/0x30)
[<c01086a8>] (sys_open+0x2c/0x30) from [<c000e200>] (ret_fast_syscall+0x0/0x30)
Code: e51b203c e51b0038 e5836000 e51b3030 (e5832000) 
---[ end trace afa6248f98ef0d42 ]---
./test.sh: line 8:  1869 Segmentation fault      $SMART $i --help
8
9
10
Comment 2 Saul Wold 2013-08-07 02:15:37 UTC
What about other hardware or Qemu targets, is this beagleboard specific?
Comment 3 Ning Zhang 2013-08-07 06:31:15 UTC
(In reply to comment #2)
> What about other hardware or Qemu targets, is this beagleboard specific?

Hi, 

I do the same test in qemuarm and 8315E, with 20130731-1 images, no this issue.
I also test beagleboard c4 with 20130625-2 image, no this issue neither.

So i think this is a beagleboard specific issue.
Comment 4 Ning Zhang 2013-08-09 09:42:31 UTC
I've done this test with below scenarios:

1. test 20130731-1 images, kernel is 3.4.52, poky is 9de0ad47462c13ac4a2d73e12e92be6c1b4e0415, test failed.
2. test 20130618-2 images, kernel is 3.4.36, poky is 590010a6525b0e1bc1de73e794764e23404591df, test pass.
3. Use scenario 1 image, downgrade kernel version with scenario 2(cp uImage, MLO files and /lib/modules/ files), which means use 3.4.36 kernel and 3.4.52 rootfs, test pass. 
4. Use scenario 2 image, upgrade kernel version with scenario 1(cp uImage, MLO files and /lib/modules/ files), which means use 3.4.52 kernel and 3.4.36 rootfs, test failed.

I tested all above on a same beagleboard xm target.

So we guess this is not a hardware issue, it is the 3.4.52 kernel issue.
Comment 5 Bruce Ashfield 2013-08-09 12:49:42 UTC
This is still related to the particular board and it's storage. if this was a general
3.4.52 issue, all boards would be seeing the problem.

I'm going to bump 3.4 to 3.4.57 shortly, so it will be worth retesting it there.
Comment 6 Bruce Ashfield 2013-09-13 04:24:27 UTC
This should be retried on 3.10.
Comment 7 Bruce Ashfield 2013-11-06 16:33:00 UTC
Has this been retested against 3.10, or are the remaining issues with the beagleboard support preventing
that test ? (those other issues should be solved soon).
Comment 8 Bruce Ashfield 2014-01-15 02:04:39 UTC
I haven't reproduced this, or heard of it on 3.10, so I'm closing
the bug as "wontfix".