Tree/branch: poky/master Commit：eb0cb7e8234f5d2e5623406e9660be91cf52f65e Image location: http://autobuilder.pokylinux.org/pub/nightly/20120712-1/machines/mpc8315e-rdb/ Kernel crashes when mpc8315e-rdb booting. Here is the boot log: ########### U-Boot 2010.09 (Oct 20 2010 - 13:29:02) MPC83XX Reset Status: CPU: e300c3, MPC8315E, Rev: 1.2 at 400 MHz, CSB: 133.333 MHz Board: Freescale MPC8315ERDB Rev 1.0 I2C: ready DRAM: 128 MiB FLASH: 8 MiB NAND: 32 MiB PCIE0: No link PCIE1: No link In: serial Out: serial Err: serial Net: eTSEC0, eTSEC1 Hit any key to stop autoboot: 0 Speed: 100, full duplex Using eTSEC0 device TFTP from server 128.224.165.20; our IP address is 128.224.178.141 Filename 'vlm-boards/17652/kernel'. Load address: 0x3000000 Loading: ################################################################# ################################################################# ################################################################# ############################################################### done Bytes transferred = 3778111 (39a63f hex) Speed: 100, full duplex Using eTSEC0 device TFTP from server 128.224.165.20; our IP address is 128.224.178.141 Filename 'vlm-boards/17652/dtb'. Load address: 0x2000000 Loading: ## done Bytes transferred = 21042 (5232 hex) ## Booting kernel from Legacy Image at 03000000 ... Image Name: Linux-3.4.4-yocto-standard Created: 2012-07-12 16:44:24 UTC Image Type: PowerPC Linux Kernel Image (gzip compressed) Data Size: 3778047 Bytes = 3.6 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 02000000 Booting using the fdt blob at 0x2000000 Uncompressing Kernel Image ... OK Loading Device Tree to 047f7000, end 047ff231 ... OK NOTICE: No or unknown board_type hwconfig specified. Assuming board with TSEC1. Using MPC831x RDB machine description Initializing cgroup subsys cpuset Initializing cgroup subsys cpu Linux version 3.4.4-yocto-standard (pokybuild@autobuilder05) (gcc version 4.7.2 20120615 (prerelease) (GCC) ) #1 PREEMPT Thu Jul 12 09:44:18 PDT 2012 bootconsole [udbg0] enabled setup_arch: bootmem mpc831x_rdb_setup_arch() Found FSL PCI host bridge at 0x00000000e0008500. Firmware bus number: 0->0 PCI host bridge /pci@e0008500 (primary) ranges: MEM 0x0000000090000000..0x000000009fffffff -> 0x0000000090000000 MEM 0x0000000080000000..0x000000008fffffff -> 0x0000000080000000 Prefetch IO 0x00000000e0300000..0x00000000e03fffff -> 0x0000000000000000 No pci config register base in dev tree, using default Found FSL PCI host bridge at 0x00000000e0009000. Firmware bus number: 0->255 PCI host bridge /pcie@e0009000 ranges: MEM 0x00000000a0000000..0x00000000afffffff -> 0x00000000a0000000 IO 0x00000000b1000000..0x00000000b17fffff -> 0x0000000000000000 No pci config register base in dev tree, using default Found FSL PCI host bridge at 0x00000000e000a000. Firmware bus number: 0->255 PCI host bridge /pcie@e000a000 ranges: MEM 0x00000000c0000000..0x00000000cfffffff -> 0x00000000c0000000 IO 0x00000000d1000000..0x00000000d17fffff -> 0x0000000000000000 arch: exit Zone PFN ranges: DMA 0x00000000 -> 0x00008000 Normal empty Movable zone start PFN for each node Early memory PFN ranges 0: 0x00000000 -> 0x00008000 Built 1 zonelists in Zone order, mobility grouping on. Total pages: 32512 Kernel command line: root=/dev/nfs rw nfsroot=128.224.165.20:/export/pxeboot/vlm-boards/17652/rootfs/ ip=128.224.178.141:128.224.165.20:128.224.178.1:::eth0:off console=ttyS0,115200 PID hash table entries: 512 (order: -1, 2048 bytes) Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) allocated 262144 bytes of page_cgroup please try 'cgroup_disable=memory' option if you don't want memory cgroups Memory: 121204k/131072k available (7676k kernel code, 9868k reserved, 320k data, 375k bss, 284k init) Kernel virtual memory layout: * 0xfffdf000..0xfffff000 : fixmap * 0xfcef7000..0xfe000000 : early ioremap * 0xc9000000..0xfcef7000 : vmalloc & ioremap SLUB: Genslabs=15, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 NR_IRQS:512 nr_irqs:512 16 IPIC (128 IRQ sources) at c9000700 clocksource: timebase mult[1e000005] shift[24] registered Console: colour dummy device 80x25 pid_max: default: 32768 minimum: 301 Security Framework initialized Mount-cache hash table entries: 512 Initializing cgroup subsys debug Initializing cgroup subsys cpuacct Initializing cgroup subsys memory Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio ftrace: allocating 19430 entries in 57 pages devtmpfs: initialized NET: Registered protocol family 16 PCI: Probing PCI hardware PCI host bridge to bus 0000:00 pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] pci_bus 0000:00: root bus resource [mem 0x90000000-0x9fffffff] pci_bus 0000:00: root bus resource [mem 0x80000000-0x8fffffff pref] PCI host bridge to bus 0001:01 pci_bus 0001:01: root bus resource [io 0xff7fe000-0xffffdfff] (bus address [0x0000-0x7fffff]) pci_bus 0001:01: root bus resource [mem 0xa0000000-0xafffffff] PCI host bridge to bus 0002:02 pci_bus 0002:02: root bus resource [io 0xfeffc000-0xff7fbfff] (bus address [0x0000-0x7fffff]) pci_bus 0002:02: root bus resource [mem 0xc0000000-0xcfffffff] bio: create slab <bio-0> at 0 Freescale Elo / Elo Plus DMA driver vgaarb: loaded SCSI subsystem initialized usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb Switching to clocksource timebase NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 4096 (order: 3, 32768 bytes) TCP bind hash table entries: 4096 (order: 2, 16384 bytes) TCP: Hash tables configured (established 4096 bind 4096) TCP: reno registered UDP hash table entries: 256 (order: 0, 4096 bytes) UDP-Lite hash table entries: 256 (order: 0, 4096 bytes) NET: Registered protocol family 1 RPC: Registered named UNIX socket transport module. RPC: Registered udp transport module. RPC: Registered tcp transport module. RPC: Registered tcp NFSv4.1 backchannel transport module. mpc8xxx_wdt: WDT driver for MPC8xxx initialized. mode:reset timeout=65535 (32 seconds) fsl-elo-dma e00082a8.dma: #0 (fsl,elo-dma-channel), irq 71 fsl-elo-dma e00082a8.dma: #1 (fsl,elo-dma-channel), irq 71 fsl-elo-dma e00082a8.dma: #2 (fsl,elo-dma-channel), irq 71 fsl-elo-dma e00082a8.dma: #3 (fsl,elo-dma-channel), irq 71 VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) Registering unionfs 2.5.10 (for 3.1.0-rc4) NFS: Registering the id_resolver key type Btrfs loaded msgmni has been set to 236 Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) io scheduler noop registered io scheduler deadline registered io scheduler cfq registered (default) Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled serial8250.0: ttyS0 at MMIO 0xe0004500 (irq = 16) is a 16550A console [ttyS0] enabled, bootconsole disabled console [ttyS0] enabled, bootconsole disabled serial8250.0: ttyS1 at MMIO 0xe0004600 (irq = 17) is a 16550A brd: module loaded fsl-sata e0018000.sata: Sata FSL Platform/CSB Driver init scsi0 : sata_fsl ata1: SATA max UDMA/133 irq 44 fsl-sata e0019000.sata: Sata FSL Platform/CSB Driver init scsi1 : sata_fsl ata2: SATA max UDMA/133 irq 45 fe000000.flash: Found 1 x16 devices at 0x0 in 16-bit bank. Manufacturer ID 0x000001 Chip ID 0x001001 ata1: Signature Update detected @ 0 msecs Amd/Fujitsu Extended Query Table at 0x0040 Amd/Fujitsu Extended Query version 1.3. fe000000.flash: Swapping erase regions for top-boot CFI table. number of CFI chips: 1 fsl_spi e0007000.spi: master is unqueued, this is deprecated fsl_spi e0007000.spi: at 0xc9024000 (irq = 19), CPU mode Freescale PowerQUICC MII Bus: probed Freescale PowerQUICC MII Bus: probed fsl-gianfar e0024000.ethernet: eth0: mac: 04:00:00:00:00:0a fsl-gianfar e0024000.ethernet: eth0: Running with NAPI enabled fsl-gianfar e0024000.ethernet: eth0: RX BD ring size for Q[0]: 256 fsl-gianfar e0024000.ethernet: eth0: TX BD ring size for Q[0]: 256 fsl-gianfar e0025000.ethernet: eth1: mac: 04:00:00:00:00:0b fsl-gianfar e0025000.ethernet: eth1: Running with NAPI enabled fsl-gianfar e0025000.ethernet: eth1: RX BD ring size for Q[0]: 256 fsl-gianfar e0025000.ethernet: eth1: TX BD ring size for Q[0]: 256 ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver /immr@e0000000/usb@23000: Invalid 'dr_mode' property, fallback to host mode fsl-ehci fsl-ehci.0: Freescale On-Chip EHCI Host Controller fsl-ehci fsl-ehci.0: new USB bus registered, assigned bus number 1 fsl-ehci fsl-ehci.0: irq 38, io mem 0xe0023000 fsl-ehci fsl-ehci.0: USB 2.0 started, EHCI 1.00 hub 1-0:1.0: USB hub found hub 1-0:1.0: 1 port detected Initializing USB Mass Storage driver... usbcore: registered new interface driver usb-storage ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300) USB Mass Storage support registered. mousedev: PS/2 mouse device common for all mice ata1.00: ATA-8: WDC WD1600AAJS-22L7A0, 01.03E01, max UDMA/133 i2c /dev entries driver mpc-i2c e0003000.i2c: timeout 1000000 us ata1.00: 312581808 sectors, multi 0: LBA48 NCQ (depth 16/32) or FUA md: raid10 personality registered for level 10 md: multipath personality registered for level -4 md: faulty personality registered for level -5 device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com usbcore: registered new interface driver usbhid usbhid: USB HID core driver TCP: cubic registered NET: Registered protocol family 10 IPv6 over IPv4 tunneling driver NET: Registered protocol family 17 Registering the dns_resolver key type console [netcon0] enabled netconsole: network logging started rtc-ds1307 0-0068: setting system clock to 2012-07-17 13:30:33 UTC (1342531833) sda: sda1 sda2 sda3 sda4 < sda5 sda6 > ADDRCONF(NETDEV_UP): eth0: link is not ready sd 0:0:0:0: [sda] Attached SCSI disk usb 1-1: new high-speed USB device number 2 using fsl-ehci ata2: No Device OR PHYRDY change,Hstatus = 0xa0000000 at24 0-0050: 32768 byte 24c256 EEPROM, writable, 1 bytes/write ata1.00: configured for UDMA/133 rtc-ds1307 0-0068: rtc core: registered ds1339 as rtc0 scsi 0:0:0:0: Direct-Access ATA WDC WD1600AAJS-2 01.0 PQ: 0 ANSI: 5 md: linear personality registered for level -1 sd 0:0:0:0: [sda] 312581808 512-byte logical blocks: (160 GB/149 GiB) sd 0:0:0:0: Attached scsi generic sg0 type 0 md: raid0 personality registered for level 0 sd 0:0:0:0: [sda] Write Protect is off md: raid1 personality registered for level 1 sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA md: raid10 personality registered for level 10 md: multipath personality registered for level -4 md: faulty personality registered for level -5 device-mapper: ioctl: 4.22.0-ioctl (2011-10-19) initialised: dm-devel@redhat.com usbcore: registered new interface driver usbhid usbhid: USB HID core driver TCP: cubic registered NET: Registered protocol family 10 IPv6 over IPv4 tunneling driver NET: Registered protocol family 17 Registering the dns_resolver key type console [netcon0] enabled netconsole: network logging started rtc-ds1307 0-0068: setting system clock to 2012-07-17 13:30:33 UTC (1342531833) sda: sda1 sda2 sda3 sda4 < sda5 sda6 > ADDRCONF(NETDEV_UP): eth0: link is not ready sd 0:0:0:0: [sda] Attached SCSI disk usb 1-1: new high-speed USB device number 2 using fsl-ehci ata2: No Device OR PHYRDY change,Hstatus = 0xa0000000 ata2: SATA link down (SStatus 0 SControl 300) hub 1-1:1.0: USB hub found hub 1-1:1.0: 4 ports detected usb 1-1.1: new high-speed USB device number 3 using fsl-ehci scsi2 : usb-storage 1-1.1:1.0 scsi 2:0:0:0: Direct-Access SanDisk Cruzer Edge 1.01 PQ: 0 ANSI: 2 sd 2:0:0:0: Attached scsi generic sg1 type 0 sd 2:0:0:0: [sdb] 7821312 512-byte logical blocks: (4.00 GB/3.72 GiB) sd 2:0:0:0: [sdb] Write Protect is off sd 2:0:0:0: [sdb] No Caching mode page present sd 2:0:0:0: [sdb] Assuming drive cache: write through sd 2:0:0:0: [sdb] No Caching mode page present sd 2:0:0:0: [sdb] Assuming drive cache: write through sdb: sdb1 sd 2:0:0:0: [sdb] No Caching mode page present sd 2:0:0:0: [sdb] Assuming drive cache: write through sd 2:0:0:0: [sdb] Attached SCSI removable disk PHY: mdio@e0024520:00 - Link is Up - 100/Full ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready IP-Config: Guessing netmask 255.255.0.0 IP-Config: Complete: device=eth0, addr=128.224.178.141, mask=255.255.0.0, gw=128.224.178.1 host=128.224.178.141, domain=, nis-domain=(none) bootserver=128.224.165.20, rootserver=128.224.165.20, rootpath= md: Waiting for all devices to be available before autodetect md: If you don't use raid, use raid=noautodetect md: Autodetecting RAID arrays. md: Scanned 0 and added 0 devices. md: autorun ... md: ... autorun DONE. VFS: Mounted root (nfs filesystem) on device 0:13. devtmpfs: mounted Freeing unused kernel memory: 284k freed INIT: version 2.88 booting Error opening /dev/fb0: No such file or directory Starting udev udev[81]: starting version 164 Not a trampoline ------------[ cut here ]------------ WARNING: at /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-ppc/build/ build/tmp/work/mpc8315e_rdb-poky-linux/linux-yocto-3.4.4+git1+a8cf77018b0faa0d29f1483ff4e5a2034dc8edd5_1+47d3076aa7e2a640a1dab8d663529a784709f697-r4.0/linux/kernel/trace/ftrace.c:1583 Modules linked in: rng_core(+) NIP: c008c98c LR: c008d05c CTR: c0409378 REGS: c719fd20 TRAP: 0700 Not tainted (3.4.4-yocto-standard) MSR: 00029032 <EE,ME,IR,DR,RI> CR: 22228482 XER: 20000000 TASK = c7184880[125] 'modprobe' THREAD: c719e000 GPR00: c008d05c c719fdd0 c7184880 ffffffea c9035008 ffffffff c0394ea0 00000014 GPR08: 00000004 c07be3d5 00000001 0001ffff 22228424 10029140 00020000 1000d394 GPR16: bf86bfdc c9035008 c07d0000 00000000 c0010bb0 00000000 c078dcf4 c07fb6a8 GPR24: c07cc114 c7a4fe40 c9035b00 00000000 c7040000 00000000 00000001 c9035008 NIP [c008c98c] ftrace_bug+0xb8/0x240 LR [c008d05c] ftrace_process_locs+0x410/0x5b0 Call Trace: [c719fdd0] [c0094e4c] ftrace_now+0x34/0x80 (unreliable) [c719fdf0] [c008d05c] ftrace_process_locs+0x410/0x5b0 [c719fe40] [c008ea38] ftrace_module_notify+0x6c/0x80 [c719fe50] [c05dc8c4] notifier_call_chain+0x78/0xc0 [c719fe70] [c0049df4] __blocking_notifier_call_chain+0x5c/0x80 [c719fe90] [c006d6e0] sys_init_module+0x9b8/0x1ae8 [c719ff40] [c0010298] ret_from_syscall+0x0/0x38 --- Exception: c01 at 0xf4a551c LR = 0xff99fe0 Instruction dump: 7fe4fb78 38630ee8 7fe5fb78 80010024 bb410008 38210020 7c0803a6 4855441c 3d20c07c 3929e3d5 89490003 694a0001 <0f0a0000> 2f8a0000 409e015c 3d20c07d ---[ end trace 70754a5864a672e1 ]--- ftrace failed to modify [<c9035008>] rng_dev_open+0x8/0x50 [rng_core] actual: 48:00:08:29 talitos e0030000.crypto: hwrng talitos e0030000.crypto: fsl,sec3.3 algorithms registered in /proc/crypto EXT4-fs (sda): VFS: Can't find ext4 filesystem EXT3-fs (sda): error: can't find ext3 filesystem on dev sda. EXT2-fs (sda): error: can't find an ext2 filesystem on dev sda. FAT-fs (sda): invalid media value (0x00) FAT-fs (sda): Can't find a valid FAT filesystem EXT4-fs (sdb): VFS: Can't find ext4 filesystem EXT3-fs (sdb): error: can't find ext3 filesystem on dev sdb. EXT2-fs (sdb): error: can't find an ext2 filesystem on dev sdb. FAT-fs (sdb): bogus number of reserved sectors FAT-fs (sdb): Can't find a valid FAT filesystem VFS: Can't find a Minix filesystem V1 | V2 | V3 on device sda. FAT-fs (sda): invalid media value (0x00) FAT-fs (sda): Can't find a valid FAT filesystem VFS: Can't find a Minix filesystem V1 | V2 | V3 on device sdb. FAT-fs (sdb): bogus number of reserved sectors FAT-fs (sdb): Can't find a valid FAT filesystem EXT2-fs (sdb1): warning: mounting unchecked fs, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs (sda2): using internal journal EXT3-fs (sda5): using internal journal kjournald starting. Commit interval 5 seconds EXT3-fs (sda2): recovery complete EXT3-fs (sda5): recovery complete EXT3-fs (sda2): mounted filesystem with ordered data mode EXT3-fs (sda5): mounted filesystem with ordered data mode EXT3-fs (sda6): warning: maximal mount count reached, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs (sda6): using internal journal EXT3-fs (sda6): recovery complete EXT3-fs (sda6): mounted filesystem with ordered data mode EXT3-fs (sda1): warning: maximal mount count reached, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs (sda1): using internal journal EXT3-fs (sda1): recovery complete EXT3-fs (sda1): mounted filesystem with ordered data mode EXT3-fs (sda3): warning: mounting unchecked fs, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs (sda3): using internal journal EXT3-fs (sda3): recovery complete EXT3-fs (sda3): mounted filesystem with ordered data mode udevadm settle - timeout of 180 seconds reached, the event queue contains: /sys/devices/e0000000.immr/e0018000.sata/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda4 (1097) Starting Bootlog daemon: bootlogd. udevd[81]: worker [91] unexpectedly returned with status 0x0100 udevd[81]: worker [91] failed while handling '/devices/e0000000.immr/e0018000.sata/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda4' ############## The u-boot settings: ############## baudrate=115200 bootcmd=run nfsboot bootdelay=6 bootfile=vlm-boards/17652/kernel consoledev=ttyS0 eth1addr=04:00:00:00:00:0B ethact=eTSEC0 ethaddr=04:00:00:00:00:0A fdtfile=vlm-boards/17652/dtb loads_echo=1 netdev=eth0 nfsboot=setenv bootargs root=/dev/nfs rw nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip:$gatewayip:$netmask:$hostname:$netdev:off console=$consoledev,$baudrate $othbootargs;tftp $loadaddr $bootfile;tftp $fdtaddr $fdtfile;bootm $loadaddr - $fdtaddr ramboot=setenv bootargs root=/dev/ram rw console=$consoledev,$baudrate $othbootargs;tftp $ramdiskaddr $ramdiskfile;tftp $loadaddr $bootfile;tftp $fdtaddr $fdtfile;bootm $loadaddr $ramdiskaddr $fdtaddr ramdiskaddr=1000000 ramdiskfile=ramfs.83xx rootpath=/export/pxeboot/vlm-boards/17652/rootfs/ usb_phy_type=utmi bootargs=root=/dev/nfs rw nfsroot=128.224.165.20:/export/pxeboot/vlm-boards/17652/rootfs/ ip=128.224.178.141:128.224.165.20:128.224.178.1:::eth0:off console=ttyS0,115200 filesize=521A fileaddr=800000 gatewayip=128.224.178.1 ipaddr=128.224.178.141 serverip=128.224.165.20 fdtaddr=2000000 bootm_low=4000000 loadaddr=3000000 stdin=serial stdout=serial stderr=serial ###############
I'll look at this, since I did the uprev to 3.4 and it passed on my board. It may be that the test board's u-boot is too old...
Created attachment 644 [details] Log file of using u-boot to reflash over itself. Be careful when using u-boot to reflash over itself. If you get things wrong, and corrupt flash, you will need an expensive JTAG tool to recover your board! Note the "md" steps are optional; they just allow the user to see what is stored in ram/flash as a double check on things, since getting it right is critical.
I've copied your kernel and your rootfs image locally and they work fine (I'll attach a log) on my board. I even inserted a USB stick since I saw you had one in your log on your board. Sitll all works OK. I'm left thinking one of two things: 1) old u-boot causing some kind of issue, corrupting RAM and causing ftrace failure (the "Not a trampoline" message is from __ftrace_make_nop call). I am using a new u-boot, based on the recent u-boot changes in OE core. 2) Your hard disk is dying; since the board hangs when a udev thread times out on reading sda4. Odd that we don't see kernel driver error messages on What we can try: A) Getting you to boot an older rootfs/kernel, that was known to work for you. If that now fails, then it is probably #2 (try reading /dev/sda4 once it is booted). If it passes that test, then it is probably #1. B) Upgrading your u-boot. I'll put the prebuilt image I'm using somewhere for you and give you upgrade instructions (actually a log file from when I updated mine, showing all the steps.)
Created attachment 645 [details] Log of successful boot, login, verified kernel version. Attached is the boot log; From /proc/version, you can see it is your kernel, and the md5sum of libc should be unique to your rootfs image as well. I got them from /export/pxeboot/vlm-boards/17652 (pek-tuxlab) -------------- 17652$md5sum rootfs/core-image-minimal-mpc8315e-rdb-20120712134431.rootfs.tar.bz2 159020e84d0b898804e9b0c5bf18aa73 rootfs/core-image-minimal-mpc8315e-rdb-20120712134431.rootfs.tar.bz2 17652$pwd /export/pxeboot/vlm-boards/17652 17652$md5sum kernel 7c61a2536e15a50ff94e6af9999e2742 kernel 17652$md5sum dtb b100dc9ca62d20161b41b8437fb1af4f dtb 17652$ ----------------- I've put the u-boot binary in /var/tmp on pek-tuxlab as well: 17652$md5sum /var/tmp/u-boot-mpc8315e-rdb.bin ad2373c5a3de8a5dd14ef91db997bf6d /var/tmp/u-boot-mpc8315e-rdb.bin 17652$
(In reply to comment #3) > I've copied your kernel and your rootfs image locally and they work fine > (I'll attach a log) on my board. I even inserted a USB stick since I saw > you had one in your log on your board. Sitll all works OK. > > I'm left thinking one of two things: > > 1) old u-boot causing some kind of issue, corrupting RAM and causing ftrace > failure (the "Not a trampoline" message is from __ftrace_make_nop call). I > am using a new u-boot, based on the recent u-boot changes in OE core. > > 2) Your hard disk is dying; since the board hangs when a udev thread times > out > on reading sda4. Odd that we don't see kernel driver error messages on > > What we can try: > > A) Getting you to boot an older rootfs/kernel, that was known to work for > you. If that now fails, then it is probably #2 (try reading /dev/sda4 once > it is booted). If it passes that test, then it is probably #1. > I tried to boot an older rootfs/kernel with 1.3 M1 (kernel version: Linux mpc8315e-rdb 3.0.32-yocto-standard #1 PREEMPT Wed Jun 13 17:39:56 PDT 2012 ppc GNU/Linux). It didn't fail, but there were some error messages for sda4 when booting: ############ Starting udev udev[77]: starting version 164 talitos e0030000.crypto: hwrng talitos e0030000.crypto: authenc-hmac-sha1-cbc-aes-talitos talitos e0030000.crypto: authenc-hmac-sha1-cbc-3des-talitos talitos e0030000.crypto: authenc-hmac-sha256-cbc-aes-talitos talitos e0030000.crypto: authenc-hmac-sha256-cbc-3des-talitos talitos e0030000.crypto: authenc-hmac-md5-cbc-aes-talitos talitos e0030000.crypto: authenc-hmac-md5-cbc-3des-talitos talitos e0030000.crypto: cbc-aes-talitos talitos e0030000.crypto: cbc-3des-talitos talitos e0030000.crypto: md5-talitos talitos e0030000.crypto: sha1-talitos talitos e0030000.crypto: sha224-talitos talitos e0030000.crypto: sha256-talitos talitos e0030000.crypto: sha384-talitos talitos e0030000.crypto: sha512-talitos EXT3-fs (sda): error: can't find ext3 filesystem on dev sda. EXT2-fs (sda): error: can't find an ext2 filesystem on dev sda. FAT-fs (sda): invalid media value (0x00) FAT-fs (sda): Can't find a valid FAT filesystem EXT3-fs (sdb): error: can't find ext3 filesystem on dev sdb. EXT2-fs (sdb): error: can't find an ext2 filesystem on dev sdb. FAT-fs (sdb): bogus number of reserved sectors FAT-fs (sdb): Can't find a valid FAT filesystem VFS: Can't find a Minix filesystem V1 | V2 | V3 on device sda. FAT-fs (sda): invalid media value (0x00) FAT-fs (sda): Can't find a valid FAT filesystem VFS: Can't find a Minix filesystem V1 | V2 | V3 on device sdb. FAT-fs (sdb): bogus number of reserved sectors FAT-fs (sdb): Can't find a valid FAT filesystem attempt to access beyond end of device sda4: rw=0, want=4, limit=2 EXT3-fs (sda4): error: unable to read superblock attempt to access beyond end of device sda4: rw=0, want=4, limit=2 EXT2-fs (sda4): error: unable to read superblock FAT-fs (sda4): bogus number of reserved sectors FAT-fs (sda4): Can't find a valid FAT filesystem attempt to access beyond end of device sda4: rw=0, want=4, limit=2 MINIX-fs: unable to read superblock FAT-fs (sda4): bogus number of reserved sectors FAT-fs (sda4): Can't find a valid FAT filesystem EXT2-fs (sdb1): warning: mounting unchecked fs, running e2fsck is recommended EXT3-fs: barriers not enabled EXT3-fs (sda1): warning: maximal mount count reached, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs (sda1): using internal journal EXT3-fs (sda1): mounted filesystem with ordered data mode EXT3-fs: barriers not enabled kjournald starting. Commit interval 5 seconds EXT3-fs (sda2): using internal journal EXT3-fs (sda2): mounted filesystem with ordered data mode EXT3-fs: barriers not enabled EXT3-fs (sda3): warning: mounting unchecked fs, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs: barriers not enabled EXT3-fs: barriers not enabled kjournald starting. Commit interval 5 seconds EXT3-fs (sda5): using internal journal EXT3-fs (sda3): using internal journal EXT3-fs (sda3): mounted filesystem with ordered data mode EXT3-fs (sda5): mounted filesystem with ordered data mode EXT3-fs (sda6): warning: maximal mount count reached, running e2fsck is recommended kjournald starting. Commit interval 5 seconds EXT3-fs (sda6): using internal journal EXT3-fs (sda6): mounted filesystem with ordered data mode ############ The sda4 is a extended partition so it can't be mounted. ########### root@mpc8315e-rdb:~# fdisk -l /dev/sda Disk /dev/sda: 160.0 GB, 160041885696 bytes 255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x09b10400 Device Boot Start End Blocks Id System /dev/sda1 63 58593750 29296844 83 Linux /dev/sda2 58595328 117186559 29295616 83 Linux /dev/sda3 117186560 175781887 29297664 83 Linux /dev/sda4 175781888 312576704 68397408+ 5 Extended /dev/sda5 175781951 238693769 31455909+ 83 Linux /dev/sda6 238695818 312576704 36940443+ 83 Linux ########## I will try it again after upgrading u-boot. > B) Upgrading your u-boot. I'll put the prebuilt image I'm using somewhere > for you and give you upgrade instructions (actually a log file from when I > updated mine, showing all the steps.)
We'd like to close on this for M2 if possible. Paul's u-boot fix was merged this morning. Yi Zhao, does this resolve the issue for you?
Marking as M2 as we still have one week left of "stabilization" according to the calendar.
Given that Yi reports "there were some error messages for sda4 when booting" on the old version (pre-3.4 uprev), I'm reasonably confident he's got a hardware issue. Plus we have the fact that the exact same binary builds boot fine on my hardware. Given that, I'm adjusting the priority accordingly.
Yi, the next/other trivial test you could do, is to boot something that gets to a shell; use fdisk to delete and recreate the offending partitions, and then mke2fs the remade partitions. Finally, in the same vein you could put smartmontools in a package list, build that, and get it so that you have "smartctl" on the target rootfs. This would allow you to run something like "smartctl -t long /dev/sda" as a disk sanity test. But at the moment, this is looking like a hardware fail.
After updated the u-boot, I re-tested it with *sato-sdk* image (not minimal image). The system could boot and I ran the weekly test on it. All cases could pass. But I found there are 2 issues: 1. There is a module crashing message when booting: ############# udev[79]: starting version 164 Not a trampoline WARNING: at /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-ppc/build/build/tmp/work/mpc8315e_rdb-poky-linux/linux-yocto-3.4.4+git1+a8cf77018b0faa0d29f1483ff4e5a2034dc8edd5_1+47d3076aa7e2a640a1dab8d663529a784709f697-r4.0/linux/kernel/trace/ftrace.c:1583 Modules linked in: rng_core(+) NIP: c008c98c LR: c008d05c CTR: c0409378 REGS: c71c9d20 TRAP: 0700 Not tainted (3.4.4-yocto-standard) MSR: 00029032 <EE,ME,IR,DR,RI> CR: 22228482 XER: 20000000 TASK = c7185100[133] 'modprobe' THREAD: c71c8000 GPR00: c008d05c c71c9dd0 c7185100 ffffffea c9035008 ffffffff c0394ea0 00000014 GPR08: 00000004 c07be3d5 00000001 0001ffff 22228424 10029140 00020000 1000d394 GPR16: bffd475c c9035008 c07d0000 00000000 c0010bb0 00000000 c078dcf4 c07fb6a8 GPR24: c07cc114 c7a4fca0 c9035b00 00000000 c72c4000 00000000 00000001 c9035008 NIP [c008c98c] ftrace_bug+0xb8/0x240 LR [c008d05c] ftrace_process_locs+0x410/0x5b0 Call Trace: [c71c9dd0] [c0094e4c] ftrace_now+0x34/0x80 (unreliable) [c71c9df0] [c008d05c] ftrace_process_locs+0x410/0x5b0 [c71c9e40] [c008ea38] ftrace_module_notify+0x6c/0x80 [c71c9e50] [c05dc8c4] notifier_call_chain+0x78/0xc0 [c71c9e70] [c0049df4] __blocking_notifier_call_chain+0x5c/0x80 [c71c9e90] [c006d6e0] sys_init_module+0x9b8/0x1ae8 [c71c9f40] [c0010298] ret_from_syscall+0x0/0x38 --- Exception: c01 at 0xf4a551c LR = 0xff99fe0 Instruction dump: 7fe4fb78 38630ee8 7fe5fb78 80010024 bb410008 38210020 7c0803a6 4855441c 3d20c07c 3929e3d5 89490003 694a0001 <0f0a0000> 2f8a0000 409e015c 3d20c07d ---[ end trace d62b350335b2e5aa ]--- ftrace failed to modify [<c9035008>] rng_dev_open+0x8/0x50 [rng_core] actual: 48:00:08:29 talitos e0030000.crypto: hwrng ############### 2. When I created a extended partition and added a logical partition on the harddisk, the udev would print some failed message. ####### udevadm settle - timeout of 180 seconds reached, the event queue contains: /sys/devices/e0000000.immr/e0018000.sata/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda4 (1082) Starting Bootlog daemon: bootlogd. udevd[79]: worker [88] unexpectedly returned with status 0x0100 udevd[79]: worker [88] failed while handling '/devices/e0000000.immr/e0018000.sata/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda4' ######## There is no such error when only primary partition on the harddisk. This issue happened on our 2 mpc8315e-rdb boards (barcode 17649, 17652). So I don't think it's just a hardware fail. I attached the detail boot log.
Created attachment 650 [details] boot log without extended partition Here is the log when there is no extended partition on harddisk
Created attachment 651 [details] boot log with extended partition Boot log when there is a extended partition on harddisk.
The original issue appears to be resolved. The two issues you describe above should be opened as new bug reports. I'm not sure either warrants a "critical" severity. I suggest marking this bug as resolved+fixed as the uboot upgrade from Paul has resolved the original reported boot issue. Paul, would you agree?
1) The udev issue definitely does look to be independent of the kernel uprev, and it also seems arch independent -- Fedora (x86) is tracking the same bug here: https://bugzilla.redhat.com/show_bug.cgi?id=670964 Based on the two logfiles recently attached, it is almost as if udev is doing something silly like trying to read the short extended partition... Not looking to be a kernel issue, anyway. 2) I've managed to solve (I think) the ftrace problem. It was introduced by the 3.4.3 stable release taking on a cherry pick of 3c75296562f43e ("powerpc: Fix kernel panic during kernel module load") which in turn also requires: ------- commit fd5a42980e1cf327b7240adf5e7b51ea41c23437 Author: roger blofeld <blofeldus@yahoo.com> powerpc/ftrace: Fix assembly trampoline register usage Just like the module loader, ftrace needs to be updated to use r12 instead of r11 with newer gcc's. ------- I can't reproduce the bug locally, but the above seems pretty cut and dry. I will make a request to have the above put in the next 3.4.x stable release. Darren, given the above, I'd say making a new udev bug might make sense, and if you wanted, you could leave this open until the ftrace fix loops around via stable. Or whatever you think, I have no special preference.
Yi, can you please open a new bug to track the udev issue? Will leave this bug open to track the ftrace fix Paul referenced, and close once merged (it isn't in linux-yocto-3.4 as of just now).
As Paul has been working this and has committed to getting the ftrace patch merged, marking as accepted.
(In reply to comment #15) > Yi, can you please open a new bug to track the udev issue? > I have opened a new bug 2855 to track it. > Will leave this bug open to track the ftrace fix Paul referenced, and close > once merged (it isn't in linux-yocto-3.4 as of just now).
This was picked up by Greg-kh, so we'll pull it in via the next 3.4.x stable update: http://git.kernel.org/?p=linux/kernel/git/stable/stable-queue.git;a=blob;f=queue-3.4/powerpc-ftrace-fix-assembly-trampoline-register-usage.patch;h=7f10db41986ff32243c8e4ee91349fc188158542;hb=0215b418053819f902759726c7c05c30ab93fc34
OK, there is really nothing more to do here then. We could leave it open and wait for the stable release - but then we'd have to discuss this bug repeatedly as it's targeted for M3. I don't think the ftrace issue merits a new bug. If someone brings it up, I think we can address it via email easily enough. Closing this out as the original issue is resolved and the additional two are adequately accounted for.
The kernel has been updated to 3.4.7 in master branch, but the ftrace module crash issue still happens. So I reopen it. gitinfo: poky/master: 19ff55a53b436d463f9577adaed15be77708f0d2 ################### root@mpc8315e-rdb:~# uname -a Linux mpc8315e-rdb 3.4.7-yocto-standard #1 PREEMPT Wed Aug 22 09:50:27 PDT 2012 ppc GNU/Linux ################# The crash log: ############### Starting udev udev[83]: starting version 164 Not a trampoline ------------[ cut here ]------------ WARNING: at /srv/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-ppc/build/build/tmp/work/mpc8315e_rdb-poky-linux/linux-yocto-3.4.7+git6+28bcd46af1d592dab39bd8a0891c872454fde8bc_3+f10500e9faad3f9f799137a3159ff5c836c88227-r4.0/linux/kernel/trace/ftrace.c:1583 Modules linked in: rng_core(+) NIP: c00957c4 LR: c0095e94 CTR: c041240c REGS: c6c99d20 TRAP: 0700 Not tainted (3.4.7-yocto-standard) MSR: 00029032 <EE,ME,IR,DR,RI> CR: 22228482 XER: 20000000 TASK = c6d13740[142] 'modprobe' THREAD: c6c98000 GPR00: c0095e94 c6c99dd0 c6d13740 ffffffea c9035008 ffffffff c039df64 00000014 GPR08: 00000004 c07ca496 00000001 0001ffff 22228422 10029140 00020000 bfd3899c GPR16: 00000000 c9035008 c07e0000 c0010bd0 00000000 00000000 c0799dcc c084a118 GPR24: c07d8114 c7a5fb50 c9035b00 00000000 c6d0d000 00000000 00000001 c9035008 NIP [c00957c4] ftrace_bug+0xb8/0x240 LR [c0095e94] ftrace_process_locs+0x410/0x5b0 Call Trace: [c6c99dd0] [c009dcb8] ftrace_now+0x34/0x80 (unreliable) [c6c99df0] [c0095e94] ftrace_process_locs+0x410/0x5b0 [c6c99e40] [c0097870] ftrace_module_notify+0x6c/0x80 [c6c99e50] [c05e59d4] notifier_call_chain+0x78/0xc0 [c6c99e70] [c004a280] __blocking_notifier_call_chain+0x5c/0x80 [c6c99e90] [c006dcd0] sys_init_module+0x9b8/0x1ae8 [c6c99f40] [c00102b8] ret_from_syscall+0x0/0x38 --- Exception: c01 at 0xff05ac0 LR = 0xfdfa020 Instruction dump: 7fe4fb78 3863aa20 7fe5fb78 80010024 bb410008 38210020 7c0803a6 48554700 3d20c07d 3929a496 89490003 694a0001 <0f0a0000> 2f8a0000 409e015c 3d20c07e ---[ end trace 26c47aef7f811a84 ]--- ftrace failed to modify [<c9035008>] rng_dev_open+0x8/0x50 [rng_core] actual: 48:00:08:29 ############
You've jumped the gun on testing this. It wasn't merged until 3.4.8 and I see your kernel that you've used for testing is 3.4.7 - so you don't have the patch. ~/git/stable/stable-queue$ git grep -l 'Not a trampoline' . releases/3.0.40/powerpc-ftrace-fix-assembly-trampoline-register-usage.patch releases/3.4.8/powerpc-ftrace-fix-assembly-trampoline-register-usage.patch releases/3.5.1/powerpc-ftrace-fix-assembly-trampoline-register-usage.patch ~/git/stable/stable-queue$
Yi, The master branch now uses the 3.4.10 kernel for the routerstationpro. Can you please retest and verify the fix?
There is no this issue on 1.3_M4 20120910 build. Branch: poky/1.3_M4 Gitinfo: dd3edc9fbb674682947e8218202fdbeef8635dd5 Image from http://autobuilder.yoctoproject.org/pub/nightly/20120910-5/machines/mpc8315e-rdb/ ################ root@mpc8315e-rdb:~# dmesg |grep ftrace ftrace: allocating 19529 entries in 58 pages root@mpc8315e-rdb:~# root@mpc8315e-rdb:~# uname -a Linux mpc8315e-rdb 3.4.9-yocto-standard #1 PREEMPT Mon Sep 10 18:12:55 PDT 2012 ppc GNU/Linux root@mpc8315e-rdb:~# ################
Verified.