Would it be possible to retest this with 6.12.5 (patches in master-next)?

Does the specific beaglebone black BSP boot and work ?

For our reference boards, we (Kevin) have tested the beaglebone black and it works, but we don't test/use the genericarm64 BSP for that.

"paulg: a quick glance at the kernel barfage and I'm thinking bad config.  You can't tie your UART to critical printk path and then try and power down the thing."

It is possible there is some config in there to do with PM of the UARTs which are being used for console messaging...

If we can test this and prove that we have a fix in master, then we can release M1.

FYI, Trevor Gamblin may have a CI pipeline to do continuous builds and tests for this board but it's not setup just yet.

I can confirm that master is still broken.

After a few 'scheduling while atomic' messages the MMC then dies too:

mmc1: Timeout waiting for hardware interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000010 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0x000bd140 | Trn mode: 0x0000002b
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x00000000 | Int stat: 0x00000003
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x0000193a | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x01dbd37f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x0000000b
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x0000000089670218
mmc1: sdhci: ============================================
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (1 ticks this GP) idle=b5bc/1/0x4000000000000002 softirq=3383/3383 fqs=2
rcu: 	(detected by 2, t=5255 jiffies, g=3573, q=282 ncpus=4)
Sending NMI from CPU 2 to CPUs 0:
mmc1: Timeout waiting for hardware interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000010 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0x00000000 | Trn mode: 0x00000023
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x0000000e | Int stat: 0x00018002
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x00000c1b | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x01dbd37f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x0000000b
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x0000000089670218
mmc1: sdhci: ============================================
rcu: rcu_preempt kthread timer wakeup didn't happen for 7737 jiffies! g3573 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=1 timer-softirq=2407
rcu: rcu_preempt kthread starved for 7742 jiffies! g3573 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
Call trace:
 __switch_to+0xe0/0x148
 __schedule+0x380/0xcf8
 schedule+0x40/0x158
 schedule_timeout+0xb0/0x1b0
 rcu_gp_fqs_loop+0x140/0x760
 rcu_gp_kthread+0x244/0x2a0
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 2 to CPUs 1:
mmc1: Timeout waiting for hardware cmd interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000010 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0xaaaa0000 | Trn mode: 0x00000023
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x0000000e | Int stat: 0x00018001
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x00000d1a | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00400900 | Resp[1]:  0x01dbd37f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x0000000b
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x0000000089670218
mmc1: sdhci: ============================================
mmc1: Timeout waiting for hardware cmd interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000010 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0xaaaa0000 | Trn mode: 0x00000023
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x0000000e | Int stat: 0x00018001
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x00000d1a | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x01dbd37f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x0000000b
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x0000000089670218
mmc1: sdhci: ============================================
mmc1: Timeout waiting for hardware cmd interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000010 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0x00000000 | Trn mode: 0x00000023
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x0000000e | Int stat: 0x00018002
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x00000c1a | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x01dbd37f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x0000000b
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x0000000089670218
mmc1: sdhci: ============================================
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...0: (1 ticks this GP) idle=b5bc/1/0x4000000000000002 softirq=3383/3383 fqs=1847
rcu: 	(detected by 2, t=26032 jiffies, g=3573, q=919 ncpus=4)
Sending NMI from CPU 2 to CPUs 0:
rcu: rcu_preempt kthread starved for 14546 jiffies! g3573 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=2
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
Call trace:
 __switch_to+0xe0/0x148
 __schedule+0x380/0xcf8
 schedule+0x40/0x158
 schedule_timeout+0xb0/0x1b0
 rcu_gp_fqs_loop+0x140/0x760
 rcu_gp_kthread+0x244/0x2a0
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: Stack dump where RCU GP kthread last ran:
CPU: 2 UID: 0 PID: 64 Comm: kworker/2:1 Tainted: G        W          6.12.8-yocto-standard #1
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Workqueue: events bpf_prog_free_deferred
pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : smp_call_function_many_cond+0x1a4/0x690
lr : smp_call_function_many_cond+0x160/0x690
sp : ffff80008278bb80
x29: ffff80008278bb80 x28: 0000000000000002 x27: 0000000000000004
x26: ffff800081ca8ea0 x25: ffff00007f99c840 x24: ffff80008013d228
x23: ffff00007f9792c0 x22: ffff800081caa6d8 x21: ffff800081be7840
x20: 0000000000000000 x19: ffff800081caa6d8 x18: ffff800082665060
x17: ffff000001019cc0 x16: 0000000000000000 x15: 0000000000001000
x14: 0000000000000000 x13: 0000000000000030 x12: 0101010101010101
x11: 7f7f7f7f7f7f7f7f x10: fefefefefefefeff x9 : ffff8000808a44fc
x8 : ffff00000040f1f8 x7 : 0000000000000000 x6 : ffff800081ca8000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff00007f93f848
x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
Call trace:
 smp_call_function_many_cond+0x1a4/0x690
 kick_all_cpus_sync+0x50/0x88
 __text_poke+0xe4/0x118
 aarch64_insn_set+0x34/0x70
 bpf_arch_text_invalidate+0x28/0x50
 bpf_prog_pack_free+0xfc/0x210
 bpf_jit_binary_pack_free+0x2c/0x140
 bpf_jit_free+0x5c/0xa8
 bpf_prog_free_deferred+0x138/0x170
 process_one_work+0x19c/0x408
 worker_thread+0x25c/0x380
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...0: (1 ticks this GP) idle=b5bc/1/0x4000000000000002 softirq=3383/3383 fqs=1847
rcu: 	(detected by 2, t=44339 jiffies, g=3573, q=919 ncpus=4)
Sending NMI from CPU 2 to CPUs 0:
rcu: rcu_preempt kthread starved for 32853 jiffies! g3573 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=2
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
Call trace:
 __switch_to+0xe0/0x148
 __schedule+0x380/0xcf8
 schedule+0x40/0x158
 schedule_timeout+0xb0/0x1b0
 rcu_gp_fqs_loop+0x140/0x760
 rcu_gp_kthread+0x244/0x2a0
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: Stack dump where RCU GP kthread last ran:
CPU: 2 UID: 0 PID: 64 Comm: kworker/2:1 Tainted: G        W          6.12.8-yocto-standard #1
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Workqueue: events bpf_prog_free_deferred
pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : smp_call_function_many_cond+0x1a4/0x690
lr : smp_call_function_many_cond+0x160/0x690
sp : ffff80008278bb80
x29: ffff80008278bb80 x28: 0000000000000002 x27: 0000000000000004
x26: ffff800081ca8ea0 x25: ffff00007f99c840 x24: ffff80008013d228
x23: ffff00007f9792c0 x22: ffff800081caa6d8 x21: ffff800081be7840
x20: 0000000000000000 x19: ffff800081caa6d8 x18: ffff800082665060
x17: ffff000001019cc0 x16: 0000000000000000 x15: 0000000000001000
x14: 0000000000000000 x13: 0000000000000030 x12: 0101010101010101
x11: 7f7f7f7f7f7f7f7f x10: fefefefefefefeff x9 : ffff8000808a44fc
x8 : ffff00000040f1f8 x7 : 0000000000000000 x6 : ffff800081ca8000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff00007f93f848
x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
Call trace:
 smp_call_function_many_cond+0x1a4/0x690
 kick_all_cpus_sync+0x50/0x88
 __text_poke+0xe4/0x118
 aarch64_insn_set+0x34/0x70
 bpf_arch_text_invalidate+0x28/0x50
 bpf_prog_pack_free+0xfc/0x210
 bpf_jit_binary_pack_free+0x2c/0x140
 bpf_jit_free+0x5c/0xa8
 bpf_prog_free_deferred+0x138/0x170
 process_one_work+0x19c/0x408
 worker_thread+0x25c/0x380
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...0: (1 ticks this GP) idle=b5bc/1/0x4000000000000002 softirq=3383/3383 fqs=1847
rcu: 	(detected by 2, t=62646 jiffies, g=3573, q=919 ncpus=4)
Sending NMI from CPU 2 to CPUs 0:
rcu: rcu_preempt kthread starved for 51160 jiffies! g3573 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=2
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
Call trace:
 __switch_to+0xe0/0x148
 __schedule+0x380/0xcf8
 schedule+0x40/0x158
 schedule_timeout+0xb0/0x1b0
 rcu_gp_fqs_loop+0x140/0x760
 rcu_gp_kthread+0x244/0x2a0
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: Stack dump where RCU GP kthread last ran:
CPU: 2 UID: 0 PID: 64 Comm: kworker/2:1 Tainted: G        W          6.12.8-yocto-standard #1
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Workqueue: events bpf_prog_free_deferred
pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : smp_call_function_many_cond+0x1a4/0x690
lr : smp_call_function_many_cond+0x160/0x690
sp : ffff80008278bb80
x29: ffff80008278bb80 x28: 0000000000000002 x27: 0000000000000004
x26: ffff800081ca8ea0 x25: ffff00007f99c840 x24: ffff80008013d228
x23: ffff00007f9792c0 x22: ffff800081caa6d8 x21: ffff800081be7840
x20: 0000000000000000 x19: ffff800081caa6d8 x18: ffff800082665060
x17: ffff000001019cc0 x16: 0000000000000000 x15: 0000000000001000
x14: 0000000000000000 x13: 0000000000000030 x12: 0101010101010101
x11: 7f7f7f7f7f7f7f7f x10: fefefefefefefeff x9 : ffff8000808a44fc
x8 : ffff00000040f1f8 x7 : 0000000000000000 x6 : ffff800081ca8000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff00007f93f848
x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
Call trace:
 smp_call_function_many_cond+0x1a4/0x690
 kick_all_cpus_sync+0x50/0x88
 __text_poke+0xe4/0x118
 aarch64_insn_set+0x34/0x70
 bpf_arch_text_invalidate+0x28/0x50
 bpf_prog_pack_free+0xfc/0x210
 bpf_jit_binary_pack_free+0x2c/0x140
 bpf_jit_free+0x5c/0xa8
 bpf_prog_free_deferred+0x138/0x170
 process_one_work+0x19c/0x408
 worker_thread+0x25c/0x380
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...0: (1 ticks this GP) idle=b5bc/1/0x4000000000000002 softirq=3383/3383 fqs=1847
rcu: 	(detected by 2, t=80953 jiffies, g=3573, q=919 ncpus=4)
Sending NMI from CPU 2 to CPUs 0:
rcu: rcu_preempt kthread starved for 69467 jiffies! g3573 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=2
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
Call trace:
 __switch_to+0xe0/0x148
 __schedule+0x380/0xcf8
 schedule+0x40/0x158
 schedule_timeout+0xb0/0x1b0
 rcu_gp_fqs_loop+0x140/0x760
 rcu_gp_kthread+0x244/0x2a0
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: Stack dump where RCU GP kthread last ran:
CPU: 2 UID: 0 PID: 64 Comm: kworker/2:1 Tainted: G        W          6.12.8-yocto-standard #1
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Workqueue: events bpf_prog_free_deferred
pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : smp_call_function_many_cond+0x1a4/0x690
lr : smp_call_function_many_cond+0x160/0x690
sp : ffff80008278bb80
x29: ffff80008278bb80 x28: 0000000000000002 x27: 0000000000000004
x26: ffff800081ca8ea0 x25: ffff00007f99c840 x24: ffff80008013d228
x23: ffff00007f9792c0 x22: ffff800081caa6d8 x21: ffff800081be7840
x20: 0000000000000000 x19: ffff800081caa6d8 x18: ffff800082665060
x17: ffff000001019cc0 x16: 0000000000000000 x15: 0000000000001000
x14: 0000000000000000 x13: 0000000000000030 x12: 0101010101010101
x11: 7f7f7f7f7f7f7f7f x10: fefefefefefefeff x9 : ffff8000808a44fc
x8 : ffff00000040f1f8 x7 : 0000000000000000 x6 : ffff800081ca8000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff00007f93f848
x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
Call trace:
 smp_call_function_many_cond+0x1a4/0x690
 kick_all_cpus_sync+0x50/0x88
 __text_poke+0xe4/0x118
 aarch64_insn_set+0x34/0x70
 bpf_arch_text_invalidate+0x28/0x50
 bpf_prog_pack_free+0xfc/0x210
 bpf_jit_binary_pack_free+0x2c/0x140
 bpf_jit_free+0x5c/0xa8
 bpf_prog_free_deferred+0x138/0x170
 process_one_work+0x19c/0x408
 worker_thread+0x25c/0x380
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...0: (1 ticks this GP) idle=b5bc/1/0x4000000000000002 softirq=3383/3383 fqs=1847
rcu: 	(detected by 2, t=99260 jiffies, g=3573, q=919 ncpus=4)
Sending NMI from CPU 2 to CPUs 0:
rcu: rcu_preempt kthread starved for 87774 jiffies! g3573 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=2
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
Call trace:
 __switch_to+0xe0/0x148
 __schedule+0x380/0xcf8
 schedule+0x40/0x158
 schedule_timeout+0xb0/0x1b0
 rcu_gp_fqs_loop+0x140/0x760
 rcu_gp_kthread+0x244/0x2a0
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20
rcu: Stack dump where RCU GP kthread last ran:
CPU: 2 UID: 0 PID: 64 Comm: kworker/2:1 Tainted: G        W          6.12.8-yocto-standard #1
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Workqueue: events bpf_prog_free_deferred
pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : smp_call_function_many_cond+0x1a4/0x690
lr : smp_call_function_many_cond+0x160/0x690
sp : ffff80008278bb80
x29: ffff80008278bb80 x28: 0000000000000002 x27: 0000000000000004
x26: ffff800081ca8ea0 x25: ffff00007f99c840 x24: ffff80008013d228
x23: ffff00007f9792c0 x22: ffff800081caa6d8 x21: ffff800081be7840
x20: 0000000000000000 x19: ffff800081caa6d8 x18: ffff800082665060
x17: ffff000001019cc0 x16: 0000000000000000 x15: 0000000000001000
x14: 0000000000000000 x13: 0000000000000030 x12: 0101010101010101
x11: 7f7f7f7f7f7f7f7f x10: fefefefefefefeff x9 : ffff8000808a44fc
x8 : ffff00000040f1f8 x7 : 0000000000000000 x6 : ffff800081ca8000
x5 : 0000000000000000 x4 : 0000000000000000 x3 : ffff00007f93f848
x2 : 0000000000000000 x1 : 0000000000000011 x0 : 0000000000000000
Call trace:
 smp_call_function_many_cond+0x1a4/0x690
 kick_all_cpus_sync+0x50/0x88
 __text_poke+0xe4/0x118
 aarch64_insn_set+0x34/0x70
 bpf_arch_text_invalidate+0x28/0x50
 bpf_prog_pack_free+0xfc/0x210
 bpf_jit_binary_pack_free+0x2c/0x140
 bpf_jit_free+0x5c/0xa8
 bpf_prog_free_deferred+0x138/0x170
 process_one_work+0x19c/0x408
 worker_thread+0x25c/0x380
 kthread+0x11c/0x128
 ret_from_fork+0x10/0x20

AMD kv260 does not see this issue with genericarm64 machine core-image-sato and kernel 6.12.8 from last week. Will also check 6.12.9 once builds succeed in my queues.

Seeing rcu stalls and reproducible hang on high IO load on rockpi4b using qemuarm64 based machine and upstream kernel.org arm64 defconfig with some amendments. These did not happen on 6.6.35 (old poky master snapshot).

https://ledge.validation.linaro.org/scheduler/job/99855

+ mount /dev/mmcblk1p7 /mnt -o rw,noatime,iversion
[    9.258121] mmc_host mmc1: Bus speed (slot 0) = 400000Hz (slot req 400000Hz, actual 400000HZ div = 0)
[    9.423482] mmc_host mmc1: Bus speed (slot 0) = 50000000Hz (slot req 50000000Hz, actual 50000000HZ div = 0)
[    9.559223] EXT4-fs (mmcblk1p7): mounted filesystem 6091b3a4-ce08-3020-93a6-f755a22ef03b r/w with ordered data mode. Quota mode: disabled.
+ cd /mnt
+ echo 'Backup filesystem to tmpfs ...'
Backup filesystem to tmpfs ...
+ pigz -c
+ tar -cf - .
[   32.238210] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   32.238813] rcu: 	4-...0: (1 GPs behind) idle=57d4/1/0x4000000000000000 softirq=1002/1004 fqs=2590
[   32.239648] rcu: 	(detected by 1, t=5252 jiffies, g=221, q=56 ncpus=6)
[   32.240257] Sending NMI from CPU 1 to CPUs 4:
[   42.241525] rcu: rcu_preempt kthread starved for 2499 jiffies! g221 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=3
[   42.242841] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[   42.243649] rcu: RCU grace-period kthread stack dump:
[   42.244100] task:rcu_preempt     state:I stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
[   42.244933] Call trace:
[   42.245158]  __switch_to+0xe8/0x148
[   42.245487]  __schedule+0x2c8/0xc08
[   42.245808]  schedule+0x3c/0x120
[   42.246105]  schedule_timeout+0x9c/0x1a8
[   42.246474]  rcu_gp_fqs_loop+0x12c/0x6e0
[   42.246862]  rcu_gp_kthread+0x210/0x260
[   42.247240]  kthread+0x118/0x130
[   42.247561]  ret_from_fork+0x10/0x20
[   42.247913] rcu: Stack dump where RCU GP kthread last ran:
[   42.248420] Sending NMI from CPU 1 to CPUs 3:
[   42.248917] NMI backtrace for cpu 3
[   42.249262] CPU: 3 UID: 0 PID: 0 Comm: swapper/3 Not tainted 6.12.9-yocto-standard #1
[   42.249990] Hardware name: radxa Radxa ROCK Pi 4A/Radxa ROCK Pi 4A, BIOS 2025.01-dirty 01/01/2025
[   42.250801] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   42.251448] pc : cpuidle_enter_state+0xbc/0x4e0
[   42.251886] lr : cpuidle_enter_state+0x270/0x4e0
[   42.252328] sp : ffff80008021bd90
[   42.252646] x29: ffff80008021bd90 x28: 0000000000000000 x27: 0000000000000000
[   42.253325] x26: 0000000000000001 x25: 0000000000000002 x24: 00000009d611a949
[   42.254003] x23: ffff1e3e019bb080 x22: 0000000000000002 x21: ffff1e3ef55074b8
[   42.254681] x20: 00000009d63b2979 x19: 0000000000000002 x18: 0000000000000000
[   42.255356] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000
[   42.256034] x14: ffff1e3e00e22300 x13: ffff5d37cab20000 x12: 0000000034d4d91d
[   42.256711] x11: 0000000000000000 x10: 0000000000001000 x9 : ffffc10728157a04
[   42.257389] x8 : 00000072b5503510 x7 : 0000000000000004 x6 : 000000011d833e13
[   42.258066] x5 : 00ffffffffffffff x4 : 0000000000c49ba6 x3 : ffff80008021bcc0
[   42.258744] x2 : 0000000159f09adf x1 : ffff1e3e00e22300 x0 : 0000000000000000
[   42.259422] Call trace:
[   42.259662]  cpuidle_enter_state+0xbc/0x4e0
[   42.260068]  cpuidle_enter+0x40/0x60
[   42.260420]  do_idle+0x1ec/0x268
[   42.260741]  cpu_startup_entry+0x40/0x50
[   42.261122]  secondary_start_kernel+0x138/0x168
[   42.261560]  __secondary_switched+0xb8/0xc0
[  105.282211] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  105.282814] rcu: 	4-...0: (1 GPs behind) idle=57d4/1/0x4000000000000000 softirq=1002/1004 fqs=10468
[  105.283657] rcu: 	(detected by 1, t=23513 jiffies, g=221, q=56 ncpus=6)
[  105.284274] Sending NMI from CPU 1 to CPUs 4:
[  115.285541] rcu: rcu_preempt kthread starved for 2500 jiffies! g221 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=5
[  115.286874] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  115.287706] rcu: RCU grace-period kthread stack dump:
[  115.288174] task:rcu_preempt     state:I stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
[  115.289040] Call trace:
[  115.289282]  __switch_to+0xe8/0x148
[  115.289636]  __schedule+0x2c8/0xc08
[  115.289981]  schedule+0x3c/0x120
[  115.290302]  schedule_timeout+0x9c/0x1a8
[  115.290681]  rcu_gp_fqs_loop+0x12c/0x6e0
[  115.291068]  rcu_gp_kthread+0x210/0x260
[  115.291446]  kthread+0x118/0x130
[  115.291767]  ret_from_fork+0x10/0x20
[  115.292118] rcu: Stack dump where RCU GP kthread last ran:
[  115.292625] Sending NMI from CPU 1 to CPUs 5:
[  115.293094] NMI backtrace for cpu 5
[  115.293418] CPU: 5 UID: 0 PID: 0 Comm: swapper/5 Not tainted 6.12.9-yocto-standard #1
[  115.294109] Hardware name: radxa Radxa ROCK Pi 4A/Radxa ROCK Pi 4A, BIOS 2025.01-dirty 01/01/2025
[  115.294885] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  115.295498] pc : cpuidle_enter_state+0x274/0x4e0
[  115.295914] lr : cpuidle_enter_state+0x270/0x4e0
[  115.296325] sp : ffff80008022bd90
[  115.296618] x29: ffff80008022bd90 x28: 0000000000000000 x27: 0000000000000000
[  115.297252] x26: 0000000000000001 x25: 0000000000000002 x24: 0000001ad7d2dda2
[  115.297883] x23: ffff1e3e019bc080 x22: 0000000000000002 x21: ffff1e3ef554b4b8
[  115.298514] x20: 0000001ad800efe7 x19: 0000000000000002 x18: 0000000000000000
[  115.299145] x17: 000000040044ffff x16: 00500074b5503510 x15: 0000000000000000
[  115.299776] x14: ffff1e3e00e24600 x13: ffff5d37cab64000 x12: 0000000034d4d91d
[  115.300407] x11: 0000000000000000 x10: 0000000000001000 x9 : ffffc10728157a04
[  115.301038] x8 : 00000074b5503510 x7 : 0000000000000004 x6 : 000000011d833e13
[  115.301669] x5 : 00ffffffffffffff x4 : 0000000000c49ba6 x3 : ffff80008022bcc0
[  115.302300] x2 : 00000001c26e2ee9 x1 : ffff1e3e00e24600 x0 : 0000000000000000
[  115.302932] Call trace:
[  115.303151]  cpuidle_enter_state+0x274/0x4e0
[  115.303530]  cpuidle_enter+0x40/0x60
[  115.303852]  do_idle+0x1ec/0x268
[  115.304142]  cpu_startup_entry+0x40/0x50
[  115.304493]  secondary_start_kernel+0x138/0x168
[  115.304897]  __secondary_switched+0xb8/0xc0
[  178.326209] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  178.326798] rcu: 	4-...0: (1 GPs behind) idle=57d4/1/0x4000000000000000 softirq=1002/1004 fqs=18343
[  178.327640] rcu: 	(detected by 1, t=41774 jiffies, g=221, q=56 ncpus=6)
[  178.328258] Sending NMI from CPU 1 to CPUs 4:
[  188.329525] rcu: rcu_preempt kthread starved for 2500 jiffies! g221 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=5
[  188.330859] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  188.331690] rcu: RCU grace-period kthread stack dump:
[  188.332160] task:rcu_preempt     state:I stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
[  188.333024] Call trace:
[  188.333265]  __switch_to+0xe8/0x148
[  188.333615]  __schedule+0x2c8/0xc08
[  188.333960]  schedule+0x3c/0x120
[  188.334281]  schedule_timeout+0x9c/0x1a8
[  188.334660]  rcu_gp_fqs_loop+0x12c/0x6e0
[  188.335045]  rcu_gp_kthread+0x210/0x260
[  188.335423]  kthread+0x118/0x130
[  188.335743]  ret_from_fork+0x10/0x20
[  188.336094] rcu: Stack dump where RCU GP kthread last ran:
[  188.336601] Sending NMI from CPU 1 to CPUs 5:
[  188.337069] NMI backtrace for cpu 5
[  188.337385] CPU: 5 UID: 0 PID: 0 Comm: swapper/5 Not tainted 6.12.9-yocto-standard #1
[  188.338074] Hardware name: radxa Radxa ROCK Pi 4A/Radxa ROCK Pi 4A, BIOS 2025.01-dirty 01/01/2025
[  188.338851] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  188.339463] pc : cpuidle_enter_state+0x274/0x4e0
[  188.339875] lr : cpuidle_enter_state+0x270/0x4e0
[  188.340285] sp : ffff80008022bd90
[  188.340579] x29: ffff80008022bd90 x28: 0000000000000000 x27: 0000000000000000
[  188.341212] x26: 0000000000000001 x25: 0000000000000002 x24: 0000002bd995da32
[  188.341843] x23: ffff1e3e019bc080 x22: 0000000000000002 x21: ffff1e3ef554b4b8
[  188.342474] x20: 0000002bd9c39275 x19: 0000000000000002 x18: 0000000000000000
[  188.343105] x17: 000000040044ffff x16: 00500074b5503510 x15: 0000000000000000
[  188.343737] x14: ffff1e3e00e24600 x13: ffff5d37cab64000 x12: 0000000034d4d91d
[  188.344367] x11: 0000000000000000 x10: 0000000000001000 x9 : ffffc10728157a04
[  188.344998] x8 : 00000074b5503510 x7 : 0000000000000004 x6 : 000000011d833e13
[  188.345629] x5 : 00ffffffffffffff x4 : 0000000000c49ba6 x3 : ffff80008022bcc0
[  188.346260] x2 : 000000022aebafbd x1 : ffff1e3e00e24600 x0 : 0000000000000000
[  188.346891] Call trace:
[  188.347110]  cpuidle_enter_state+0x274/0x4e0
[  188.347491]  cpuidle_enter+0x40/0x60
[  188.347811]  do_idle+0x1ec/0x268
[  188.348099]  cpu_startup_entry+0x40/0x50
[  188.348450]  secondary_start_kernel+0x138/0x168
[  188.348852]  __secondary_switched+0xb8/0xc0
[  251.370209] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  251.370799] rcu: 	4-...0: (1 GPs behind) idle=57d4/1/0x4000000000000000 softirq=1002/1004 fqs=26220
[  251.371641] rcu: 	(detected by 2, t=60035 jiffies, g=221, q=56 ncpus=6)
[  251.372257] Sending NMI from CPU 2 to CPUs 4:
[  261.373523] rcu: rcu_preempt kthread starved for 2500 jiffies! g221 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=5
[  261.374854] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  261.375685] rcu: RCU grace-period kthread stack dump:
[  261.376153] task:rcu_preempt     state:I stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
[  261.377017] Call trace:
[  261.377258]  __switch_to+0xe8/0x148
[  261.377610]  __schedule+0x2c8/0xc08
[  261.377956]  schedule+0x3c/0x120
[  261.378278]  schedule_timeout+0x9c/0x1a8
[  261.378657]  rcu_gp_fqs_loop+0x12c/0x6e0
[  261.379043]  rcu_gp_kthread+0x210/0x260
[  261.379421]  kthread+0x118/0x130
[  261.379741]  ret_from_fork+0x10/0x20
[  261.380093] rcu: Stack dump where RCU GP kthread last ran:
[  261.380600] Sending NMI from CPU 2 to CPUs 5:
[  261.381066] NMI backtrace for cpu 5
[  261.381382] CPU: 5 UID: 0 PID: 0 Comm: swapper/5 Not tainted 6.12.9-yocto-standard #1
[  261.382071] Hardware name: radxa Radxa ROCK Pi 4A/Radxa ROCK Pi 4A, BIOS 2025.01-dirty 01/01/2025
[  261.382847] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  261.383459] pc : cpuidle_enter_state+0x274/0x4e0
[  261.383871] lr : cpuidle_enter_state+0x270/0x4e0
[  261.384282] sp : ffff80008022bd90
[  261.384575] x29: ffff80008022bd90 x28: 0000000000000000 x27: 0000000000000000
[  261.385208] x26: 0000000000000001 x25: 0000000000000002 x24: 0000003cdb58d6c2
[  261.385839] x23: ffff1e3e019bc080 x22: 0000000000000002 x21: ffff1e3ef554b4b8
[  261.386471] x20: 0000003cdb8684c4 x19: 0000000000000002 x18: 0000000000000000
[  261.387102] x17: 000000040044ffff x16: 00500074b5503510 x15: 0000000000000000
[  261.387733] x14: ffff1e3e00e24600 x13: ffff5d37cab64000 x12: 0000000034d4d91d
[  261.388364] x11: 0000000000000000 x10: 0000000000001000 x9 : ffffc10728157a04
[  261.388995] x8 : 00000074b5503510 x7 : 0000000000000004 x6 : 000000011d833e13
[  261.389626] x5 : 00ffffffffffffff x4 : 0000000000c49ba6 x3 : ffff80008022bcc0
[  261.390257] x2 : 0000000293693282 x1 : ffff1e3e00e24600 x0 : 0000000000000000
[  261.390888] Call trace:
[  261.391107]  cpuidle_enter_state+0x274/0x4e0
[  261.391486]  cpuidle_enter+0x40/0x60
[  261.391806]  do_idle+0x1ec/0x268
[  261.392095]  cpu_startup_entry+0x3c/0x50
[  261.392445]  secondary_start_kernel+0x138/0x168
[  261.392848]  __secondary_switched+0xb8/0xc0
[  324.414253] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  324.414841] rcu: 	4-...0: (1 GPs behind) idle=57d4/1/0x4000000000000000 softirq=1002/1004 fqs=34097
[  324.415683] rcu: 	(detected by 0, t=78296 jiffies, g=221, q=338 ncpus=6)
[  324.416308] Sending NMI from CPU 0 to CPUs 4:
[  334.417575] rcu: rcu_preempt kthread starved for 2495 jiffies! g221 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=2
[  334.418909] rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  334.419741] rcu: RCU grace-period kthread stack dump:
[  334.420211] task:rcu_preempt     state:I stack:0     pid:18    tgid:18    ppid:2      flags:0x00000008
[  334.421075] Call trace:
[  334.421315]  __switch_to+0xe8/0x148
[  334.421666]  __schedule+0x2c8/0xc08
[  334.422011]  schedule+0x3c/0x120
[  334.422334]  schedule_timeout+0x9c/0x1a8
[  334.422713]  rcu_gp_fqs_loop+0x12c/0x6e0
[  334.423100]  rcu_gp_kthread+0x210/0x260
[  334.423478]  kthread+0x118/0x130
[  334.423798]  ret_from_fork+0x10/0x20
[  334.424150] rcu: Stack dump where RCU GP kthread last ran:
[  334.424657] Sending NMI from CPU 0 to CPUs 2:
[  334.425152] NMI backtrace for cpu 2
[  334.425496] CPU: 2 UID: 0 PID: 0 Comm: swapper/2 Not tainted 6.12.9-yocto-standard #1
[  334.426223] Hardware name: radxa Radxa ROCK Pi 4A/Radxa ROCK Pi 4A, BIOS 2025.01-dirty 01/01/2025
[  334.427034] pstate: 40000005 (nZcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  334.427681] pc : cpuidle_enter_state+0xbc/0x4e0
[  334.428119] lr : cpuidle_enter_state+0x270/0x4e0
[  334.428562] sp : ffff800080213d90
[  334.428879] x29: ffff800080213d90 x28: 0000000000000000 x27: 0000000000000000
[  334.429558] x26: 0000000000000001 x25: 0000000000000002 x24: 0000004ddd1c657a
[  334.430235] x23: ffff1e3e019ba880 x22: 0000000000000002 x21: ffff1e3ef54e54b8
[  334.430913] x20: 0000004ddd4ab8b1 x19: 0000000000000002 x18: 00000000add959ef
[  334.431590] x17: 000000040044ffff x16: 00500072b5503510 x15: 0000000000000000
[  334.432268] x14: ffff1e3e00e21180 x13: ffff5d37caafe000 x12: 0000000034d4d91d
[  334.432946] x11: 0000000000000000 x10: 0000000000001000 x9 : ffffc10728157a04
[  334.433622] x8 : 00000072b5503510 x7 : 0000000000000004 x6 : 000000011d833e13
[  334.434298] x5 : 00ffffffffffffff x4 : 0000000000c49ba6 x3 : ffff800080213cc0
[  334.434975] x2 : 00000002fbe6bd22 x1 : ffff1e3e00e21180 x0 : 0000000000000000
[  334.435652] Call trace:
[  334.435893]  cpuidle_enter_state+0xbc/0x4e0
[  334.436300]  cpuidle_enter+0x40/0x60
[  334.436652]  do_idle+0x1ec/0x268
[  334.436972]  cpu_startup_entry+0x40/0x50
[  334.437353]  secondary_start_kernel+0x138/0x168
[  334.437790]  __secondary_switched+0xb8/0xc0

FWIW, the same kernel which hangs on rockpi4b, works on qemu, kv260, zcu102 and synquacer boards without any kernel issues.

rockpi4b issue filed to https://bugzilla.kernel.org/show_bug.cgi?id=219704

Created attachment 5094 [details]
Beagle console log

Attaching a full console log from my beagleplay.

TI did some (slow) bisecting and isolated it down to a 6.7 change:

commit 8700a7ea5519fb0b3bad2362adfeac358c2119ce
Author: Tony Lindgren <tony@atomide.com>
Date:   Wed Oct 4 09:26:48 2023 +0300

    serial: 8250_omap: Drop pm_runtime_irq_safe()

A revert to stop the kernel hang has been merged into meta-yocto 4951957.  Keeping this open but at a lower priority to resolve the underlying 'opening bad tty' issue.

Kernel hang and call trace are gone in 5.2 M3 rc1.

Just found this in linux master:

commit 910ef438e93cd2ceb70c72caea418710d648feef
Author: John Ogness <john.ogness@linutronix.de>
Date:   Tue Jan 7 22:33:00 2025 +0106

    serial: 8250: Provide flag for IER toggling for RS485

    For RS485 mode, if SER_RS485_RX_DURING_TX is not available, the
    console ->write() callback needs to enable/disable Tx. It does
    this by calling the ->rs485_start_tx() and ->rs485_stop_tx()
    callbacks. However, some of these callbacks also disable/enable
    interrupts and makes power management calls. This causes 2
    problems for console writing:

    1. A console write can occur in contexts that are illegal for
       pm_runtime_*(). It is not even necessary for console writing
       to use pm_runtime_*() because a console already does this in
       serial8250_console_setup() and serial8250_console_exit().

    2. The console ->write() callback already handles
       disabling/enabling the interrupts by properly restoring the
       previous IER value.

    Add an argument @toggle_ier to the ->rs485_start_tx() and
    ->rs485_stop_tx() callbacks to specify if they may disable/enable
    receive interrupts while using pm_runtime_*(). Console writing
    will not allow the toggling.

Looks relevant, will test.

Move to a HIGH for 5.2 since 
  kv260 and some other boards are DOA in 5.2 for genericarm64 w/o a fix

See:
https://lists.yoctoproject.org/g/poky/message/13592

If this problem and the one in the link above are not the same, 
please create a new defect and revert the milestone and importance 
of this defect.

Thanks.

(In reply to Randy MacLeod from comment #16)
> Move to a HIGH for 5.2 since 
>   kv260 and some other boards are DOA in 5.2 for genericarm64 w/o a fix
> 
> See:
> https://lists.yoctoproject.org/g/poky/message/13592
> 
> If this problem and the one in the link above are not the same, 
> please create a new defect and revert the milestone and importance 
> of this defect.
> 
> Thanks.

I don't think these bugs are related. kv260 issue is about agetty startup on target with poky-altcfg distro, which currently is using only build time configured SERIAL_CONSOLES and agetty is not starter to tty's detected at runtime.

Thanks for the clarification Mikko.

I'm on vacation as of today and heading out the door soon so 
would you mind creating a new defect for the  kv260 issue  ?

Richard asked for one in yesterday's bug review meeting.

../Randy

(In reply to Randy MacLeod from comment #18)
> Thanks for the clarification Mikko.
> 
> I'm on vacation as of today and heading out the door soon so 
> would you mind creating a new defect for the  kv260 issue  ?
> 
> Richard asked for one in yesterday's bug review meeting.
> 
> ../Randy

Done, created:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15830

Thanks Mikko.

This issue occurs again in 5.3 M3 with kernel 6.16.

Image Location: https://valkyrie.yocto.io/pub/releases/yocto-5.3_M3.rc1/machines/genericarm64/

Git rev: oecore: 5298d01663f897e38380116edf23973d2a156cbc
         meta-yocto: fe65e142d0d9ba0e51ff9175ffa82e902f982a20


The system can boot up. But after login, The system randomly hangs:

root@genericarm64:~# BUG: scheduling while atomic: getty/1078/0x00000002
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio dp83td510 realtek snd_soc_hdmi_codec tps65219_pwrbutton tps65219_regulator gpio_tps65219 xhci_plat_hcd dwc3 ite_it66121 tps65219 udc_co
re rtc_bq32k at24 roles ti_am65_cpsw_nuss k3_cppi_desc_pool k3_udma_glue ghash_ce sha1_ce snd_soc_simple_card snd_soc_simple_card_utils gpio_regulator mdio_gpio mdio_bitbang fixed gpio_keys snd_soc_davinci_mcasp
 sdhci_am654 snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma k3_j72xx_bandgap dwc3_am62 rtc_ti_k3 k3_udma snd_soc_core irq_ti_sci_inta k3_psil_lib k3_ringacc snd_compress ac97_bus snd_pcm_dmaengine tidss sci_clk
 snd_pcm rti_wdt reset_ti_sci irq_ti_sci_intr am65_cpts phylink ti_sci_pm_domains virt_dma snd_timer pinctrl_single snd drm_display_helper gpio_davinci cec drm_dma_helper omap_mailbox sa2ul soundcore omap_hwspin
lock syscon_clk phy_gmii_sel 8250_omap i2c_omap mux_core spi_omap2_mcspi display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 1 UID: 0 PID: 1078 Comm: getty Not tainted 6.16.9-yocto-standard #1 PREEMPT 
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Call trace:
 show_stack+0x24/0x40 (C)
 dump_stack_lvl+0x64/0x88
 dump_stack+0x1c/0x2c
 __schedule_bug+0x5c/0x80
 __schedule+0xa3c/0xc78
 schedule+0x40/0x130
 schedule_timeout+0x8c/0x130
 wait_for_completion_timeout+0x88/0x170
 ti_sci_set_device_state+0x134/0x220
 ti_sci_cmd_get_device_exclusive+0x24/0x40
 ti_sci_pd_power_on+0x38/0x78 [ti_sci_pm_domains]
 _genpd_power_on+0xa8/0x170
 genpd_power_on+0xe4/0x268
 genpd_runtime_resume+0xc8/0x240
 __rpm_callback+0x54/0x230
 rpm_callback+0x84/0xa0
 rpm_resume+0x454/0x660
 __pm_runtime_resume+0x58/0xa8
 omap8250_set_mctrl+0x38/0xd8 [8250_omap]
 serial8250_set_mctrl+0x2c/0x60
 uart_update_mctrl+0x98/0x120
 uart_shutdown+0x118/0x178
 uart_hangup+0x78/0x190
 __tty_hangup.part.0+0x408/0x440
 tty_vhangup_session+0x24/0x40
 disassociate_ctty.part.0+0x44/0x1b8
 disassociate_ctty+0x30/0x48
 do_exit+0x6e4/0x9e0
 do_group_exit+0x3c/0xc0
 __arm64_sys_exit_group+0x24/0x30
 invoke_syscall.constprop.0+0x54/0xe8
 do_el0_svc+0x48/0xd8
 el0_svc+0x40/0x168
 el0t_64_sync_handler+0x114/0x140
 el0t_64_sync+0x1a4/0x1a8
BUG: scheduling while atomic: getty/1078/0x00000000
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio dp83td510 realtek snd_soc_hdmi_codec tps65219_pwrbutton tps65219_regulator gpio_tps65219 xhci_plat_hcd dwc3 ite_it66121 tps65219 udc_co
re rtc_bq32k at24 roles ti_am65_cpsw_nuss k3_cppi_desc_pool k3_udma_glue ghash_ce sha1_ce snd_soc_simple_card snd_soc_simple_card_utils gpio_regulator mdio_gpio mdio_bitbang fixed gpio_keys snd_soc_davinci_mcasp
 sdhci_am654 snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma k3_j72xx_bandgap dwc3_am62 rtc_ti_k3 k3_udma snd_soc_core irq_ti_sci_inta k3_psil_lib k3_ringacc snd_compress ac97_bus snd_pcm_dmaengine tidss sci_clk
 snd_pcm rti_wdt reset_ti_sci irq_ti_sci_intr am65_cpts phylink ti_sci_pm_domains virt_dma snd_timer pinctrl_single snd drm_display_helper gpio_davinci cec drm_dma_helper omap_mailbox sa2ul soundcore omap_hwspin
lock syscon_clk phy_gmii_sel 8250_omap i2c_omap mux_core spi_omap2_mcspi display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 0 UID: 0 PID: 1078 Comm: getty Tainted: G        W           6.16.9-yocto-standard #1 PREEMPT 
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Call trace:
 show_stack+0x24/0x40 (C)
 dump_stack_lvl+0x64/0x88
 dump_stack+0x1c/0x2c
 __schedule_bug+0x5c/0x80
 __schedule+0xa3c/0xc78
 schedule+0x40/0x130
 rpm_resume+0x190/0x660
 __pm_runtime_resume+0x58/0xa8
 omap_8250_pm+0x34/0x1b0 [8250_omap]
 serial8250_pm+0x24/0x60
 uart_hangup+0xfc/0x190
 __tty_hangup.part.0+0x408/0x440
 tty_vhangup_session+0x24/0x40
 disassociate_ctty.part.0+0x44/0x1b8
 disassociate_ctty+0x30/0x48
 do_exit+0x6e4/0x9e0
 do_group_exit+0x3c/0xc0
 __arm64_sys_exit_group+0x24/0x30
 invoke_syscall.constprop.0+0x54/0xe8
 do_el0_svc+0x48/0xd8
 el0_svc+0x40/0x168
 el0t_64_sync_handler+0x114/0x140
 el0t_64_sync+0x1a4/0x1a8
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1078 at /usr/src/kernel/kernel/sched/sched.h:2490 __schedule+0xa2c/0xc78
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio dp83td510 realtek snd_soc_hdmi_codec tps65219_pwrbutton tps65219_regulator gpio_tps65219 xhci_plat_hcd dwc3 ite_it66121 tps65219 udc_co
re rtc_bq32k at24 roles ti_am65_cpsw_nuss k3_cppi_desc_pool k3_udma_glue ghash_ce sha1_ce snd_soc_simple_card snd_soc_simple_card_utils gpio_regulator mdio_gpio mdio_bitbang fixed gpio_keys snd_soc_davinci_mcasp
 sdhci_am654 snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma k3_j72xx_bandgap dwc3_am62 rtc_ti_k3 k3_udma snd_soc_core irq_ti_sci_inta k3_psil_lib k3_ringacc snd_compress ac97_bus snd_pcm_dmaengine tidss sci_clk
 snd_pcm rti_wdt reset_ti_sci irq_ti_sci_intr am65_cpts phylink ti_sci_pm_domains virt_dma snd_timer pinctrl_single snd drm_display_helper gpio_davinci cec drm_dma_helper omap_mailbox sa2ul soundcore omap_hwspin
lock syscon_clk phy_gmii_sel 8250_omap i2c_omap mux_core spi_omap2_mcspi display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 0 UID: 0 PID: 1078 Comm: getty Tainted: G        W           6.16.9-yocto-standard #1 PREEMPT 
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : __schedule+0xa2c/0xc78
lr : __schedule+0xa1c/0xc78
sp : ffff8000846dba90
x29: ffff8000846dbac0 x28: ffff00000e2e6a80 x27: ffff00007f94d1c0
x26: 0000000000000001 x25: ffff00000e2e70e8 x24: 0000000000000000
x23: ffff8000813d4138 x22: 0000000000000000 x21: ffff800081fab708
x20: ffff00000e2e6a80 x19: ffff0000007811c0 x18: ffff800082d77090
x17: ffff000000683b40 x16: 0000000000000004 x15: 00000000ffffffff
x14: 0000000000000000 x13: ffff8000846db223 x12: ffff8000846db000
x11: 00000000fffffffd x10: 00000000ffffefff x9 : ffff8000813d3e9c
x8 : ffff00000e2e6a80 x7 : 0000000000017fe8 x6 : 00000000b8d49654
x5 : 03ffffffffffffff x4 : 0000000000000015 x3 : 0000000000000000
x2 : 0000000002d93d81 x1 : 000000000b550fa0 x0 : ffff0000007811c0
Call trace:
 __schedule+0xa2c/0xc78 (P)
 schedule+0x40/0x130
 rpm_resume+0x190/0x660
 __pm_runtime_resume+0x58/0xa8
 omap_8250_pm+0x34/0x1b0 [8250_omap]
 serial8250_pm+0x24/0x60
 uart_hangup+0xfc/0x190
 __tty_hangup.part.0+0x408/0x440
 tty_vhangup_session+0x24/0x40
 disassociate_ctty.part.0+0x44/0x1b8
 disassociate_ctty+0x30/0x48
 do_exit+0x6e4/0x9e0
 do_group_exit+0x3c/0xc0
 __arm64_sys_exit_group+0x24/0x30
 invoke_syscall.constprop.0+0x54/0xe8
 do_el0_svc+0x48/0xd8
 el0_svc+0x40/0x168
 el0t_64_sync_handler+0x114/0x140
 el0t_64_sync+0x1a4/0x1a8
---[ end trace 0000000000000000 ]---
mmc1: Timeout waiting for hardware interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000060 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0x0048cd28 | Trn mode: 0x0000002b
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x00000000 | Int stat: 0x00000003
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x0000193a | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x00e6cf7f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x00000003
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x00000000f4208290
mmc1: sdhci: ============================================
mmc1: Timeout waiting for hardware interrupt.
mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
mmc1: sdhci: Sys addr:  0x00000060 | Version:  0x00001004
mmc1: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
mmc1: sdhci: Argument:  0x00000000 | Trn mode: 0x00000023
mmc1: sdhci: Present:   0x01f70000 | Host ctl: 0x000000df
mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
mmc1: sdhci: Timeout:   0x0000000e | Int stat: 0x00018002
mmc1: sdhci: Int enab:  0x03ff000b | Sig enab: 0x03ff000b
mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
mmc1: sdhci: Cmd:       0x00000c1b | Max curr: 0x00000000
mmc1: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x00e6cf7f
mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000900
mmc1: sdhci: Host ctl2: 0x00000003
mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x00000000f4208290
mmc1: sdhci: ============================================
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu:    (detected by 3, t=5256 jiffies, g=26165, q=238 ncpus=4)
rcu: All QSes seen, last rcu_preempt kthread activity 5237 (4294946989-4294941752), jiffies_till_next_fqs=1, root ->qsmask 0x0
rcu: rcu_preempt kthread timer wakeup didn't happen for 5241 jiffies! g26165 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200
rcu:    Possible timer handling issue on cpu=1 timer-softirq=5280
rcu: rcu_preempt kthread starved for 5242 jiffies! g26165 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x200 ->cpu=1
rcu:    Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R stack:0     pid:16    tgid:16    ppid:2      task_flags:0x208040 flags:0x00000010
Call trace:
 __switch_to+0xdc/0x148 (T)
 __schedule+0x3f8/0xc78
 schedule+0x40/0x130
 schedule_timeout+0x8c/0x130
 rcu_gp_fqs_loop+0x12c/0x6e8
 rcu_gp_kthread+0x220/0x278
 kthread+0x14c/0x240
 ret_from_fork+0x10/0x20


This issue only happens on kernel 6.16:
root@genericarm64:~# uname -a
Linux genericarm64 6.16.9-yocto-standard #1 SMP PREEMPT Mon Sep 29 01:39:04 UTC 2025 aarch64 GNU/Linux
root@genericarm64:~#

In kernel 6.12 on genericarm64-alt image, there is no such issue.

I wonder if I misplaced the serial patches from Mikko on 6.16, when the genericarm64 work was underway, I noticed that they were being bbappended instead of being in linux-yocto.

I created branches for the genericarm64 work, and put the revert + serial patch onto those branches.

I don't have hardware to test this on, but let me do a build and at least confirm that the serial patches look like they are all in place.

6.16 genericarm64 kernel does have the same uart revert patches as 6.12:

https://lists.yoctoproject.org/g/linux-yocto/message/15568

and its dependency

https://lists.yoctoproject.org/g/linux-yocto/message/15569

I think we need to debug/bisect further what is breaking this now
on beagleplay. The meta-yocto-bsp bbappend changes seem to be correctly applied to linux-yocto v6.16/standard/genericarm64 branch used by the kernel recipe.

I came to the same conclusion, and I just check that 6.17 also has the serial changes applied to the genericarm64 branches.

So we are good on that front

I wonder if this is always happening on -alt, i.e. is there a systemd interaction ?

I don't have the hardware to test this, so I'm not much good for bisecting and boot testing. If only qemu would expose the issue, then we could do some long running boot loops.

(In reply to Bruce Ashfield from comment #24)
> I came to the same conclusion, and I just check that 6.17 also has the
> serial changes applied to the genericarm64 branches.
> 
> So we are good on that front
> 
> I wonder if this is always happening on -alt, i.e. is there a systemd
> interaction ?

This issue happens on poky distro with sysvinit and kernel 6.16. The poky-alt distro with systemd doesn't have this issue because the default kernel is 6.12 in this distro.


> 
> I don't have the hardware to test this, so I'm not much good for bisecting
> and boot testing. If only qemu would expose the issue, then we could do some
> long running boot loops.

I got access to a BeaglePlay and am running
https://valkyrie.yocto.io/pub/releases/yocto-5.3_M3.rc1/machines/genericarm64/core-image-sato-genericarm64.rootfs-20251017083208.wic.zst
on it. Serial console and ethernet works correctly. So far no hangs.

My device has an older u-boot firmware which I have not changed or updated:

U-Boot SPL 2024.07 (Aug 30 2024 - 14:27:24 +0000)
SYSFW ABI: 3.1 (firmware rev 0x000a '10.0.2--v10.00.02 (Fiery Fox)')
Changed A53 CPU frequency to 1250000000Hz (T grade) in DT
SPL initial stack usage: 13400 bytes
Trying to boot from MMC1
Starting ATF on ARM64 core...

NOTICE:  BL31: v2.11.0(release):v2.11.0
NOTICE:  BL31: Built : 14:27:24, Aug 30 2024
I/TC: 
I/TC: OP-TEE version: 4.3.0 (gcc version 12.2.0 (Debian 12.2.0-14)) #1 Fri Aug 30 14:27:24 UTC 2024 aarch64
I/TC: WARNING: This OP-TEE configuration might be insecure!
I/TC: WARNING: Please check https://optee.readthedocs.io/en/latest/architecture/porting_guidelines.html
I/TC: Primary CPU initializing
I/TC: GIC redistributor base address not provided
I/TC: Assuming default GIC group status and modifier
I/TC: SYSFW ABI: 3.1 (firmware rev 0x000a '10.0.2--v10.00.02 (Fiery Fox)')
I/TC: HUK Initialized
I/TC: Primary CPU switching to normal world boot

U-Boot SPL 2024.07 (Aug 30 2024 - 14:27:24 +0000)
SYSFW ABI: 3.1 (firmware rev 0x000a '10.0.2--v10.00.02 (Fiery Fox)')
SPL initial stack usage: 1760 bytes
Trying to boot from MMC1


U-Boot 2024.07 (Aug 30 2024 - 14:27:24 +0000)

SoC:   AM62X SR1.0 GP
Model: BeagleBoard.org BeaglePlay
DRAM:  2 GiB
Core:  99 devices, 28 uclasses, devicetree: separate
MMC:   mmc@fa10000: 0, mmc@fa00000: 1, mmc@fa20000: 2
Loading Environment from nowhere... OK
In:    serial@2800000
Out:   serial@2800000
Err:   serial@2800000
Net:   am65_cpsw_nuss_port ethernet@8000000port@1: phy_connect() failed

Boot to serial console prompt works but there are some error messages which to me are unrelated:

Linux version 6.16.9-yocto-standard (oe-user@oe-host) (aarch64-poky-linux-gcc (GCC) 15.2.0, GNU ld (GNU Binutils) 2.45.0.20250908) #1 SMP PREEMPT Mon Sep 29 01:39:04 UTC 2025
KASLR disabled due to lack of seed
Machine model: BeagleBoard.org BeaglePlay
...
Console: colour dummy device 80x25
printk: legacy console [tty0] enabled
...
2800000.serial: ttyS2 at MMIO 0x2800000 (irq = 476, base_baud = 3000000) is a 8250
printk: legacy console [ttyS2] enabled
...
2850000.serial: ttyS0 at MMIO 0x2850000 (irq = 477, base_baud = 3000000) is a 8250
2860000.serial: ttyS1 at MMIO 0x2860000 (irq = 478, base_baud = 3000000) is a 8250
...
davinci-mcasp 2b10000.audio-controller: Sample-rate is off by 400 PPM
davinci-mcasp 2b10000.audio-controller: stream has more channels (2) than are enabled in mcasp (0)
davinci-mcasp 2b10000.audio-controller: ASoC error (-22): at snd_soc_dai_hw_params() on 2b10000.audio-controller
 davinci-mcasp.0-i2s-hifi: ASoC error (-22): at __soc_pcm_hw_params() on davinci-mcasp.0-i2s-hifi
davinci-mcasp 2b10000.audio-controller: Sample-rate is off by 400 PPM
davinci-mcasp 2b10000.audio-controller: stream has more channels (2) than are enabled in mcasp (0)
davinci-mcasp 2b10000.audio-controller: ASoC error (-22): at snd_soc_dai_hw_params() on 2b10000.audio-controller
 davinci-mcasp.0-i2s-hifi: ASoC error (-22): at __soc_pcm_hw_params() on davinci-mcasp.0-i2s-hifi
davinci-mcasp 2b10000.audio-controller: Sample-rate is off by 400 PPM
...
Poky (Yocto Project Reference Distro) 5.2.99+snapshot-5298d01663f897e38380116edf23973d2a156cbc genericarm64 /dev/ttyS2

genericarm64 login: root

WARNING: Poky is a reference Yocto Project distribution that should be used for
testing and development purposes only. It is recommended that you create your
own distribution for production use.

root@genericarm64:~# uname -a
Linux genericarm64 6.16.9-yocto-standard #1 SMP PREEMPT Mon Sep 29 01:39:04 UTC 2025 aarch64 GNU/Linux

I will try stressing and testing the device some more but so far I've not reproduced this problem.

Tried with image
https://valkyrie.yocto.io/pub/releases/yocto-5.3_M3.rc1/machines/genericarm64/core-image-sato-genericarm64.rootfs-20251017083208.wic.zst

and was able to reproduce:

root@genericarm64:~# uname -a
Linux genericarm64 6.16.9-yocto-standard #1 SMP PREEMPT Mon Sep 29 01:39:04 UTC 2025 aarch64 GNU/Linux
root@genericarm64:~# lsmod
Module                  Size  Used by
bnep                   20480  2
wl18xx                110592  0
wlcore                212992  1 wl18xx
mac80211              933888  2 wl18xx,wlcore
libarc4                12288  1 mac80211
cfg80211              851968  3 wl18xx,wlcore,mac80211
wlcore_sdio            12288  0
snd_soc_hdmi_codec     16384  1
dp83td510              12288  1
realtek                36864  1
xhci_plat_hcd          16384  0
tps65219_pwrbutton     12288  0
tps65219_regulator     16384  4
gpio_tps65219          12288  0
ite_it66121            20480  0
dwc3                  176128  0
udc_core               57344  1 dwc3
roles                  12288  1 dwc3
tps65219               28672  0
at24                   20480  0
rtc_bq32k              12288  0
ghash_ce               16384  0
snd_soc_simple_card    16384  0
sha1_ce                12288  0
gpio_regulator         12288  1
snd_soc_simple_card_utils    20480  1 snd_soc_simple_card
mdio_gpio              12288  3
mdio_bitbang           12288  1 mdio_gpio
fixed                  12288  12
gpio_keys              16384  0
sdhci_am654            20480  0
k3_j72xx_bandgap       12288  0
ti_am65_cpsw_nuss      81920  0
k3_cppi_desc_pool      12288  1 ti_am65_cpsw_nuss
dwc3_am62              12288  0
k3_udma_glue           20480  1 ti_am65_cpsw_nuss
rtc_ti_k3              16384  1
ti_sci_pm_domains      12288  0
k3_udma                57344  20 k3_udma_glue
k3_psil_lib            49152  2 k3_udma,k3_udma_glue
irq_ti_sci_inta        16384  0
k3_ringacc             20480  2 k3_udma,k3_udma_glue
sci_clk                12288  29
reset_ti_sci           12288  0
pinctrl_single         24576  132
rti_wdt                12288  0
am65_cpts              20480  1 ti_am65_cpsw_nuss
irq_ti_sci_intr        12288  0
phylink                40960  1 ti_am65_cpsw_nuss
virt_dma               12288  1 k3_udma
gpio_davinci           12288  21
sa2ul                  32768  0
omap_mailbox           12288  0
tidss                  53248  1
snd_soc_davinci_mcasp    36864  2
snd_soc_ti_udma        12288  1 snd_soc_davinci_mcasp
snd_soc_ti_edma        12288  1 snd_soc_davinci_mcasp
snd_soc_ti_sdma        12288  1 snd_soc_davinci_mcasp
phy_gmii_sel           16384  2
drm_display_helper    151552  1 tidss
snd_soc_core          233472  7 snd_soc_davinci_mcasp,snd_soc_ti_sdma,snd_soc_ti_edma,snd_soc_hdmi_codec,snd_soc_ti_udma,snd_soc_simple_card_utils,snd_soc_simple_card
cec                    45056  1 drm_display_helper
snd_compress           24576  1 snd_soc_core
ac97_bus               12288  1 snd_soc_core
snd_pcm_dmaengine      12288  1 snd_soc_core
drm_dma_helper         16384  1 tidss
snd_pcm               122880  6 snd_soc_davinci_mcasp,snd_soc_hdmi_codec,snd_compress,snd_soc_simple_card_utils,snd_soc_core,snd_pcm_dmaengine
snd_timer              36864  1 snd_pcm
snd                    77824  5 snd_soc_hdmi_codec,snd_timer,snd_compress,snd_soc_core,snd_pcm
soundcore              12288  1 snd
omap_hwspinlock        12288  0
8250_omap              24576  0
syscon_clk             12288  0
spi_omap2_mcspi        20480  0
i2c_omap               16384  0
mux_core               16384  1 i2c_omap
display_connector      12288  0
aes_neon_bs            24576  0
aes_neon_blk           24576  1 aes_neon_bs
aes_ce_blk             32768  0
aes_ce_cipher          12288  1 aes_ce_blk
root@genericarm64:~# BUG: scheduling while atomic: getty/1038/0x00000002
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio snd_soc_hdmi_codec dp83td510 realtek xhci_plat_hcd tps65219_pwrbutton tps65219_regulator gpio_tps65219 ite_it66121 dwc3 udc_core roles tps65219 at24 rtc_bq32k ghash_ce snd_soc_simple_card sha1_ce gpio_regulator snd_soc_simple_card_utils mdio_gpio mdio_bitbang fixed gpio_keys sdhci_am654 k3_j72xx_bandgap ti_am65_cpsw_nuss k3_cppi_desc_pool dwc3_am62 k3_udma_glue rtc_ti_k3 ti_sci_pm_domains k3_udma k3_psil_lib irq_ti_sci_inta k3_ringacc sci_clk reset_ti_sci pinctrl_single rti_wdt am65_cpts irq_ti_sci_intr phylink virt_dma gpio_davinci sa2ul omap_mailbox tidss snd_soc_davinci_mcasp snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma phy_gmii_sel drm_display_helper snd_soc_core cec snd_compress ac97_bus snd_pcm_dmaengine drm_dma_helper snd_pcm snd_timer snd soundcore omap_hwspinlock 8250_omap syscon_clk spi_omap2_mcspi i2c_omap mux_core display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 1 UID: 0 PID: 1038 Comm: getty Not tainted 6.16.9-yocto-standard #1 PREEMPT 
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Call trace:
 show_stack+0x24/0x40 (C)
 dump_stack_lvl+0x64/0x88
 dump_stack+0x1c/0x2c
 __schedule_bug+0x5c/0x80
 __schedule+0xa3c/0xc78
 schedule+0x40/0x130
 schedule_timeout+0x8c/0x130
 wait_for_completion_timeout+0x88/0x170
 ti_sci_set_device_state+0x134/0x220
 ti_sci_cmd_get_device_exclusive+0x24/0x40
 ti_sci_pd_power_on+0x38/0x78 [ti_sci_pm_domains]
 _genpd_power_on+0xa8/0x170
 genpd_power_on+0xe4/0x268
 genpd_runtime_resume+0xc8/0x240
 __rpm_callback+0x54/0x230
 rpm_callback+0x84/0xa0
 rpm_resume+0x454/0x660
 __pm_runtime_resume+0x58/0xa8
 omap8250_set_mctrl+0x38/0xd8 [8250_omap]
 serial8250_set_mctrl+0x2c/0x60
 uart_update_mctrl+0x98/0x120
 uart_shutdown+0x118/0x178
 uart_hangup+0x78/0x190
 __tty_hangup.part.0+0x408/0x440
 tty_vhangup_session+0x24/0x40
 disassociate_ctty.part.0+0x44/0x1b8
 disassociate_ctty+0x30/0x48
 do_exit+0x6e4/0x9e0
 do_group_exit+0x3c/0xc0
 __arm64_sys_exit_group+0x24/0x30
 invoke_syscall.constprop.0+0x54/0xe8
 do_el0_svc+0x48/0xd8
 el0_svc+0x40/0x168
 el0t_64_sync_handler+0x114/0x140
 el0t_64_sync+0x1a4/0x1a8
BUG: scheduling while atomic: getty/1038/0x00000000
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio snd_soc_hdmi_codec dp83td510 realtek xhci_plat_hcd tps65219_pwrbutton tps65219_regulator gpio_tps65219 ite_it66121 dwc3 udc_core roles tps65219 at24 rtc_bq32k ghash_ce snd_soc_simple_card sha1_ce gpio_regulator snd_soc_simple_card_utils mdio_gpio mdio_bitbang fixed gpio_keys sdhci_am654 k3_j72xx_bandgap ti_am65_cpsw_nuss k3_cppi_desc_pool dwc3_am62 k3_udma_glue rtc_ti_k3 ti_sci_pm_domains k3_udma k3_psil_lib irq_ti_sci_inta k3_ringacc sci_clk reset_ti_sci pinctrl_single rti_wdt am65_cpts irq_ti_sci_intr phylink virt_dma gpio_davinci sa2ul omap_mailbox tidss snd_soc_davinci_mcasp snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma phy_gmii_sel drm_display_helper snd_soc_core cec snd_compress ac97_bus snd_pcm_dmaengine drm_dma_helper snd_pcm snd_timer snd soundcore omap_hwspinlock 8250_omap syscon_clk spi_omap2_mcspi i2c_omap mux_core display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 2 UID: 0 PID: 1038 Comm: getty Tainted: G        W           6.16.9-yocto-standard #1 PREEMPT 
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
Call trace:
 show_stack+0x24/0x40 (C)
 dump_stack_lvl+0x64/0x88
 dump_stack+0x1c/0x2c
 __schedule_bug+0x5c/0x80
 __schedule+0xa3c/0xc78
 schedule+0x40/0x130
 rpm_resume+0x190/0x660
 __pm_runtime_resume+0x58/0xa8
 omap_8250_pm+0x34/0x1b0 [8250_omap]
 serial8250_pm+0x24/0x60
 uart_hangup+0xfc/0x190
 __tty_hangup.part.0+0x408/0x440
 tty_vhangup_session+0x24/0x40
 disassociate_ctty.part.0+0x44/0x1b8
 disassociate_ctty+0x30/0x48
 do_exit+0x6e4/0x9e0
 do_group_exit+0x3c/0xc0
 __arm64_sys_exit_group+0x24/0x30
 invoke_syscall.constprop.0+0x54/0xe8
 do_el0_svc+0x48/0xd8
 el0_svc+0x40/0x168
 el0t_64_sync_handler+0x114/0x140
 el0t_64_sync+0x1a4/0x1a8
------------[ cut here ]------------
WARNING: CPU: 2 PID: 1038 at /usr/src/kernel/kernel/sched/sched.h:2490 __schedule+0xa2c/0xc78
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio snd_soc_hdmi_codec dp83td510 realtek xhci_plat_hcd tps65219_pwrbutton tps65219_regulator gpio_tps65219 ite_it66121 dwc3 udc_core roles tps65219 at24 rtc_bq32k ghash_ce snd_soc_simple_card sha1_ce gpio_regulator snd_soc_simple_card_utils mdio_gpio mdio_bitbang fixed gpio_keys sdhci_am654 k3_j72xx_bandgap ti_am65_cpsw_nuss k3_cppi_desc_pool dwc3_am62 k3_udma_glue rtc_ti_k3 ti_sci_pm_domains k3_udma k3_psil_lib irq_ti_sci_inta k3_ringacc sci_clk reset_ti_sci pinctrl_single rti_wdt am65_cpts irq_ti_sci_intr phylink virt_dma gpio_davinci sa2ul omap_mailbox tidss snd_soc_davinci_mcasp snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma phy_gmii_sel drm_display_helper snd_soc_core cec snd_compress ac97_bus snd_pcm_dmaengine drm_dma_helper snd_pcm snd_timer snd soundcore omap_hwspinlock 8250_omap syscon_clk spi_omap2_mcspi i2c_omap mux_core display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 2 UID: 0 PID: 1038 Comm: getty Tainted: G        W           6.16.9-yocto-standard #1 PREEMPT 
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
pstate: 204000c5 (nzCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : __schedule+0xa2c/0xc78
lr : __schedule+0xa1c/0xc78
sp : ffff80008409ba90
x29: ffff80008409bac0 x28: ffff000003423540 x27: ffff00007f94d1c0
x26: 0000000000000001 x25: ffff000003423ba8 x24: 0000000000000000
x23: ffff8000813d4138 x22: 0000000000000000 x21: ffff800081fab708
x20: ffff000003423540 x19: ffff0000007811c0 x18: ffff800083df5090
x17: ffff000000683c00 x16: 0000000000000004 x15: 00000000ffffffff
x14: 0000000000000000 x13: ffff80008409b223 x12: ffff80008409b000
x11: 00000000fffffffd x10: 00000000ffffefff x9 : ffff8000813d3e9c
x8 : ffff000003423540 x7 : 0000000000017fe8 x6 : 00000000bf6ef390
x5 : 03ffffffffffffff x4 : 0000000000000015 x3 : 0000000000000000
x2 : 0000000002d93d81 x1 : 000000000065e002 x0 : ffff0000007811c0
Call trace:
 __schedule+0xa2c/0xc78 (P)
 schedule+0x40/0x130
 rpm_resume+0x190/0x660
 __pm_runtime_resume+0x58/0xa8
 omap_8250_pm+0x34/0x1b0 [8250_omap]
 serial8250_pm+0x24/0x60
 uart_hangup+0xfc/0x190
 __tty_hangup.part.0+0x408/0x440
 tty_vhangup_session+0x24/0x40
 disassociate_ctty.part.0+0x44/0x1b8
 disassociate_ctty+0x30/0x48
 do_exit+0x6e4/0x9e0
 do_group_exit+0x3c/0xc0
 __arm64_sys_exit_group+0x24/0x30
 invoke_syscall.constprop.0+0x54/0xe8
 do_el0_svc+0x48/0xd8
 el0_svc+0x40/0x168
 el0t_64_sync_handler+0x114/0x140
 el0t_64_sync+0x1a4/0x1a8
---[ end trace 0000000000000000 ]---
Unable to handle kernel read from unreadable memory at virtual address 0000000000000b58
Mem abort info:
  ESR = 0x0000000096000004
  EC = 0x25: DABT (current EL), IL = 32 bits
  SET = 0, FnV = 0
  EA = 0, S1PTW = 0
  FSC = 0x04: level 0 translation fault
Data abort info:
  ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
  CM = 0, WnR = 0, TnD = 0, TagAccess = 0
  GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
user pgtable: 4k pages, 48-bit VAs, pgdp=0000000088851000
[0000000000000b58] pgd=0000000000000000, p4d=0000000000000000
Internal error: Oops: 0000000096000004 [#1]  SMP
Modules linked in: bnep wl18xx wlcore mac80211 libarc4 cfg80211 wlcore_sdio snd_soc_hdmi_codec dp83td510 realtek xhci_plat_hcd tps65219_pwrbutton tps65219_regulator gpio_tps65219 ite_it66121 dwc3 udc_core roles tps65219 at24 rtc_bq32k ghash_ce snd_soc_simple_card sha1_ce gpio_regulator snd_soc_simple_card_utils mdio_gpio mdio_bitbang fixed gpio_keys sdhci_am654 k3_j72xx_bandgap ti_am65_cpsw_nuss k3_cppi_desc_pool dwc3_am62 k3_udma_glue rtc_ti_k3 ti_sci_pm_domains k3_udma k3_psil_lib irq_ti_sci_inta k3_ringacc sci_clk reset_ti_sci pinctrl_single rti_wdt am65_cpts irq_ti_sci_intr phylink virt_dma gpio_davinci sa2ul omap_mailbox tidss snd_soc_davinci_mcasp snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma phy_gmii_sel drm_display_helper snd_soc_core cec snd_compress ac97_bus snd_pcm_dmaengine drm_dma_helper snd_pcm snd_timer snd soundcore omap_hwspinlock 8250_omap syscon_clk spi_omap2_mcspi i2c_omap mux_core display_connector aes_neon_bs aes_neon_blk aes_ce_blk aes_ce_cipher
CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Tainted: G        W           6.16.9-yocto-standard #1 PREEMPT 
Tainted: [W]=WARN
Hardware name: beagle BeagleBoard.org BeaglePlay/BeagleBoard.org BeaglePlay, BIOS 2024.07 07/01/2024
pstate: 604000c5 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : sched_balance_update_blocked_averages+0x54/0x570
lr : sched_balance_update_blocked_averages+0x54/0x570
sp : ffff8000825fbdb0
x29: 0000000000000000 x28: ffff0000007811c0 x27: ffff00007f94d1c0
x26: 0000000000000001 x25: ffff000000781828 x24: 0000000000000000
x23: ffff8000813d43a4 x22: 0000000000000000 x21: ffff800081fab708
x20: ffff800080133cd0 x19: 0000000000000000 x18: ffff8000824b5030
x17: ffff000000683c00 x16: 0000000000000004 x15: 00000000ffffffff
x14: 0000000000000000 x13: ffff80008409b223 x12: ffff80008409b000
x11: 00000000000000c0 x10: 0000000000000b00 x9 : ffff8000825fbd60
x8 : ffff000000781d20 x7 : 0000000000000001 x6 : 0000000000000000
x5 : ffff000002979d50 x4 : ffff0000007811c0 x3 : ffff000003423540
x2 : 0000000000000000 x1 : ffff0000007811c0 x0 : 0000000000000092
Call trace:
 sched_balance_update_blocked_averages+0x54/0x570 (P)
Code: d50343df 52800001 aa1303e0 97ffc494 (f945ae61) 
---[ end trace 0000000000000000 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
SMP: stopping secondary CPUs
SMP: failed to stop secondary CPUs 0,2
Kernel Offset: disabled
CPU features: 0x0000,00000000,02000801,0400421b
Memory Limit: none
---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

I must have been checking wrong things in linux-yocto kernel tree.

Looks like
https://lists.yoctoproject.org/g/linux-yocto/message/15568
and
https://lists.yoctoproject.org/g/linux-yocto/message/15569

are missing from v6.16/standard/genericarm64 branch.

(In reply to Mikko Rapeli from comment #28)
> I must have been checking wrong things in linux-yocto kernel tree.
> 
> Looks like
> https://lists.yoctoproject.org/g/linux-yocto/message/15568
> and
> https://lists.yoctoproject.org/g/linux-yocto/message/15569
> 
> are missing from v6.16/standard/genericarm64 branch.

Here is a "poky-altcfg" distro with systemd, kernel 6.16 and core-image-sato-sdk image with those two kernel patches applied (on top of some other unrelated genericarm64 kernel changes):

https://people.linaro.org/~mikko.rapeli/beagle_tty_hang/core-image-sato-sdk-genericarm64.rootfs.wic.zst

I am not seeing this crash on that image but will continue testing a bit more.

Will also test a "poky" distro build too with sysvinit since there the issue reproduced. I presume this issue would also happen on systemd though.

Bruce, could you pick those kernel patches to the v6.16/standard/genericarm64 and queue an update to linux-yocto? I think also 6.17 branch will need this.

oeqa runtime test executed via testexport look good with the patches applied, no hangs/crashes from kernel:

https://people.linaro.org/~mikko.rapeli/beagle_tty_hang/runtime-results.log

Failures are expected:

 * stap fails due to testexport not having full build env
 * opkg fails due to package repo not available in test env
 * parselogs fails due to new kernel/driver error messages

2025-10-29 14:01:12,610 - runtime - INFO - RESULTS:
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - buildcpio.BuildCpioTest.test_cpio: PASSED (249.64s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - buildlzip.BuildLzipTest.test_lzip: PASSED (68.12s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - connman.ConnmanTest.test_connmand_help: PASSED (1.56s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - connman.ConnmanTest.test_connmand_running: PASSED (1.34s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - date.DateTest.test_date: PASSED (14.94s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - df.DfTest.test_df: PASSED (1.48s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - gcc.GccCompileTest.test_gcc_compile: PASSED (7.86s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - gcc.GccCompileTest.test_gpp2_compile: PASSED (7.88s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - gcc.GccCompileTest.test_gpp_compile: PASSED (9.43s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - gcc.GccCompileTest.test_make: PASSED (6.40s)
2025-10-29 14:01:12,610 - runtime - INFO - RESULTS - gi.GObjectIntrospectionTest.test_python: PASSED (2.61s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - go.GoHelloworldTest.test_gohelloworld: PASSED (2.02s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - kernelmodule.KernelModuleTest.test_kernel_module: PASSED (112.86s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - ldd.LddTest.test_ldd: PASSED (3.73s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - logrotate.LogrotateTest.test_logrotate_newlog: PASSED (7.40s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - logrotate.LogrotateTest.test_logrotate_wtmp: PASSED (6.23s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - oe_syslog.SyslogTest.test_syslog_running: PASSED (1.54s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - oe_syslog.SyslogTestConfig.test_syslog_logger: PASSED (3.20s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - oe_syslog.SyslogTestConfig.test_syslog_restart: PASSED (2.60s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - pam.PamBasicTest.test_pam: PASSED (4.99s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - parselogs.ParseLogsTest.test_get_context: PASSED (0.00s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - perl.PerlTest.test_perl_works: PASSED (1.56s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - ping.PingTest.test_ping: PASSED (0.02s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - python.PythonTest.test_python3: PASSED (1.27s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - scp.ScpTest.test_scp_file: PASSED (3.85s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - ssh.SSHTest.test_ssh: PASSED (1.14s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - systemd.SystemdBasicTests.test_systemd_basic: PASSED (1.45s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - systemd.SystemdBasicTests.test_systemd_failed: PASSED (2.50s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - systemd.SystemdBasicTests.test_systemd_list: PASSED (2.34s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time: PASSED (1.57s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - systemd.SystemdJournalTests.test_systemd_journal: PASSED (1.77s)
2025-10-29 14:01:12,611 - runtime - INFO - RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable: PASSED (12.53s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - systemd.SystemdServiceTests.test_systemd_status: PASSED (1.20s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - systemd.SystemdServiceTests.test_systemd_stop_start: PASSED (5.20s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - xorg.XorgTest.test_xorg_running: PASSED (2.67s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - apt.AptRepoTest.test_apt_install_from_repo: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - buildgalculator.GalculatorTest.test_galculator: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfBasicTest.test_dnf_help: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfBasicTest.test_dnf_history: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfBasicTest.test_dnf_info: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfBasicTest.test_dnf_search: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfBasicTest.test_dnf_version: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_exclude: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_install: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_install_dependency: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_install_from_disk: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_install_from_http: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_installroot: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_installroot_usrmerge: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_makecache: SKIPPED (0.00s)
2025-10-29 14:01:12,612 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_reinstall: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - dnf.DnfRepoTest.test_dnf_repoinfo: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - go.GoCompileTest.test_go_compile: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - go.GoCompileTest.test_go_module: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - oe_syslog.SyslogTestConfig.test_syslog_startup_config: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectfail: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - ptest.PtestRunnerTest.test_ptestrunner_expectsuccess: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rpm.RpmBasicTest.test_rpm_help: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rpm.RpmBasicTest.test_rpm_query: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rpm.RpmBasicTest.test_rpm_query_nonroot: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rpm.RpmInstallRemoveTest.test_check_rpm_install_removal_log_file_size: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rpm.RpmInstallRemoveTest.test_rpm_install: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rpm.RpmInstallRemoveTest.test_rpm_remove: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rust.RustCLibExampleTest.test_rust_c_lib_example: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rust.RustCompileTest.test_cargo_compile: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - rust.RustCompileTest.test_rust_compile: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - systemd.SystemdServiceTests.test_systemd_coredump_minidebuginfo: SKIPPED (0.00s)
2025-10-29 14:01:12,613 - runtime - INFO - RESULTS - systemd.SystemdServiceTests.test_systemd_disable_enable_ro: SKIPPED (0.00s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - weston.WestonTest.test_wayland_info: SKIPPED (0.00s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - weston.WestonTest.test_weston_can_initialize_new_wayland_compositor: SKIPPED (0.00s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - weston.WestonTest.test_weston_running: SKIPPED (0.00s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - weston.WestonTest.test_weston_supports_xwayland: SKIPPED (0.00s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - stap.StapTest.test_stap: ERROR (0.00s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - opkg.OpkgRepoTest.test_opkg_install_from_repo: FAILED (2.89s)
2025-10-29 14:01:12,614 - runtime - INFO - RESULTS - parselogs.ParseLogsTest.test_parselogs: FAILED (11.64s)
2025-10-29 14:01:12,617 - runtime - INFO - SUMMARY:
2025-10-29 14:01:12,617 - runtime - INFO - runtime () - Ran 75 tests in 576.250s
2025-10-29 14:01:12,617 - runtime - INFO - runtime - FAIL - Required tests failed (successes=35, skipped=37, failures=2, errors=1)

Will now run the same tests with "poky" distro and sysvinit.

The serial patches are staged to be applied to the genericarm64 branch, and should be there.

I'm double checking now on 6.16 and 6.17.

If they are missing, I'll take care of bumping the SRCREVs to pick them up (I'm waiting on my slow builder to get the kernel build done).

The changes are on the branches for both 6.16 and 6.17:

[/home/bruc...-yocto.git]> git log --oneline -- drivers/tty/serial/8250/8250_port.c  |grep -i revert
06207c4dd5e24 Revert "serial: 8250: unexport serial8250_rpm_*() functions"
[/home/bruc...-yocto.git]> git branch --contains 06207c4dd5e24
* v6.16/standard/genericarm64

[/home/bruc...-yocto.git]> git log --oneline -- drivers/tty/serial/8250/8250_port.c  |grep -i revert
81ce86a643c08 Revert "serial: 8250: unexport serial8250_rpm_*() functions"
[/home/bruc...-yocto.git]> git branch --contains 81ce86a643c08
* v6.17/standard/genericarm64


I see the KBRANCH being set for genericarm64, but no SRCREV is set, that appears to be the missing piece.

Once my build runs, I'll confirm and send an explicit SRCREV set to the yocto-bsp bbappend.

(In reply to Bruce Ashfield from comment #32)
> The changes are on the branches for both 6.16 and 6.17:
> 
> [/home/bruc...-yocto.git]> git log --oneline --
> drivers/tty/serial/8250/8250_port.c  |grep -i revert
> 06207c4dd5e24 Revert "serial: 8250: unexport serial8250_rpm_*() functions"
> [/home/bruc...-yocto.git]> git branch --contains 06207c4dd5e24
> * v6.16/standard/genericarm64
> 
> [/home/bruc...-yocto.git]> git log --oneline --
> drivers/tty/serial/8250/8250_port.c  |grep -i revert
> 81ce86a643c08 Revert "serial: 8250: unexport serial8250_rpm_*() functions"
> [/home/bruc...-yocto.git]> git branch --contains 81ce86a643c08
> * v6.17/standard/genericarm64
> 
> 
> I see the KBRANCH being set for genericarm64, but no SRCREV is set, that
> appears to be the missing piece.
> 
> Once my build runs, I'll confirm and send an explicit SRCREV set to the
> yocto-bsp bbappend.

Oh yes. And because the linux-yocto base SRCREV_machine commit from v6.16/standard/base branch is also in v6.16/standard/genericarm64 branch we did not see complaints from the missing SRCREV_machine:genericarm64.

This makes linux-yocto kernel updates a bit trickier since bumps need to be submitted to meta-yocto-bsp too. Maybe SRCREV_machine:genericarm64 should be set already in to oe-core side linux-yocto recipe to avoid this kind of issues. Then  missing updates to this or other machines would be easy to spot.

My scripts do know how to update the bbappend for the h/w references, but since I didn't have a seed / starting SRCREV it was skipping it.

But yes, I agree that carrying a baseline / fallback commit for the references in the main recipe isn't an issue and in fact, makes usability much better. Someone can jump off the end of what is in the main .bb file, versus having to track down the bbappend.

My build is still running, but I'll come up with something and send it along once it completes.

Fix for 6.16 linux-yocto kernels: https://lists.openembedded.org/g/openembedded-core/message/225515

Fixed in poky and oe-core/meta-yocto master branch.

Verified with 5.3 rc4.