Bug 6183 - Resume from S3 fails
Summary: Resume from S3 fails
Status: RESOLVED FIXED
Alias: None
Product: MinnowBoard MAX Firmware
Classification: Hardware Platforms
Component: minnowmax-edk2 (show other bugs)
Version: unspecified
Hardware: MinnowBoard Max x86_64
: Medium critical
Target Milestone: Pre-Production Relea
Assignee: David_Wei
QA Contact:
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2014-04-18 20:18 UTC by Stephen K Jolley
Modified: 2014-10-28 03:29 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
Firmware resume log (33.33 KB, text/plain)
2014-06-23 17:15 UTC, Darren Hart
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stephen K Jolley 2014-04-18 20:18:33 UTC
Resume from S3 fails
  - echo mem > /sys/power/state
  - The J1 LED blinks slowly
  - Pressing S1 does not resume the board

BYT-I should support it.  Not sure how much effort will be required by Firmware to fix.
Comment 1 Darren Hart 2014-06-23 17:00:27 UTC
With the current firmware image (6/17 debug), the light no longer blinks after issueing the "echo mem > /sys/power/state" call, but the monitor does go blank. Trying to resume via the USB keyboard does nothing. Pressing the S1 button results in EFI/firmware messages on the serial console, some errors, and ultimately an assert and then nothing more.

As a test, using the same OS image (Yocto 5/13 build) on the Intel DE3815TYBE NUC (Same SoC), issueing the "echo mem > /sys/power/state" command results in the display blanking and the power LED blinking. Pressing a key on the USB keyboard results in a successful resume.

Based on this, I believe the problem lies in the firmware. The procedure described here can be used to verify firmware changes in support of this bug.
Comment 2 Darren Hart 2014-06-23 17:15:22 UTC
Created attachment 2016 [details]
Firmware resume log

On a subsequent attempt, pressing a key on the USB keyboard did result in firmware messages over serial, so perhaps the wakeup is initiated as expected, the output however was the same and the resume did not complete.
Comment 3 Ivan Rouzanov 2014-07-23 23:56:19 UTC
With 7/18 engineering drop (MNW2CRB1.X64.0071.D30.1407181653) I see exact same assert when resuming from Windows 8.1.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
ERROR: C80000002:V3031002 I0 86D70125-BAA3-4296-A62F-602BEBBB9081

ASSERT_EFI_ERROR (Status = Not Found)

PEI_ASSERT!: c:\mnw2\MdeModulePkg\Core\DxeIplPeim\DxeLoad.c (232): !EFI_ERROR (Status)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Fro JTAG, the core is stuck in the loop:

0x10:000000007AFD2407  mov dword ptr [ebp - 0x04], 0x00000000
0x10:000000007AFD240E  mov eax, dword ptr [ebp - 0x04]
0x10:000000007AFD2411  test eax, eax
0x10:000000007AFD2413  jz $-0x05 ;a=7afd240e

This is on the UEFI side before handoff to the OS resume vector.
The assert suggests that S3Resume2 PPI was not found.

Looking at the Platform Package DSC, it seems that S3Resume2 PPI is included under Components.IA32 only and not under Components.X64 which would explain why it is missing in 64-bit.
Comment 4 Darren Hart 2014-08-04 17:39:47 UTC
I tested suspend/resume using the 7/28 image from David Wei using a current Yocto Project intel-corei7-64 build.
 
$ echo mem > /sys/power/state
 
The system becomes unresponsive as expected. Pressing the power button results in a lot of S3 resume debug output from the shell as before, but now does make it to the OS handoff, where I see the following output:
 
Transfer to 16bit OS waking vector - 991D0
[   40.553922] xhci_hcd 0000:00:14.0: Setup ERROR: setup context command for slot 2.
[   40.706798] xhci_hcd 0000:00:14.0: Setup ERROR: setup context command for slot 2.
root@intel-corei7-64:~#
 
Unfortunately, we get some xhci errors, but the debug port (serial console) is no functional. What I type is not echoed on the terminal, nor do commands execute after pressing enter (as would be the case if it were just the termios being messed up).
 
From the HDMI side, the screen goes black on suspend and comes back on at resume, but the USB keyboard and mouse do not work.
 
From what I can tell, the system is non-responsive after resume. The same OS image works as expected on an Intel NUC with the same SoC.
Comment 5 Darren Hart 2014-08-07 20:33:32 UTC
Using a 3.16-rc7 kernel, the XHCI messages went away, but it still hung. On a whim I started disconnecting things, the Seacat Lure (with mini PCI wifi) - didn't help, then the Ethernet cable, and it started working.

With the following connected:
Power
Ethernet
HDMI Display
Serial Cable
USB 3.0 Flash Drive (OS)
Lenovo keyboard/mouse (USB 2.0 port)

When I would issue:
echo mem > /sys/power/state
and then press the power button to resume, the console and HDMI display would come back, but would not accept keyboard or mouse input. A looping test did not continue.

If I removed the Ethernet cable, the resume worked correctly via the power button and via pressing a key on the keyboard. The input worked and the looping test in the shell continues to print.

Interestingly, after a successful resume, I plugged the Ethernet cable back in and it immediately froze up.

My suspicion is the Ethernet device is being left in a bad state after suspend, so it fails to resume, and fails to function after resume if the cable is added post-resume.
Comment 6 Darren Hart 2014-08-07 20:36:59 UTC
Note, with the new kernel all I see on the console output upon resume is:

>>>>SecStartup
>>>>MemoryInit Done

Regardless of if it is successful or not.
Comment 7 Darren Hart 2014-08-07 23:24:25 UTC
Windows 64 doesn't experience this problem. Problem may well lie with the r8169.c Linux kernel Ethernet driver. Darren will investigate, instrument, and see what can be learned.
Comment 8 David_Wei 2014-08-28 10:35:19 UTC
Quick Update: This S3 network issue is caused by PCIe root port 0. States of Root port 0 is not properly saved and restored for S3. The weired thing is that the Realtek R8169 is actually on root port 2.
Comment 9 John 'Warthog9' Hawley 2014-09-08 17:56:40 UTC
Given the thread, I'm not sure why this got closed (as I'm not seeing anyone reporting it works)

quick test seems to indicate that suspend / resume works under Linux with 0.73 firmware:

root@intel-corei7-64:/sys/power# echo "mem" > state
[  365.163754] PM: Syncing filesystems ... done.
[  365.224165] Freezing user space processes ... (elapsed 0.001 seconds) done.
[  365.233379] Freezing remaining freezable tasks ... (elapsed 0.000 seconds) done.
[  365.242612] Suspending console(s) (use no_console_suspend to debug)

>>>>SecStartup
>>>>MemoryInit Done
[  365.270329] serial 00:04: disabled
[  365.323582] PM: suspend of devices complete after 56.819 msecs
[  365.325619] PM: late suspend of devices complete after 2.019 msecs
[  365.326765] r8169 0000:02:00.0: System wakeup enabled by ACPI
[  365.338058] xhci_hcd 0000:00:14.0: System wakeup enabled by ACPI
[  365.348723] PM: noirq suspend of devices complete after 23.059 msecs
[  365.348802] ACPI: Preparing to enter system sleep state S3
[  365.349736] PM: Saving platform NVS memory
[  365.354191] Disabling non-boot CPUs ...
[  365.355753] smpboot: CPU 1 is now offline
[  365.357475] ACPI: Low-level resume complete
[  365.357558] PM: Restoring platform NVS memory
[  365.358709] Enabling non-boot CPUs ...
[  365.358794] x86: Booting SMP configuration:
[  365.358797] smpboot: Booting Node 0 Processor 1 APIC 0x4
[  365.378722] CPU1 is up
[  365.379603] ACPI: Waking up from system sleep state S3
[  365.380161] acpi LNXPOWER:02: Turning OFF
[  365.380259] acpi LNXPOWER:01: Turning OFF
[  365.380331] acpi LNXPOWER:00: Turning OFF
[  365.402394] xhci_hcd 0000:00:14.0: System wakeup disabled by ACPI
[  365.424934] PM: noirq resume of devices complete after 44.514 msecs
[  365.426235] PM: early resume of devices complete after 1.172 msecs
[  365.426721] r8169 0000:02:00.0: System wakeup disabled by ACPI
[  365.465591] r8169 0000:02:00.0 eth0: link down
[  365.618523] PM: resume of devices complete after 192.023 msecs
[  365.762559] Restarting tasks ... done.
[  365.769972] video LNXVIDEO:00: Restoring backlight state
root@intel-corei7-64:/sys/power# [  366.837483] [drm] Enabling RC6 states: RC6 off, RC6p off, RC6pp off


NOTE: D2 stays active in this state, and I have not verified that the power draw drops to suspend levels.  I can confirm the HDMI screen goes black, and comes back on resume.
Comment 10 Darren Hart 2014-09-08 18:55:41 UTC
John, during your test, did you have an Ethernet cable plugged in and the interface configured?
Comment 11 Wang Liwei 2014-10-28 00:11:59 UTC
(In reply to comment #8)
> Quick Update: This S3 network issue is caused by PCIe root port 0. States of
> Root port 0 is not properly saved and restored for S3. The weired thing is
> that the Realtek R8169 is actually on root port 2.

Is info saved and restored by Firmware or by Linux?
Comment 12 David_Wei 2014-10-28 03:29:40 UTC
Firmware and OS drivers have different roles when saving/restoring info for S3. In terms of PCIe sub-system, firmware is responsible for low level settings, such as link training and common clock setting, payload size setting etc.