Bug 8311

Correction: 
  It doesn't recover from sleep state.

I could NOT reproduce this on my 32bit nuc or my 32bit nuc

32bit NUC DC3217IYE
Intel(R) Core(TM) i3-3217U CPU @ 1.80GHz
- Worked

64bit NUC D54250WYK
Intel(R) Core(TM) i5-4250U CPU @ 1.30GHz
- Worked

Can you provide the dmesg logs and maybe the Xorg logs.  Also please try to start a Terminal on the target and see if you get any keyboard response after the resume.

Please provide some additional logs as requested.

Created attachment 2758 [details]
dmesg file

dmesg and Xorg.0.log attached

Created attachment 2759 [details]
xorg

Since it does not wake up, the attached dmesg and Xorg.log are not really 
useful, I need them after the sleep really!

Something else to try, this might help determine if it's Linux Yocto Kernel, 
setting or ...

Please try the following:
1) using intel-corei7-64 kernel (4.1 based) - If this fails then:
2) try older genericx86-64 3.19 kernel - if this fails then try
3) Another Linux OS (preferably one with newer kernel)

I tried to test this issue using an image with 3.19 kernel version for genericx86-64 and I discovered that if I try to wake up the device in the first minute it resumes, but if I let the DUT in sleep state for 5 or more minutes it doesn't recover anymore. It happens also when I use an intel-corei7-64 kernel (4.1 based).

But this issue can be reproduced only on NUCC5i5RYH model of NUC. Using a  D54250WYK model I can't reproduce the bug.

Tried:
1) intel-corei7-64 kernel (4.1 based) - NUCC5i5RYH 
   Wakes up on first 2 minutes or so 
   If stays longer, it won't
  
2) Genericx86-64 (3.19 kernel) 
   Same behaviour - NUCC5i5RYH 
   Not happening on older NUC version
  
3) Other OS 
   Fedora liveboot on NUCC5i5RYH doesn't reproduce (tried with 10 minutes sleep)

Which version of Fedora Live boot and more specifically which kernel?

Another thing that I observed related to this issue is that is not reproducible 100%. Sometimes it wakes up even after 10 minutes(observed on genericx86-64 with 3.19 kernel).

The issue reproducibility percentage is about 70%.

Cristina, some testing tips here:
https://01.org/linuxgraphics/documentation/development/how-debug-suspend-resume-issues

Unfortunately we don't much use modules so you can't unload them...
Possibly useful tests:
* text mode suspend, maybe using core-image-minimal?
* the "Check if resume hang is caused by graphics driver" test (see link)
* "suspend/resume by skipping bios" test (see link)

After that, the next step would probably be enabling PM_DEBUG and PM_TRACE in the kernel and trying resume tracing (https://www.kernel.org/doc/Documentation/power/s2ram.txt).

Saul, we wouldn't have kernels with those enabled easily available?

I can build a kernel with those enabled, I hope to have one of those NUC5i5RYH machines by next week. It seems to be a hardware specific issue.

This was REOPENED

Still trying to reproduce this, has this been an issue with the new 2.1_m1?  I 
tested both with genericx86-64 and a core-i7 BSP both were able to wake up 
after 10+ minutes of mem level sleep.

Changed to need info for more testing.

Tested with core-image-sato-sdk image for genericx86-64, built locally, poky commit: 2cd061a29a94fdcdfec86732e6bcbb71c12c4afc and the issue can be reproduced, but isn't 100% reproducible(4/10). I tried to take the dmesg before and after resuming from sleep using the steps from the link posted by Jussi in comment 11 ("Check if resume hang is caused by graphics driver" test), but the after-dmesg doesn't exist after reboot. 

The model of NUC is NUC5i5RYH.

Created attachment 2919 [details]
dmesg_log

I tried to take the dmesg log when it doesn't resume, also, by using the command "dmesg -w | tee log_sleep.txt" (see the attachment)

*** Bug 9135 has been marked as a duplicate of this bug. ***

(In reply to comment #19)
> *** Bug 9135 has been marked as a duplicate of this bug. ***

Reproducible 100% in NUC6i5SYH. 
BUILD: 2.1_M2.rc3: 152914f2983c5d69001de1d46ce99547fa1e75fe
TEPS TO REPRODUCE: 
1. boot system and launch terminal; check output of "date" and launch script "continue.sh"
2. echo "mem" > /sys/power/state
3. After system go into S3 mode, move mouse or press any key to make it resume
4. Check "date" and script "continue.sh"
5. Check if application can work as normal

continue.sh as below:

#################
#!/bin/sh

i=1
while [ 0 ]
do
 echo $i
 sleep 1
 i=$((i+1))
done
#################

EXPECTED OUTCOME: Screen should resume back and script can run continuously incrementing the i's value from where it was before.

ACTUAL OUTCOME: The screen was not resumed successfully, after several tries.
http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc3/machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg

We couldnt' reproduce the issue anymore on NUC5i5RYH for 2.1_M2.rc3.

But, at stated as in an above comment, for this UC5i5RYH it wasn't 100% reproductible.

Assigned to myself since there is another bz8712, which could be a different issue though.

http://autobuilder.yoctoproject.org/pub/releases/yocto-2.1_M2.rc3/machines/genericx86-64/core-image-sato-sdk-genericx86-64.hddimg 

I am not sure if we are facing different issues cross different NUC models. But I can reproduce issue with about image. Once NUC goes to sleep mode (Seeing led goes orange blinking mode), display (HDMI) cannot come back. The kernel in the image is 4.1.7 and I have to add "i915.preliminary_hw_support=1" in command line before booting kernel to get a working display. But this trick cannot help on this suspend-resume issue. I will do some investigation on my NUC gen 6 first.

I narrowed it down a little bit. Based on my test on core-minimal-build:

The 4.4 kernel supported in OE doesn't show this issue on my NUC gen 6
The 4.1.7 which is current kernel by default shows 100% this issue on same NUC gen 6 machine.

I will try sato-sdk later though I feel it is a kernel bug irrelevant to poky distro.
 
I am not sure how soon genericx86-64 will move to 4.4 recipe. Otherwise I need to find fixes to backport.

Since this is filed against 2.0 [1.9] (using the 4.1 kernel) we should try to 
identify a fix for 4.1 as some people will continue to use the 4.1 kernel in 
2.1 Yocto Project.

After 11 iterations between yocto-linux 4.1.17 and 4.4.1, bisecting indicates a good revision which should be reasonable for next step:

Bisecting: 0 revisions left to test after this (roughly 0 steps)
[f377ea88b862bf7151be96d276f4cb740f8e1c41] Merge branch 'drm-next' of git://people.freedesktop.org/~airlied/linux


It shows backtraces as warnings in kernel log, but screen and console can come back and functional, so it may not be so good as skl support on 4.4.

There should be multiple i915 fixes in that merged drm branch for 4.3, we could have a close look, though I am not sure if the final fix can be ported back to 4.1.15/17 yocto kernel very well.

I attached bisect log for reference.

Created attachment 3009 [details]
linux_yocto_bisect_between_4.1.17_4.4.1

This is a following up based on 1st round full kernel bisecting to find patches for a back porting to 4.1 for this issue.

1st tier: f377ea8 in 4.3 has been verified and tested for this bug and it is "good" point that screen and system can resumed successfully. I took it as the next step. This commit is a merge one with 2 parents at 2nd level in history :

2nd tier: 51e771c, 73bf1b7

51e771c:
system: alive after resume (network okay)
screen: full-screen color strips after resume. No console.

73bf1b7:
system: hang after resume (network not working)
screen: black screen after resume

Conclusion of 2nd tier:
The acceptable functional status for this ticket is actually a co-effect from two insufficient branches which have their own issues respectively. That also means there should not be a silver-bullet fix to dig out. The two parents are merge commits so we need to take further steps to narrow down any fix set.

3rd tier: children abebcdf and 01b944fe of 51e771c. Children 5b78cb6 and 35c7a95 of 73bf1b7. There are the test result:

abebcdf of 51e771c:
system: alive after resume (network okay)
screen: full-screen color strips after resume. No console.

01b944fe of 51e771c:
system: hang after resume (network not working)
screen: black screen after resume

5b78cb6 of 73bf1b7:
system: hang after resume (network not working)
screen: black screen after resume

35c7a95 of 73bf1b7:
system: hang after resume (network not working)
screen: black screen after resume

Conclusion of 3rd tier:
If we want to chase down further, abebcdf plus any of fixes of screen from children of 73bf1b7 could be a good try. Then we can check exact patch set within abebcdf.

Unfortunately, all of these commits in 3rd tier are merge commits. We are facing 8 branches in history now. And symptoms are not helpful since some children have no effect on it. I feel eventually the task will become "back port skylake graphic driver from 4.3 to 4.1 in yocto. This seems beyond a simple bug fix.

Saul, could you let me know your thoughts of direction to resolve this bug based on these findings? I think technically we could bring an answer, but just don't know how long it could take to bring a satisfying patch set for 4.1.

Based on discussion today. Skylake graphic support of 4.1 kernel is not ready and our 4.4 kernel works fine for this issue. Close it as won't fix.

Based on discussion today. Skylake graphic support of 4.1 kernel is not ready and our 4.4 kernel works fine for this issue. Close it as won't fix.