Bug 8264

Summary:	LogrotateTest Error and Fail on x86-64-lsb
Product:	[Build System, Metadata & Runtime] OE-Core	Reporter:	Beth Flanagan <elizabeth.flanagan>
Component:	Scripts and Tools	Assignee:	Leonardo Sandoval Gonzalez <leonardo.sandoval.gonzalez>
Status:	VERIFIED FIXED	QA Contact:	Bogdan Alexandru Voiculescu <bogdanx.a.voiculescu>
Severity:	normal
Priority:	Medium+	CC:	bogdanx.a.voiculescu, ross.burton
Version:	1.7.3
Target Milestone:	2.0
Hardware:	x86
OS:	Multiple
Whiteboard:
OS type for building Yocto:	---	Type of Regression:	---
Verified:		Documentation change:	No (bug/feature does not impact docs)
Bug Depends on:
Bug Blocks:	7983

Description Beth Flanagan 2015-09-07 12:31:53 UTC

https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-lsb

Logrotate can't create directory during setup/or timesout, causing ERROR in LogrotateTest.

| ======================================================================
| ERROR: test_2_logrotate (oeqa.runtime.logrotate.LogrotateTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/utils/decorators.py", line 102, in wrapped_f
|     return func(*args)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f
|     return f(*args)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/runtime/logrotate.py", line 28, in test_2_logrotate
|     self.assertTrue(int(output)>=3, msg = "new logfile could not be created. List of files within log directory: %s" %(self.target.run('ls -la /home/root/logrotate_dir')[1]))
| ValueError: invalid literal for int() with base 10: 'ls: cannot access /home/root/logrotate_dir/: No such file or directory\n0'
| 
| ======================================================================
| FAIL: test_1_logrotate_setup (oeqa.runtime.logrotate.LogrotateTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f
|     return f(*args)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/runtime/logrotate.py", line 18, in test_1_logrotate_setup
|     self.assertEqual(status, 0, msg = "Could not create logrotate_dir. Output: %s" % output)
| AssertionError: -15 != 0 : Could not create logrotate_dir. Output:
| Process killed - no output for 300 seconds. Total running time: 300 seconds.
| 
| ----------------------------------------------------------------------
| Ran 59 tests in 5879.028s
| 
| FAILED (failures=1, errors=1, skipped=8)

Comment 1 Leonardo Sandoval Gonzalez 2015-09-23 13:40:50 UTC

Another issue coming from 

https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-lsb/builds/502

| ======================================================================
| FAIL: test_2_logrotate (oeqa.runtime.logrotate.LogrotateTest)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/utils/decorators.py", line 102, in wrapped_f
|     return func(*args)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/utils/decorators.py", line 90, in wrapped_f
|     return f(*args)
|   File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-64-lsb/build/meta/lib/oeqa/runtime/logrotate.py", line 26, in test_2_logrotate
|     self.assertEqual(status, 0, msg = "logrotate service could not be reloaded. Status and output: %s and %s" % (status, output))
| AssertionError: 143 != 0 : logrotate service could not be reloaded. Status and output: 143 and
| Process killed - no output for 300 seconds. Total running time: 300 seconds.
| 
| ----------------------------------------------------------------------

Still investigating the root cause.

Comment 2 Leonardo Sandoval Gonzalez 2015-09-28 13:52:45 UTC

Seems to be a random failure. Since Sep/16. this issue has not shown again. Unfortunately, when I tried to the log from Sep/16, the log was already removed, so I have no system data to check. I will keep monitoring.

Comment 3 Leonardo Sandoval Gonzalez 2015-09-29 14:57:19 UTC

There are two problem not related to logrotate but the logrotate unit tests helped to bring them out

1) @skip* decorators: A patch sent t to the mailing list titled '[PATCH] oeqa/utils/decorators: Append the testname without the full path' which allows indicating unit tests without full path. This will prevent running test_2_logrotate if this depends on test_1_logrotate_setup through the decorator @skipUnlessPassed, as observed on https://bugzilla.yoctoproject.org/show_bug.cgi?id=8264#c0

2) timeouts: sometimes, there are timeouts when running either mkdir or logrotate commands. I do not know for sure what is causing this type of failures, but it may the case that the connection (ssh) is being lost for a while, so timeout is triggered. This is not related to the logrotate package, it is a connectivity issue and should be reported as a separate bug. Log reported on https://bugzilla.yoctoproject.org/show_bug.cgi?id=8264#c1

Comment 4 Bogdan Alexandru Voiculescu 2015-10-23 13:04:40 UTC

Fixed in build nightly-x86-64-lsb #528: 
https://autobuilder.yoctoproject.org/main/builders/nightly-x86-64-lsb/builds/528/steps/Running%20Sanity%20Tests/logs/stdio


| test_1_logrotate_setup (oeqa.runtime.logrotate.LogrotateTest) ... ok
| test_2_logrotate (oeqa.runtime.logrotate.LogrotateTest) ... ok