Bug 6132

Summary: Have persistent log data (/var/log)
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Mark Hatle <mark.hatle>
Component: coreAssignee: Chen Qi <Qi.Chen>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: bluelightning, jeff.polk, kristi, meta.mr.watcher, meta.watcher, milinevskyy, net147, Qi.Chen, richard.purdie, srifenbark
Version: 1.7   
Target Milestone: 2.4 M3   
Hardware: All   
OS: Multiple   
Whiteboard: 06 Jan 2017: IN PROGRESS REVIEW
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)

Description Mark Hatle 2014-04-08 15:36:58 UTC
There are some systems that require persistent log data.  Currently /var/log is defaulted to a tmpfs mount via the populate-volatiles subsystem.

In order to change this, it's required to modify the fs-perms, base-files and initscripts packages.  There should be a distribution switch to do this.

It may also be appropriate to ensure logrotate behavior is appropriate for disk (persistent) storage vs tmpfs based storage.
Comment 1 Chen Qi 2014-11-25 06:03:29 UTC
I've seen emails about persistent /var/log on Yocto mailing list.
I think this feature is important and needs to be done in 1.8.

//Chen Qi
Comment 2 Chen Qi 2015-03-25 01:58:20 UTC
Hi Mark,

Could you please help review the patches?

git://git.openembedded.org/openembedded-core-contrib ChenQi/persistent-var-log
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/persistent-var-log

Thanks,
Chen Qi
Comment 3 Scott Rifenbark 2015-06-02 20:28:07 UTC
Hi, 

It looks like the documentation for this bug would be to provide a glossary description for VOLATILE_LOG_DIR variable, which we currently do not document.  Aside from documenting this variable and perhaps adding some information else where in the YP doc set showing its use, would there be any other end-user documentation points for this issue?

Thanks, 
Scott
Comment 4 Chen Qi 2016-11-15 06:39:50 UTC
I'll rebase the patches and send them out again.
Comment 5 Scott Rifenbark 2016-11-22 00:00:05 UTC
Hey, 

Seems weird that we use "" to override "log" and make it persistent and not some more meaningful value for VOLATILE_LOG_DIR.  Anyway... to make sure I understand how VOLATILE_LOG_DIR works it basically defaults to "log", which makes the /var/log volatile.  If the user wants to save this log and make it persistent, they simply set VOLATILE_LOG_DIR to "".  

Are there any other user steps they need to take to get this done?

Thanks, 
Scott
Comment 6 Dmytro Milinevskyi 2016-12-08 14:35:43 UTC
Hello Chen, 
could you please bump the revisions of the recipes you've modified?
It's needed to have "smart" updates to work.
Thanks
Comment 7 Mark Hatle 2016-12-08 15:19:55 UTC
Are you using the PR server?  (You should be when doing target package upgrades.)

Community policy is not to bump PR numbers manually, except for a few extreme cases.

If you are using the PR server, and the PR did not update, please let us know (in a new bug) what the conditions are, as it may be a bug.
Comment 8 Dmytro Milinevskyi 2016-12-08 15:41:47 UTC
Mark, thanks for your reply. We are not using PR service as we can't be 100% sure that the build history is always persisted. I see your point and will bump the PRs of the packages manually in the private branch.
Thanks
Comment 9 Scott Rifenbark 2016-12-10 00:03:10 UTC
Any feedback on Comment 3 and Comment 5?  For documentation?

thanks, 
Scott
Comment 10 Chen Qi 2016-12-12 01:55:01 UTC
(In reply to comment #9)
> Any feedback on Comment 3 and Comment 5?  For documentation?
> 
> thanks, 
> Scott

Sorry I didn't notice it.
I've changed VOLATILE_LOG_DIR to default to "yes", and we use 'no' to override the default value to make /var/log on target persistent.
For documentation, apart from glossary description for VOLATILE_LOG_DIR, a section about 'making /var/log persistent' might be needed. I've seen some users on yocto mailing list requiring this feature, so making it clear in docs might be better.
The only problem here is that the patches are not accepted yet. When they are merged I'll change the state of this bug.

Regards,
Chen Qi
Comment 11 Scott Rifenbark 2016-12-30 18:24:56 UTC
Hi, 

I updated a couple spots in the ref-manual to deal with this issue:

 1 - in the "Image Features" section there is a list of features the user can
     apply.  One being "post-install-logging", which talks about the /var/log
     directory.  I have created a note in that entry mentioning the 
     VOLATILE_LOG_DIR variable as the way to control the persistence of 
     this target directory.  See http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#ref-features-image

 2 - In the reference manual, I created a new glossary entry for the 
     VOLATILE_LOG_DIR variable.  See http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#var-VOLATILE_LOG_DIR

Take a look and let me know if these are okay. 

Thanks, 
Scott
Comment 12 Scott Rifenbark 2017-01-06 15:29:38 UTC
I updated the "Whiteboard" information.  Please let me know if the the doc changes I have made are acceptable.

Scott
Comment 13 Scott Rifenbark 2017-01-18 21:47:44 UTC
So you changed the target... does that mean we are good?  If so, let me know and I can update the status to RESOLVED and put the doc flag to "done."

Thanks, 
Scott
Comment 14 Chen Qi 2017-01-19 01:50:05 UTC
(In reply to comment #13)
> So you changed the target... does that mean we are good?  If so, let me know
> and I can update the status to RESOLVED and put the doc flag to "done."
> 
> Thanks, 
> Scott

Got it.
Comment 15 Scott Rifenbark 2017-01-19 23:53:43 UTC
Thanks, 

Setting to RESOLVED and putting the doc flag to "done."

Scott
Comment 16 Chen Qi 2017-08-16 11:54:51 UTC
I've no idea why I replied 'got it' ...
And this answer led to communication mistake, resulting in this bug changed into a wrong state.
I've sent out rebased patches to the mailing list, waiting for review.
Sorry for my mistake.
Comment 17 Scott Rifenbark 2017-08-16 18:42:05 UTC
Ok - So I am not sure where we are on the doc changes.  You sent out rebased patches for the fix.  Do the documentation changes I made in Comment 11 still hold true?

Thanks,
Scott
Comment 18 Chen Qi 2017-08-17 01:34:14 UTC
(In reply to comment #17)
> Ok - So I am not sure where we are on the doc changes.  You sent out rebased
> patches for the fix.  Do the documentation changes I made in Comment 11
> still hold true?
> 
> Thanks,
> Scott

It's true for my patches. But my patches haven't been merged.
If they are merged, then there's no need to change the doc.

Regards,
Chen Qi
Comment 19 Chen Qi 2017-08-21 02:19:52 UTC
Merged in master. No need for changing the current doc.
Sorry for my mistake.

Regards,
Chen Qi