Bug 1589

Summary: Disk space monitoring
Product: [Build System, Metadata & Runtime] Meta-yocto Reporter: Richard Purdie <richard.purdie>
Component: meta-yoctoAssignee: Robert Yang <liezhi.yang>
Status: VERIFIED FIXED QA Contact:
Severity: enhancement    
Priority: Low CC: jessica.zhang, liang.li2, liezhi.yang, poky.bs.watcher, poky.watcher, song.liu, yi.zhao
Version: unspecified   
Target Milestone: 1.2 M3   
Hardware: x86   
OS: Multiple   
Whiteboard: (P3) (QA testing completed) Done
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Richard Purdie 2011-10-04 07:09:45 UTC 
It would be good to monitor disk availability during builds and warn the user if it is running low. 

This gets complicated by the problem that different parts of the build may be on different mount points/partitions. This functionality may only focus on a few directories, for example: poky/build and poky/build/downloads initially to solve this problem.

We'd also want to track available inode numbers as well as space.
Comment 1 Robert Yang 2011-11-06 18:19:32 UTC 
This is in M2
Comment 2 Robert Yang 2011-11-06 18:22:40 UTC 
Sorry, it should be in M1
Comment 3 Robert Yang 2011-11-10 23:21:54 UTC 
Move it to M2, since i should work on bug 1229: bitbake-runtask doesn't work, Wind River
is using bitbake-runtask in yocto 1.0, and may go on using it in 1.2, but it doesn't work currently.
Comment 4 Robert Yang 2011-12-13 00:33:05 UTC 
Here is the design document:

* Summary
  Monitor disk availability and warn the user if it is
  running low, the user should specify the following values:

  # Set the directories to monitor for disk usage
  BB_DISKMON_DIRS = "${TMPDIR} ${DL_DIR}"
  # Set the minimum amount of disk space to warn
  BB_DISKMON_MINSPACE = "1GB"
  # Set the monitor frequency
  BB_DISKMON_INTERVAL = "60s"

  If the variables are not set, it doesn't get activated, it would be 
  disabled by default.

* Implementation
  1) The main code will be in lib/bb/monitordisk.py
  2) The bin/bitbake will invoke the code in lib/bb/monitordisk.py
  3) Use the multiprocessing and time module, the multiprocessing is 
     used to fork a process to monitor the diskspace, and use the time's
     sleep as a timer
  4) Use the "print" to print the warning message since the monitor 
     process is a subprocess of bitbake, it can't use the same the logger
     with bitbake.
  5) The warning format is similar to:
     WARNING: the space of /buildarea/test/downloads is running low (2GB left)
Comment 5 Robert Yang 2012-01-16 01:53:06 UTC 
Move it to M3 since it is in patch review, and the Chinese New Year will begin.
Comment 7 liang li 2012-02-29 02:31:31 UTC 
verified on:
git info: 927bec5f72230be4f2452d9ef5a747b2c00cb781
git branch: poky/master
follow test steps below:
1.set build env.
2.add following info to local.conf :
BB_DISKMON_DIRS = "STOPTASKS,${TMPDIR},40G,4510K"
BB_DISKMON_WARNINTERVAL = "50M,5K"
3.check free disk space of filesystem which tmp/ is mounted,only 35G left.
4.run "bitbake man",the build process will stop and nothing will build.
5.change "STOPTASKS" to "ABORT",build process will stop and nothing will build.
6.change "STOPTASKS" to "WARN", build process will continue and warnings will output to screen before build and when disk space decrease 50M lower, warning will also output to screen in build process.
7.also check free inodes of the filesystem which tmp/ is mounted,the same result to the free disk space.