| Summary: | Signs of a race in do_testimage | ||
|---|---|---|---|
| Product: | [QA/Testing] Build Testing | Reporter: | Joshua Lock <joshuagloe> |
| Component: | general | Assignee: | Aníbal Limón <anibal.limon> |
| Status: | RESOLVED FIXED | QA Contact: | |
| Severity: | normal | ||
| Priority: | High | CC: | alejandro.hernandez, anibal.limon, leonardo.sandoval.gonzalez, liezhi.yang, meta.mr.watcher, meta.watcher |
| Version: | 2.2 | ||
| Target Milestone: | 2.2 M4 | ||
| Hardware: | x86 | ||
| OS: | Multiple | ||
| Whiteboard: | |||
| OS type for building Yocto: | --- | Type of Regression: | --- |
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |
|
Description
Joshua Lock
2016-09-06 09:58:17 UTC
This sure looks like a race condition, since the file does exist, I'm guessing it was created afterwards: ls -l /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/build/results-bitbake-worker.log lrwxrwxrwx 1 pokybuild users 116 Sep 5 15:13 /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/build/results-bitbake-worker.log -> /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-x86-lsb/build/build/results-bitbake-worker.20160905220801.log The line where it fails is inside an if, that's supposed to check that:
0192: # Create symlink to the current log
0193: if os.path.lexists(linkfile):
*** 0194: os.remove(linkfile)
According to python's documentation:
os.path.lexists:
Return True if path refers to an existing path. Returns True for broken symbolic links. Equivalent to exists() on platforms lacking os.lstat().
So it could return true for a broken link, but that still doesn't explain why the os.remove() would fail for a broken link.
Or this is a very specific race, where the file is removed, an maybe recreated by some other process just after checking if it existed
(In reply to comment #2) > The line where it fails is inside an if, that's supposed to check that: > > 0192: # Create symlink to the current log > 0193: if os.path.lexists(linkfile): > *** 0194: os.remove(linkfile) > > > According to python's documentation: > > os.path.lexists: > > Return True if path refers to an existing path. Returns True for broken > symbolic links. Equivalent to exists() on platforms lacking os.lstat(). > > So it could return true for a broken link, but that still doesn't explain > why the os.remove() would fail for a broken link. > > Or this is a very specific race, where the file is removed, an maybe > recreated by some other process just after checking if it existed It should be a race issue, for example: bitbake <image1> <image2> -c testimage Then two testimage may handle it at the same, I think that we can rename results-bitbake-worker.log to something liek core-image-minimal-testimage.log to fix the problem. (In reply to comment #3) > > It should be a race issue, for example: > bitbake <image1> <image2> -c testimage > > Then two testimage may handle it at the same, I think that we can rename > results-bitbake-worker.log to something liek > core-image-minimal-testimage.log to fix the problem. I see no harm in renaming the worker log to some image-specific name, and I agree I believ this would likely fix the issue. (In reply to comment #4) > (In reply to comment #3) > > > > > It should be a race issue, for example: > > bitbake <image1> <image2> -c testimage > > > > Then two testimage may handle it at the same, I think that we can rename > > results-bitbake-worker.log to something liek > > core-image-minimal-testimage.log to fix the problem. > > I see no harm in renaming the worker log to some image-specific name, and I > agree I believ this would likely fix the issue. It's a reasonable solution, one I was also considering. The trouble is how do we get the name of the target bitbake was invoked with into the decorator that sets up the log filename? AFAICT the decorator and the classes it decorates have no idea what the target the parent bitbake process was invoked with is. (In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #3) > > > > > > > > It should be a race issue, for example: > > > bitbake <image1> <image2> -c testimage > > > > > > Then two testimage may handle it at the same, I think that we can rename > > > results-bitbake-worker.log to something liek > > > core-image-minimal-testimage.log to fix the problem. > > > > I see no harm in renaming the worker log to some image-specific name, and I > > agree I believ this would likely fix the issue. > > It's a reasonable solution, one I was also considering. The trouble is how > do we get the name of the target bitbake was invoked with into the decorator > that sets up the log filename? > > AFAICT the decorator and the classes it decorates have no idea what the > target the parent bitbake process was invoked with is. Another possible solution will be usage of bb.utils.lock before and after the soft link related code. |