| Summary: | [archiver.bbclass] symlinks causes infinite recursive directory loop | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Valentin Cobelea <valentin.cobelea> | ||||||||||
| Component: | core | Assignee: | Robert Yang <liezhi.yang> | ||||||||||
| Status: | RESOLVED FIXED | QA Contact: | |||||||||||
| Severity: | normal | ||||||||||||
| Priority: | Medium | CC: | meta.mr.watcher, meta.watcher, sgw, valentin.cobelea | ||||||||||
| Version: | unspecified | ||||||||||||
| Target Milestone: | 1.6 M4 | ||||||||||||
| Hardware: | Other | ||||||||||||
| OS: | ppc | ||||||||||||
| See Also: | https://bugzilla.yoctoproject.org/show_bug.cgi?id=5113 | ||||||||||||
| Whiteboard: | |||||||||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||||||||
| Verified: | Documentation change: | Don't know | |||||||||||
| Attachments: |
|
||||||||||||
|
Description
Valentin Cobelea
2013-08-06 09:27:41 UTC
Created attachment 1410 [details]
starting solution (fixed udev package issue)
Additional information: -the build was based on the 'meta-fsl-ppc' layer; -image type: 'core-image-minimal'; Also since the 'diff' has no mechanism to not follow symlinks, we need to have this covered in the 'archiver.bbclass' (this is what the proposed solution does). There are plenty of other packages, with synlinks inside/after build, that might fail while doing the 'do_dumpdata_create_diff_gz'. Also the 'subprocess.call()' is not bullet proof, we should use 'subprocess.Popen()' in order to have more control over the spawned process (http://docs.python.org/2/library/subprocess.html#popen-constructor). We could trigger 'terminate' to it, in case it goes on indefinitely. As an alternative solution for getting all the "folder" symlink is:
import os
for root, dirs, files in os.walk('./',followlinks=True):
if (os.path.islink(root)):
#add it to the exclude list
This could replace the whole 'get_sym_links_list()' method from the attached patch.
Created attachment 1414 [details]
only recursive symlinks are added to the
Refined version of the proposed solution is attached here ('recursive-solution.tar.gz').
Now only the recursive symlinks are added to the exclusion file used by DIFF.
If you have a solution, please accept this bug and provide a solution to the OE-Core list as a proper patch. There are only the proposed solutions (both attached), but these are only starting points for the solution I am hopping to see from anyone in charge of the 'archiver.bbclass'. Even with what I made the problem still replicates but each time for another package, this would be why I consider my solutions only a part of the real deal and a starting point. (In reply to comment #6) > There are only the proposed solutions (both attached), but these are only > starting points for the solution I am hopping to see from anyone in charge > of the 'archiver.bbclass'. > Even with what I made the problem still replicates but each time for another > package, this would be why I consider my solutions only a part of the real > deal and a starting point. Hi Valentin Cobelea, I can't reproduce the error, and I check the udev-182-r8.diff.gz it is OK: -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182-r8.diff.gz Here is the local.conf: MACHINE ?= "p4080ds" CONF_VERSION = "1" ARCHIVER_MODE ?= "patched" ARCHIVER_MODE[type] ?= "tar" I took a look at your patches, I'm sorry that I didn't know what did it try to fix, would you please provide more info ? (In reply to comment #7) Hi, The issue occurred randomly, not on a specific package. We consider that there should be several test runs (several full builds with the archiver mode enabled for "patched" and "tar" p- as you already tried) in order to reproduce this issue which makes build not completing. My patches addressed the recursive symlinks (the last one is directly for this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing each of them from the list. Best regards, Valentin > (In reply to comment #6) > > There are only the proposed solutions (both attached), but these are only > > starting points for the solution I am hopping to see from anyone in charge > > of the 'archiver.bbclass'. > > Even with what I made the problem still replicates but each time for another > > package, this would be why I consider my solutions only a part of the real > > deal and a starting point. > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > udev-182-r8.diff.gz it is OK: > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > r8.diff.gz > > Here is the local.conf: > MACHINE ?= "p4080ds" > CONF_VERSION = "1" > ARCHIVER_MODE ?= "patched" > ARCHIVER_MODE[type] ?= "tar" > > I took a look at your patches, I'm sorry that I didn't know what did it try > to fix, would you please provide more info ? (In reply to comment #8) > (In reply to comment #7) > Hi, > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: 1) For your second patch: - What is the readlink_abspath(l) used for ? Seems that no one calls it. - What does this comment mean, please ? + # get symlinks and folders in two separated lists + # using only 'root' we will have here only the symlinks pointing to directories 2) I think that you are trying to recursive symlinks issue ? But the diff can detect this on my host: $ diff -Nur dir1/ dir2/ diff: dir1/aa/ln3: recursive directory loop Suppose that there are recursive symlinks in dir1. Maybe there are other problems ? I've tried to build the udev many times, didn't see the error, I think that if it is caused by recursive symlink, we can find it every time. Any more comments, please ? // Robert > The issue occurred randomly, not on a specific package. > We consider that there should be several test runs (several full builds with > the archiver mode enabled for "patched" and "tar" p- as you already tried) > in order to reproduce this issue which makes build not completing. > > My patches addressed the recursive symlinks (the last one is directly for > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > each of them from the list. > > Best regards, > Valentin > > > > (In reply to comment #6) > > > There are only the proposed solutions (both attached), but these are only > > > starting points for the solution I am hopping to see from anyone in charge > > > of the 'archiver.bbclass'. > > > Even with what I made the problem still replicates but each time for another > > > package, this would be why I consider my solutions only a part of the real > > > deal and a starting point. > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > udev-182-r8.diff.gz it is OK: > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > r8.diff.gz > > > > Here is the local.conf: > > MACHINE ?= "p4080ds" > > CONF_VERSION = "1" > > ARCHIVER_MODE ?= "patched" > > ARCHIVER_MODE[type] ?= "tar" > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > to fix, would you please provide more info ? (In reply to comment #9) Hello, Please use only the "only recursive symlinks are added to the (5.95 KB, application/gzip) 2013-08-06 13:48 UTC," patch, this addresses what I think a fix for the symlinks. The idea behind the algorithm there is that I am excluding the recursive symlinks by crating a list of folders and another one of symlinks. Those that are recursive are not added to the list, hence the issue will not occur. Also, please run full builds, not for 'udev' only, the issue occurs for any package (usually system related, like udev is) that has recursive symlinks. If you consider that there is a better solution to this issue, please apply it, my attached patches were just proposed solutions. Thank you, Valentin > (In reply to comment #8) > > (In reply to comment #7) > > Hi, > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > > 1) For your second patch: > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > - What does this comment mean, please ? > + # get symlinks and folders in two separated lists > + # using only 'root' we will have here only the symlinks pointing to > directories > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > can detect this on my host: > > $ diff -Nur dir1/ dir2/ > diff: dir1/aa/ln3: recursive directory loop > > Suppose that there are recursive symlinks in dir1. > > Maybe there are other problems ? I've tried to build the udev many times, > didn't see the error, I think that if it is caused by recursive symlink, we > can find it every time. > > Any more comments, please ? > > // Robert > > > The issue occurred randomly, not on a specific package. > > We consider that there should be several test runs (several full builds with > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > in order to reproduce this issue which makes build not completing. > > > > My patches addressed the recursive symlinks (the last one is directly for > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > each of them from the list. > > > > Best regards, > > Valentin > > > > > > > (In reply to comment #6) > > > > There are only the proposed solutions (both attached), but these are only > > > > starting points for the solution I am hopping to see from anyone in charge > > > > of the 'archiver.bbclass'. > > > > Even with what I made the problem still replicates but each time for another > > > > package, this would be why I consider my solutions only a part of the real > > > > deal and a starting point. > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > r8.diff.gz > > > > > > Here is the local.conf: > > > MACHINE ?= "p4080ds" > > > CONF_VERSION = "1" > > > ARCHIVER_MODE ?= "patched" > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > to fix, would you please provide more info ? (In reply to comment #9) > (In reply to comment #8) > > (In reply to comment #7) > > Hi, > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > s/still quite/still don't quite/ > 1) For your second patch: > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > - What does this comment mean, please ? > + # get symlinks and folders in two separated lists > + # using only 'root' we will have here only the symlinks pointing to > directories > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > can detect this on my host: > s/trying to recursive/trying to fix the recursive/ > $ diff -Nur dir1/ dir2/ > diff: dir1/aa/ln3: recursive directory loop > > Suppose that there are recursive symlinks in dir1. > > Maybe there are other problems ? I've tried to build the udev many times, > didn't see the error, I think that if it is caused by recursive symlink, we > can find it every time. > > Any more comments, please ? > > // Robert > > > The issue occurred randomly, not on a specific package. > > We consider that there should be several test runs (several full builds with > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > in order to reproduce this issue which makes build not completing. > > > > My patches addressed the recursive symlinks (the last one is directly for > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > each of them from the list. > > > > Best regards, > > Valentin > > > > > > > (In reply to comment #6) > > > > There are only the proposed solutions (both attached), but these are only > > > > starting points for the solution I am hopping to see from anyone in charge > > > > of the 'archiver.bbclass'. > > > > Even with what I made the problem still replicates but each time for another > > > > package, this would be why I consider my solutions only a part of the real > > > > deal and a starting point. > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > r8.diff.gz > > > > > > Here is the local.conf: > > > MACHINE ?= "p4080ds" > > > CONF_VERSION = "1" > > > ARCHIVER_MODE ?= "patched" > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > to fix, would you please provide more info ? (In reply to comment #11) Got it, thank you. Please see my reply above. Best regards, Valentin > (In reply to comment #9) > > (In reply to comment #8) > > > (In reply to comment #7) > > > Hi, > > > > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > > > > s/still quite/still don't quite/ > > > 1) For your second patch: > > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > > - What does this comment mean, please ? > > + # get symlinks and folders in two separated lists > > + # using only 'root' we will have here only the symlinks pointing to > > directories > > > > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > > can detect this on my host: > > > > s/trying to recursive/trying to fix the recursive/ > > > $ diff -Nur dir1/ dir2/ > > diff: dir1/aa/ln3: recursive directory loop > > > > Suppose that there are recursive symlinks in dir1. > > > > Maybe there are other problems ? I've tried to build the udev many times, > > didn't see the error, I think that if it is caused by recursive symlink, we > > can find it every time. > > > > Any more comments, please ? > > > > // Robert > > > > > The issue occurred randomly, not on a specific package. > > > We consider that there should be several test runs (several full builds with > > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > > in order to reproduce this issue which makes build not completing. > > > > > > My patches addressed the recursive symlinks (the last one is directly for > > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > > each of them from the list. > > > > > > Best regards, > > > Valentin > > > > > > > > > > (In reply to comment #6) > > > > > There are only the proposed solutions (both attached), but these are only > > > > > starting points for the solution I am hopping to see from anyone in charge > > > > > of the 'archiver.bbclass'. > > > > > Even with what I made the problem still replicates but each time for another > > > > > package, this would be why I consider my solutions only a part of the real > > > > > deal and a starting point. > > > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > > r8.diff.gz > > > > > > > > Here is the local.conf: > > > > MACHINE ?= "p4080ds" > > > > CONF_VERSION = "1" > > > > ARCHIVER_MODE ?= "patched" > > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > > to fix, would you please provide more info ? (In reply to comment #10) > (In reply to comment #9) > > Hello, > > Please use only the > "only recursive symlinks are added to the (5.95 KB, application/gzip) > 2013-08-06 13:48 UTC," patch, this addresses what I think a fix for the > symlinks. > Yes, this is what I use, and I wonder what is the function readlink_abspath() used for ? Seems no one calls it. > The idea behind the algorithm there is that I am excluding the recursive > symlinks by crating a list of folders and another one of symlinks. Those > that are recursive are not added to the list, hence the issue will not occur. > > Also, please run full builds, not for 'udev' only, the issue occurs for any > package (usually system related, like udev is) that has recursive symlinks. > Yes, I've also tried the full builds several times, but didn't find the error, would you please show me a recursive symlink for the package? > If you consider that there is a better solution to this issue, please apply > it, my attached patches were just proposed solutions. > I'm glad to fix it if I can understand it correctly. // Robert > Thank you, > Valentin > > > > (In reply to comment #8) > > > (In reply to comment #7) > > > Hi, > > > > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > > > > 1) For your second patch: > > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > > - What does this comment mean, please ? > > + # get symlinks and folders in two separated lists > > + # using only 'root' we will have here only the symlinks pointing to > > directories > > > > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > > can detect this on my host: > > > > $ diff -Nur dir1/ dir2/ > > diff: dir1/aa/ln3: recursive directory loop > > > > Suppose that there are recursive symlinks in dir1. > > > > Maybe there are other problems ? I've tried to build the udev many times, > > didn't see the error, I think that if it is caused by recursive symlink, we > > can find it every time. > > > > Any more comments, please ? > > > > // Robert > > > > > The issue occurred randomly, not on a specific package. > > > We consider that there should be several test runs (several full builds with > > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > > in order to reproduce this issue which makes build not completing. > > > > > > My patches addressed the recursive symlinks (the last one is directly for > > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > > each of them from the list. > > > > > > Best regards, > > > Valentin > > > > > > > > > > (In reply to comment #6) > > > > > There are only the proposed solutions (both attached), but these are only > > > > > starting points for the solution I am hopping to see from anyone in charge > > > > > of the 'archiver.bbclass'. > > > > > Even with what I made the problem still replicates but each time for another > > > > > package, this would be why I consider my solutions only a part of the real > > > > > deal and a starting point. > > > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > > r8.diff.gz > > > > > > > > Here is the local.conf: > > > > MACHINE ?= "p4080ds" > > > > CONF_VERSION = "1" > > > > ARCHIVER_MODE ?= "patched" > > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > > to fix, would you please provide more info ? (In reply to comment #13) > (In reply to comment #10) > > (In reply to comment #9) > > > > Hello, > > > > Please use only the > > "only recursive symlinks are added to the (5.95 KB, application/gzip) > > 2013-08-06 13:48 UTC," patch, this addresses what I think a fix for the > > symlinks. > > > > Yes, this is what I use, and I wonder what is the function > readlink_abspath() used for ? Seems no one calls it. I think I just forget it there... ignore it > > > The idea behind the algorithm there is that I am excluding the recursive > > symlinks by crating a list of folders and another one of symlinks. Those > > that are recursive are not added to the list, hence the issue will not occur. > > > > Also, please run full builds, not for 'udev' only, the issue occurs for any > > package (usually system related, like udev is) that has recursive symlinks. > > > > Yes, I've also tried the full builds several times, but didn't find the > error, > would you please show me a recursive symlink for the package? If you add debug to the python script area I changed you will notice several recursive symlinks. At least this is what we've seen a month ago. BR, Valentin > > > If you consider that there is a better solution to this issue, please apply > > it, my attached patches were just proposed solutions. > > > > I'm glad to fix it if I can understand it correctly. > > // Robert > > > Thank you, > > Valentin > > > > > > > (In reply to comment #8) > > > > (In reply to comment #7) > > > > Hi, > > > > > > > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > > > > > > 1) For your second patch: > > > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > > > - What does this comment mean, please ? > > > + # get symlinks and folders in two separated lists > > > + # using only 'root' we will have here only the symlinks pointing to > > > directories > > > > > > > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > > > can detect this on my host: > > > > > > $ diff -Nur dir1/ dir2/ > > > diff: dir1/aa/ln3: recursive directory loop > > > > > > Suppose that there are recursive symlinks in dir1. > > > > > > Maybe there are other problems ? I've tried to build the udev many times, > > > didn't see the error, I think that if it is caused by recursive symlink, we > > > can find it every time. > > > > > > Any more comments, please ? > > > > > > // Robert > > > > > > > The issue occurred randomly, not on a specific package. > > > > We consider that there should be several test runs (several full builds with > > > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > > > in order to reproduce this issue which makes build not completing. > > > > > > > > My patches addressed the recursive symlinks (the last one is directly for > > > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > > > each of them from the list. > > > > > > > > Best regards, > > > > Valentin > > > > > > > > > > > > > (In reply to comment #6) > > > > > > There are only the proposed solutions (both attached), but these are only > > > > > > starting points for the solution I am hopping to see from anyone in charge > > > > > > of the 'archiver.bbclass'. > > > > > > Even with what I made the problem still replicates but each time for another > > > > > > package, this would be why I consider my solutions only a part of the real > > > > > > deal and a starting point. > > > > > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > > > r8.diff.gz > > > > > > > > > > Here is the local.conf: > > > > > MACHINE ?= "p4080ds" > > > > > CONF_VERSION = "1" > > > > > ARCHIVER_MODE ?= "patched" > > > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > > > to fix, would you please provide more info ? (In reply to comment #13) In addition to the previous comment. Below you can see several recursive symlinks. > (In reply to comment #10) > > (In reply to comment #9) > > > > Hello, > > > > Please use only the > > "only recursive symlinks are added to the (5.95 KB, application/gzip) > > 2013-08-06 13:48 UTC," patch, this addresses what I think a fix for the > > symlinks. > > > > Yes, this is what I use, and I wonder what is the function > readlink_abspath() used for ? Seems no one calls it. > > > The idea behind the algorithm there is that I am excluding the recursive > > symlinks by crating a list of folders and another one of symlinks. Those > > that are recursive are not added to the list, hence the issue will not occur. > > > > Also, please run full builds, not for 'udev' only, the issue occurs for any > > package (usually system related, like udev is) that has recursive symlinks. > > > > Yes, I've also tried the full builds several times, but didn't find the > error, > would you please show me a recursive symlink for the package? diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem: recursive directory loop diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/11:0/subsystem: recursive directory loop diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/7:0: recursive directory loop diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/7:1/subsystem: recursive directory loop diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/7:2/subsystem: recursive directory loop diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/7:3/subsystem: recursive directory loop diff: poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/7:4/subsystem: recursive directory loop > > > If you consider that there is a better solution to this issue, please apply > > it, my attached patches were just proposed solutions. > > > > I'm glad to fix it if I can understand it correctly. > > // Robert > > > Thank you, > > Valentin > > > > > > > (In reply to comment #8) > > > > (In reply to comment #7) > > > > Hi, > > > > > > > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > > > > > > 1) For your second patch: > > > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > > > - What does this comment mean, please ? > > > + # get symlinks and folders in two separated lists > > > + # using only 'root' we will have here only the symlinks pointing to > > > directories > > > > > > > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > > > can detect this on my host: > > > > > > $ diff -Nur dir1/ dir2/ > > > diff: dir1/aa/ln3: recursive directory loop > > > > > > Suppose that there are recursive symlinks in dir1. > > > > > > Maybe there are other problems ? I've tried to build the udev many times, > > > didn't see the error, I think that if it is caused by recursive symlink, we > > > can find it every time. > > > > > > Any more comments, please ? > > > > > > // Robert > > > > > > > The issue occurred randomly, not on a specific package. > > > > We consider that there should be several test runs (several full builds with > > > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > > > in order to reproduce this issue which makes build not completing. > > > > > > > > My patches addressed the recursive symlinks (the last one is directly for > > > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > > > each of them from the list. > > > > > > > > Best regards, > > > > Valentin > > > > > > > > > > > > > (In reply to comment #6) > > > > > > There are only the proposed solutions (both attached), but these are only > > > > > > starting points for the solution I am hopping to see from anyone in charge > > > > > > of the 'archiver.bbclass'. > > > > > > Even with what I made the problem still replicates but each time for another > > > > > > package, this would be why I consider my solutions only a part of the real > > > > > > deal and a starting point. > > > > > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > > > r8.diff.gz > > > > > > > > > > Here is the local.conf: > > > > > MACHINE ?= "p4080ds" > > > > > CONF_VERSION = "1" > > > > > ARCHIVER_MODE ?= "patched" > > > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > > > to fix, would you please provide more info ? (In reply to comment #15) > (In reply to comment #13) > > In addition to the previous comment. > Below you can see several recursive symlinks. > > > (In reply to comment #10) > > > (In reply to comment #9) > > > > > > Hello, > > > > > > Please use only the > > > "only recursive symlinks are added to the (5.95 KB, application/gzip) > > > 2013-08-06 13:48 UTC," patch, this addresses what I think a fix for the > > > symlinks. > > > > > > > Yes, this is what I use, and I wonder what is the function > > readlink_abspath() used for ? Seems no one calls it. > > > > > The idea behind the algorithm there is that I am excluding the recursive > > > symlinks by crating a list of folders and another one of symlinks. Those > > > that are recursive are not added to the list, hence the issue will not occur. > > > > > > Also, please run full builds, not for 'udev' only, the issue occurs for any > > > package (usually system related, like udev is) that has recursive symlinks. > > > > > > > Yes, I've also tried the full builds several times, but didn't find the > > error, > > would you please show me a recursive symlink for the package? > > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/0:16/subsystem: recursive directory loop > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/11:0/subsystem: recursive directory loop > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/7:0: recursive directory loop > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/7:1/subsystem: recursive directory loop > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/7:2/subsystem: recursive directory loop > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/7:3/subsystem: recursive directory loop > diff: > poky/build_p4080ds/tmp/work/ppce500mc-enea-linux/udev/182-r8/udev-182/test/ > sys/block/loop0/bdi/subsystem/7:4/subsystem: recursive directory loop > Seems that your udev is different from mine, mine is -r7, yours is -r8, I will try your patch and send a path to the mailing list later. Thanks // Robert > > > > > If you consider that there is a better solution to this issue, please apply > > > it, my attached patches were just proposed solutions. > > > > > > > I'm glad to fix it if I can understand it correctly. > > > > // Robert > > > > > Thank you, > > > Valentin > > > > > > > > > > (In reply to comment #8) > > > > > (In reply to comment #7) > > > > > Hi, > > > > > > > > > > > > > Hi Valentin, thanks for your reply, but I'm sorry I still quite understand: > > > > > > > > 1) For your second patch: > > > > - What is the readlink_abspath(l) used for ? Seems that no one calls it. > > > > - What does this comment mean, please ? > > > > + # get symlinks and folders in two separated lists > > > > + # using only 'root' we will have here only the symlinks pointing to > > > > directories > > > > > > > > > > > > 2) I think that you are trying to recursive symlinks issue ? But the diff > > > > can detect this on my host: > > > > > > > > $ diff -Nur dir1/ dir2/ > > > > diff: dir1/aa/ln3: recursive directory loop > > > > > > > > Suppose that there are recursive symlinks in dir1. > > > > > > > > Maybe there are other problems ? I've tried to build the udev many times, > > > > didn't see the error, I think that if it is caused by recursive symlink, we > > > > can find it every time. > > > > > > > > Any more comments, please ? > > > > > > > > // Robert > > > > > > > > > The issue occurred randomly, not on a specific package. > > > > > We consider that there should be several test runs (several full builds with > > > > > the archiver mode enabled for "patched" and "tar" p- as you already tried) > > > > > in order to reproduce this issue which makes build not completing. > > > > > > > > > > My patches addressed the recursive symlinks (the last one is directly for > > > > > this, from 2013-08-06 13:48 UTC) which are generating the issue, by removing > > > > > each of them from the list. > > > > > > > > > > Best regards, > > > > > Valentin > > > > > > > > > > > > > > > > (In reply to comment #6) > > > > > > > There are only the proposed solutions (both attached), but these are only > > > > > > > starting points for the solution I am hopping to see from anyone in charge > > > > > > > of the 'archiver.bbclass'. > > > > > > > Even with what I made the problem still replicates but each time for another > > > > > > > package, this would be why I consider my solutions only a part of the real > > > > > > > deal and a starting point. > > > > > > > > > > > > Hi Valentin Cobelea, I can't reproduce the error, and I check the > > > > > > udev-182-r8.diff.gz it is OK: > > > > > > > > > > > > > > > > > > -rw-rw-r-- 2 robert robert 1987507 Aug 30 22:53 > > > > > > tmp/deploy/sources/powerpc-poky-linux/GPLv2.0+LGPLv2.1+/udev-182-r8/udev-182- > > > > > > r8.diff.gz > > > > > > > > > > > > Here is the local.conf: > > > > > > MACHINE ?= "p4080ds" > > > > > > CONF_VERSION = "1" > > > > > > ARCHIVER_MODE ?= "patched" > > > > > > ARCHIVER_MODE[type] ?= "tar" > > > > > > > > > > > > I took a look at your patches, I'm sorry that I didn't know what did it try > > > > > > to fix, would you please provide more info ? (In reply to comment #16) Thank you very much. Valentin (In reply to comment #17) > (In reply to comment #16) > Thank you very much. > > Valentin Hi Valentin, I have added your signed-off and sent oe-core mailing list: http://lists.openembedded.org/pipermail/openembedded-core/2013-September/083759.html Thanks for the contribution:-) Created attachment 1456 [details]
exclude the recursive symlink
(In reply to comment #18) > (In reply to comment #17) > > (In reply to comment #16) > > Thank you very much. > > > > Valentin > > Hi Valentin, I have added your signed-off and sent oe-core mailing list: > > http://lists.openembedded.org/pipermail/openembedded-core/2013-September/ > 083759.html > > Thanks for the contribution:-) Hi Valentin, I've added a new patch, it would be great it you can help me to test it. More problems have been found, see also 5113 Hi, I did seen a random behavior in what it comes to that specific area of code. My opinion is that bug 5113 is the main cause of all these issues regarding the archiver.bbclass. I think that should be addressed first since has indeed no sense in what it does. BR, Valentin (In reply to comment #20) Yes, I can help you with testing the patch(s), let me know when you have a "final version" - that address to all things - for it so that I can start some builds. BR, Valentin > (In reply to comment #18) > > (In reply to comment #17) > > > (In reply to comment #16) > > > Thank you very much. > > > > > > Valentin > > > > Hi Valentin, I have added your signed-off and sent oe-core mailing list: > > > > http://lists.openembedded.org/pipermail/openembedded-core/2013-September/ > > 083759.html > > > > Thanks for the contribution:-) > > Hi Valentin, > > I've added a new patch, it would be great it you can help me to test it. (In reply to comment #23) > (In reply to comment #20) > > Yes, I can help you with testing the patch(s), let me know when you have a > "final version" - that address to all things - for it so that I can start > some builds. > > BR, > Valentin > > > (In reply to comment #18) > > > (In reply to comment #17) > > > > (In reply to comment #16) > > > > Thank you very much. > > > > > > > > Valentin > > > > > > Hi Valentin, I have added your signed-off and sent oe-core mailing list: > > > > > > http://lists.openembedded.org/pipermail/openembedded-core/2013-September/ > > > 083759.html > > > > > > Thanks for the contribution:-) > > > > Hi Valentin, > > > > I've added a new patch, it would be great it you can help me to test it. Hi Valentin, After more investigation, many other problems are found, so I'd like to refactor the 4 classes in 1.6, please see 5113, move it to 1.6. (In reply to comment #24) > (In reply to comment #23) > > (In reply to comment #20) > > > > Yes, I can help you with testing the patch(s), let me know when you have a > > "final version" - that address to all things - for it so that I can start > > some builds. > > > > BR, > > Valentin > > > > > (In reply to comment #18) > > > > (In reply to comment #17) > > > > > (In reply to comment #16) > > > > > Thank you very much. > > > > > > > > > > Valentin > > > > > > > > Hi Valentin, I have added your signed-off and sent oe-core mailing list: > > > > > > > > http://lists.openembedded.org/pipermail/openembedded-core/2013-September/ > > > > 083759.html > > > > > > > > Thanks for the contribution:-) > > > > > > Hi Valentin, > > > > > > I've added a new patch, it would be great it you can help me to test it. > > Hi Valentin, > > After more investigation, many other problems are found, so I'd like to > refactor the 4 classes in 1.6, please see 5113, move it to 1.6. Hi, Thank you for all the investigation, it makes clear now why the issue was occurring on a random basis. I am fine with the postpone of the class refactor to v1.6. Best regards, Valentin |