| Summary: | devtool modify of linux kernel with cfg files in subdirectories fails and tries to copy cfg file | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Build System, Metadata & Runtime] OE-Core | Reporter: | Brett Bergquist <bbergquist> | ||||
| Component: | Scripts and Tools | Assignee: | Siva Balasubramanian <sivakumar.bs> | ||||
| Status: | RESOLVED FIXED | QA Contact: | |||||
| Severity: | normal | ||||||
| Priority: | Medium | CC: | bruce.ashfield, ccasciato, randy.macleod, ross.burton, sivakumar.bs | ||||
| Version: | unspecified | ||||||
| Target Milestone: | 6.1 | ||||||
| Hardware: | x86 | ||||||
| OS: | Multiple | ||||||
| Whiteboard: | |||||||
| OS type for building Yocto: | --- | Type of Regression: | --- | ||||
| Verified: | Documentation change: | No (bug/feature does not impact docs) | |||||
| Attachments: |
|
||||||
Maybe we should just copy the absolute file? Hi Brett, Does this occur on the master branch as well? Are you interested in working to fix it, given some advice? Randy for the YP triage group. If by master branch, we are talking about the poky layer, I will try that and report back. I am willing to fix the issue, but I am struggling a little on what the section of code conceptually is trying to do. What I see earlier in the code is that all of the local files (represented by the dictionary 'local_files') have already been resolved to absolute paths. So in local_files we have a key and absolute path value. The code in question is opening the 'scc' file that lists the kernel config files and the references in there could be just a file name. I am assuming that this was trying to do was to take the relative filename and then produce a absolute path to the file relative to the path of the 'scc' file. I have worked around locally by checking to see if the relative filename refrences a file in the 'local_files' and if the value is an absolute path, this is used as the path to the kernel configuration file and copy that file instead of a computed path relative to the directory of the 'scc' file. This is working locally and I can supply a patch for consideration. Brett, You seem to be on the right track. Please send the change you have so far to the list to get some feedback and/or to have it merged as a step along the path. Thanks! Created attachment 4968 [details]
Patch to look for local full patch config files
This is what I have locally. The patch first gets the config file path from the 'local_files' dictionary. If then checks to see if the path for the config file is absolute and if so it uses that path. If not, it does what the code currently does and computes the path relative to the 'scc' file path.
This works but I am not confident that this is the correct direction as I have not spent enough time to completely understand what is trying to be done.
Bruce or someone else may take a look but if not, please send the patch as an RFC to the oe-core list. Changes to workdir on master may have fix this problem since there are no longer copies being made. Please check. Bulk move of 5.1 bugs to 5.2. -- YP bug review (Randy) Moving to 5.3. Brett do you hope to have time to work on this bug? Brett I'm moving this to unassigned so take it back if you find time. If you want to help someone else, I think a simpler reproducer may be needed but I'm skimming this during a meeting so maybe not. The original failure no longer reproduces on master — the code that computed the .cfg path from the .scc file's directory was removed in ce8190c519 ("devtool: Drop oe-local-files and simplify"), confirming Randy's Comment 7.
To lock this in I wrote an oe-selftest regression test (test_devtool_modify_kernel_cfg_subdirs). Verified it FAILS on the pre-fix tree (d9328e3b0b) with the exact reported FileNotFoundError (copying .cfg from the .scc's bsp/ dir instead of cfg/) and PASSES on current master. Sent to the oe-core list:
Suggest resolving once the test merges.
https://patchwork.yoctoproject.org/project/oe-core/patch/20260618091736.2319572-1-sivakumar.bs@gmail.com/
|
This is with the kirkstone Yocto release. The kernel recipe has --- FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:${THISDIR}/${PN}/cfg:${THISDIR}/${PN}/bsp:${THISDIR}/${PN}/patches:" --- and --- SRC_URI:append = " file://cp-config.scc" SRC_URI:append = " file://apl-spi-001.cfg" SRC_URI:append = " file://cp-clock.cfg" --- These cfg files are within the 'cfg' subdirectory. Within 'standard.py' we have in 'modify' --- for key in local_files.copy(): if key.endswith('scc'): sccfile = open(local_files[key], 'r') for l in sccfile: line = l.split() if line and line[0] in ('kconf', 'patch'): cfg = os.path.join(os.path.dirname(local_files[key]), line[-1]) if not cfg in local_files.values(): local_files[line[-1]] = cfg shutil.copy2(cfg, workdir) sccfile.close() --- This opens and reads the 'scc' file and then computes a path to the 'cfg' file based upon the directory of the 'scc' file. In this case, the 'scc' file is in a subdirectory called 'bsp' and the 'cfg' file is in a simply subdirectory called 'cfg'. This then fails to copy the 'cfg' file and reports a file not found error. --- Parsing recipes: 100% |####################################################################################################################################################################| Time: 0:00:23 Parsing of 2832 .bb files complete (0 cached, 2832 parsed). 4341 targets, 95 skipped, 3 masked, 0 errors. INFO: Mapping virtual/kernel to linux-canoga Traceback (most recent call last): File "/home/brett/yocto/layers/poky/scripts/devtool", line 338, in <module> ret = main() File "/home/brett/yocto/layers/poky/scripts/devtool", line 325, in main ret = args.func(args, config, basepath, workspace) File "/home/brett/yocto/layers/poky/scripts/lib/devtool/standard.py", line 852, in modify shutil.copy2(cfg, workdir) File "/usr/lib/python3.8/shutil.py", line 435, in copy2 copyfile(src, dst, follow_symlinks=follow_symlinks) File "/usr/lib/python3.8/shutil.py", line 264, in copyfile with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst: FileNotFoundError: [Errno 2] No such file or directory: '/home/brett/yocto/layers/meta-canoga-bsp/recipes-kernel/linux/linux-canoga/bsp/cp-clock.cfg' brett@brett-Precision-7740:~/yocto/builds/tsx-dev_build (tsx-dev) $ --- The 'local_files' has the complete absolute path to the local 'cfg' file. It would seem to me that if the path to the 'cfg' is absolute and the file exists at that location, we can simply skip trying to copy the 'cfg' this way.