Bug 14282

Summary: Allow % and @ in working dir path
Product: [Build System, Metadata & Runtime] BitBake Reporter: steve broshar <sbroshar>
Component: bitbakeAssignee: Richard Purdie <richard.purdie>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: poky.bs.watcher, poky.watcher, randy.macleod, yf3yu
Version: unspecified   
Target Milestone: 3.4 M1   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description steve broshar 2021-03-07 14:40:22 UTC 
Yocto build fails when either @ or % are in the working directory path. This is problematic since Jenkins uses these chars in certain scenarios. Jenkins provides a way to use something other than @, but so far have not found a way to ask it to not use %. Thing is, a Unix path can contain any character; no character is illegal. Therefore, it seems reasonable that yocto should support most if not all chars too -- especially @ and % which are not super common, but not super weird (like nonprintable chars).

FYI, yocto fails on @ from a message from the sanity checker saying @ is not allowed. Although frustrating at least it tells me what it doesn't like.

For %, yocto is less helpful. There is no warning or error message about %. It fails with a rather unrelated error "TypeError: float argument required, not str". My guess is that yocto treat the % as string formatting tag or something. So, there's two ways to look at this: 1) should warn/error about % or 2) should allow %

Bug https://bugzilla.yoctoproject.org/show_bug.cgi?id=4772 is about @. Looks like it prompted a change to sanity checker to check for @. But, would be preferable to simply allow @. Then again, I have a workaround for this char.

My blocking problem is %. I don't have a way to have Jenkins not use % ... without changing our release process which clearly is not desirable -- as that is the tail wagging the dog. Tools should allow me to implement processes; not drive processes.
Comment 1 Randy MacLeod 2021-03-11 16:05:27 UTC 
We won't support this but we can get the sanity checker to issue and error for both @ and %. 

Several people in the YP community use Jenkins so ask on the Yocto list to see how they work around this issue if you like.
Comment 2 Yi Fan Yu 2021-03-11 16:07:46 UTC 
2 fixes: 1) Sanity check for '%'
2) figure out why is it trying to str format the path
Comment 3 steve broshar 2021-03-11 16:41:15 UTC 
When I asked Jenkins about the @ issue, they told me to talk to yocto :)

https://issues.jenkins.io/browse/JENKINS-65034

And you say I should talk to Jenkins about these issues. So easy to point finger. ... better would be responsiveness to user needs.

But, as is, I have to work around the deficiencies of both tools.
Comment 4 Richard Purdie 2021-03-11 21:23:41 UTC 
The big picture here is that Yocto Project builds thousands of pieces of open source code, each have their own environments. Some are standardised and some less so. In order to support this we'd need to assert that all of those pieces of software accept these characters without issue. There are some common systems, sure, there are also plenty that are unique. Adding this support would be a pretty huge undertaking and the question is how valuable is that work compared to say reproducibility or license compliance tools improvements.

In an ideal world we'd love to add support for full utf-8 paths. In the real world, we have some compromises.

I once set out to support spaces due to the prevalence on certain systems. I gave up when I found most scrips and autotools didn't work with it. I suspect "@" is in a similar position. For "%", we certainly shouldn't have tracebacks and need to fix that. Figuring out where its backtracing from within bitbake would be useful if you are able to do that. If that is the only issue, great but if not, we'd likely need to what we did with "%".

The position changes if you know of anyone with the time to do the work, its a simple question of priorities and costs vs benefits. We don't have a huge team of people waiting to work on this kind of thing, much as I wish otherwise.
Comment 5 Richard Purdie 2021-04-15 12:51:58 UTC 
Patch out for review to detect "%" in paths.