Bug 12568

Summary: application with "clone" can't work in bitbake devshell
Product: [Yocto Project Subprojects] Pseudo Reporter: Yang <yang.y.wang>
Component: pseudoAssignee: Seebs <seebs>
Status: RESOLVED NOTABUG QA Contact:
Severity: normal    
Priority: Medium+ CC: alex.kanavin, stephano, yp.pseudo.watcher, yp.watcher
Version: master   
Target Milestone: 2.5 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know

Description Yang 2018-02-23 02:05:22 UTC
I'm going to Yocto devshell by 
> bitbake zlib -c devshell

In this devshell, pseudo root is used and libpseudo.so is preload by default.

I want to use my compiler (not gcc) to build the zlib in this devshell. However the compiler internally calls function "clone" and this function can't work in the devshell due to libpseudo.so which caused the segmentation fault.

The issue can be reproduced by the following test case:
$cat test.c

#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <dirent.h>
#include <unistd.h>

#define STACK_SIZE 10*1024*1024

int do_smth()
{
    int old_mask;
    old_mask = umask(0077);
    int i;
    for(i = 0; i < 10; i++) {printf("%d\n", i);}

    umask(old_mask);
    _exit(100);
}

int main()
{
    int            pid;
    int          status;

    char* child_stack = (char*)malloc(STACK_SIZE*sizeof(char));

    pid = clone(do_smth, child_stack+STACK_SIZE-1,
                         SIGCHLD, NULL);

    waitpid(pid, &status, 0);
    printf("child status %d\n", status);
}


In devshell:
bash-4.2# gcc -c test.c
bash-4.2# gcc -o test test.o
bash-4.2# ./test
child status 139

The cloned pid is killed and can't run successfully.

I was able to run the test case with PSEUDO_UNLOAD environment. However I'm not sure if this is the right way to run the program which trigger the function "clone". Does that mean I have to set this environment for any program which calls the function "clone", "fork" or related functions?

Thanks.
-Yang
Comment 1 Alexander Kanavin 2018-03-01 17:24:35 UTC
Peter, can you please comment?
Comment 2 Seebs 2018-03-01 17:55:26 UTC
I can't imagine why you would think this would apply to fork(), since we run shells under pseudo just fine, so obviously fork and exec work.

I can't compile the test program as-is. With suitable headers added, I do indeed see a segfault from it. A couple of observations:

1. Don't cast the return from malloc.
2. sizeof(char) is an error. Don't do that. Really, seriously, never do that.
3. You didn't parenthesize STACK_SIZE.
4. You didn't return a value from main.
5. All of that seems to be a side issue, but I note that if you change "-1" to "-16", the segfault goes away. Or just remove the "-1". I think you're hitting an alignment issue here. (But the documentation does seem to suggest that "the top byte" should have been the right answer, so.)

I am not honestly sure why the program works only outside of pseudo, but if you change the arguments, it seems to work. I'm not at all sure why.

(Also, your do_smth() has the wrong signature. I don't think that matters on ia32, but really it should have the correct signature and take a single argument of type "void *".)
Comment 3 Seebs 2018-03-01 20:27:48 UTC
Update: I figured out why this appears to be pseudo-related.

It is, in fact, necessary that the stack pointer be suitably aligned. But, it's only necessary for *some* operations.

The underlying operation failing is in the regex code:

 re_token_t start_token = *token;

If the pointer 'token' is not suitably aligned, this is an error. If the stack was not suitably aligned, the pointer isn't suitably aligned, and we core dump.

Pseudo happens to compile a regular expression during startup, which is why this triggers when pseudo is running, but if you just call regcomp() in do_smth(), it will fail reliably even without pseudo running.

The documentation for clone(2) appears to be misleading at best. So far as I can tell, "child_stack + STACK_SIZE" works. (Also you almost certainly don't need a 10MB stack.)
Comment 4 Alexander Kanavin 2018-03-05 06:44:59 UTC
So, NOTABUG I suppose? Please reopen if you disagree.
Comment 5 Yang 2018-03-05 17:32:24 UTC
Thanks for the detailed analysis!
So it seems if the process in the pseudo environment invokes clone, the stack address which passed to clone function should be 16 bytes aligned. This looks like a bit tricky as this requirement is not caused by the cloned function (do_smth) but specifically caused by the pseudo (due to regex code in pseudo). The process itself is actually have no idea of the alignment request if the cloned function does not require it. 
Since this alignment request is caused by pseudo, can pseudo report a meaningful message in case of the alignment is not satisfied? Otherwise any program which call clone function may potentially trigger the error if the stack is not aligned.
Thanks.
-Yang
Comment 6 Seebs 2018-03-05 17:44:11 UTC
Pseudo is purely coincidental here. Any program which calls regcomp() would hit this, as would many other programs that use many other operations.

The stack should be aligned. If it isn't, some stuff will crash. This will be true whether or not you are using pseudo. This may have revealed the bug sooner, but it was always a bug. (That said, the clone(2) documentation should be better. But we don't control that.)