While executing some git commands this morning, I noticed that it decided all by itself how many threads it could use for delta compression: > $ git push --mirror github > Counting objects: 162905, done. > Delta compression using up to 4 threads. > Compressing objects: 100% (42808/42808), done. > Writing objects: 25% (41080/162905), 9.31 MiB | 102 KiB/s It would be nice if bitbake could do the same, i.e., introspect the system to determine how many cores are available and try to make the best use of them. Then users could put this in local.conf: > BB_NUMBER_THREADS = "${NUM_CORES}" > PARALLEL_MAKE = "${NUM_CORES}" Also, I think bitbake defaults to "1" for both of these if not set? Since that gives an excruciatingly slow user experience, maybe some heuristic could be used to pick a more sensible default value, something along the lines of: 1 if ${NUM_CORES} = 1 2 if ${NUM_CORES} = 2 2 if ${NUM_CORES} = 3 ${NUM_CORES}/2 otherwise (The idea is not to bring the machine to its knees if the user doesn't explicitly specify "${NUM_CORES}" in local.conf.) Just thinking out loud. All I know is: the current defaults for these variables are totally unsuitable for real-world use, and it's tricky for a new user to figure out what the 'right' values should be.
Looking into this a bit more, I see that this info is readily available from python: >>> import multiprocessing >>> multiprocessing.cpu_count() 4 So maybe adding Yet Another Variable isn't necessary. Instead, just set values for BB_NUMBER_THREADS and PARALLEL_MAKE using a python snippet that can be overridden by the user?
I don't think a NUM_CORES variable makes sense since that data is already available to the system as you later mention. I'm also reluctant to start doing magic things when the user making the build should probably make a concious choice about what settings to use. There is already an ehancement request about automatic settings of these variables so making this as a duplicate of that. *** This bug has been marked as a duplicate of bug 2528 ***