The output of bitbake --help should be grouped to make it easier to read. For example showing first the options that a normal user would use (-C --runall etc), then debugging options, then internal flags (or even hide those behind --help-internal?)
Would this be a good opportunity to also rewrite the argument handling using argparse instead of optparse?
Yes, we should switch to argparse.
Some work has been done on this, but not yet merged: https://lists.openembedded.org/g/bitbake-devel/topic/94943006
https://git.yoctoproject.org/poky/commit/bitbake/lib/bb/main.py?id=d35d4032025fbecf8dca56d97bf313a6e98564c0 moved to argparse
Using argparse.add_argument_group(). I split in 4 groups: * General options * Server options * Debugging options * Internal options See updated help output below. @Richard: * Does the grouping of each argument seem correct? * Should I sort somehow the arguments in each group? * Is the "Server" group a good idea? $ bitbake --help usage: bitbake [-b BUILDFILE] [-k] [-f] [-c CMD] [-C INVALIDATE_STAMP] [-r PREFILE] [-R POSTFILE] [-v] [-D] [-q] [-n] [-S SIGNATURE_HANDLER] [-p] [-s] [-e] [-g] [-I EXTRA_ASSUME_PROVIDED] [-l DEBUG_DOMAINS] [-P] [-u UI] [--token XMLRPCTOKEN] [--revisions-changed] [--server-only] [-B BIND] [-T SERVER_TIMEOUT] [--no-setscene] [--skip-setscene] [--setscene-only] [--remote-server REMOTE_SERVER] [-m] [--observe-only] [--status-only] [-w WRITEEVENTLOG] [--runall RUNALL] [--runonly RUNONLY] [--version] [-h] [recipename/target ...] It is assumed there is a conf/bblayers.conf available in cwd or in BBPATH which will provide the layer, BBFILES and other configuration information. General options: recipename/target Execute the specified task (default is 'build') for these target recipes (.bb files). -b BUILDFILE, --buildfile BUILDFILE Execute tasks from a specific .bb recipe directly. WARNING: Does not handle any dependencies from other recipes. -k, --continue Continue as much as possible after an error. While the target that failed and anything depending on it cannot be built, as much as possible will be built before stopping. -f, --force Force the specified targets/task to run (invalidating any existing stamp file). -c CMD, --cmd CMD Specify the task to execute. The exact options available depend on the metadata. Some examples might be 'compile' or 'populate_sysroot' or 'listtasks' may give a list of the tasks available. -C INVALIDATE_STAMP, --clear-stamp INVALIDATE_STAMP Invalidate the stamp for the specified task such as 'compile' and then run the default task for the specified target(s). -r PREFILE, --read PREFILE Read the specified file before bitbake.conf. -R POSTFILE, --postread POSTFILE Read the specified file after bitbake.conf. -q, --quiet Output less log message data to the terminal. You can specify this more than once. -n, --dry-run Don't execute, just go through the motions. -u UI, --ui UI The user interface to use (knotty, ncurses, taskexp or teamcity - default knotty). --runall RUNALL Run the specified task for any recipe in the taskgraph of the specified target (even if it wouldn't otherwise have run). --runonly RUNONLY Run only the specified task within the taskgraph of the specified targets (and any task dependencies those tasks may have). --version Show programs version and exit -h, --help Show this help message and exit Server options: --token XMLRPCTOKEN Specify the connection token to be used when connecting to a remote server. -B BIND, --bind BIND The name/address for the bitbake xmlrpc server to bind to. -T SERVER_TIMEOUT, --idle-timeout SERVER_TIMEOUT Set timeout to unload bitbake server due to inactivity, set to -1 means no unload, default: Environment variable BB_SERVER_TIMEOUT. --remote-server REMOTE_SERVER Connect to the specified server. -m, --kill-server Terminate any running bitbake server. --observe-only Connect to a server as an observing-only client. --status-only Check the status of the remote bitbake server. Debugging options: -v, --verbose Enable tracing of shell tasks (with 'set -x'). Also print bb.note(...) messages to stdout (in addition to writing them to ${T}/log.do_<task>). -D, --debug Increase the debug level. You can specify this more than once. -D sets the debug level to 1, where only bb.debug(1, ...) messages are printed to stdout; -DD sets the debug level to 2, where both bb.debug(1, ...) and bb.debug(2, ...) messages are printed; etc. Without -D, no debug messages are printed. Note that -D only affects output to stdout. All debug messages are written to ${T}/log.do_taskname, regardless of the debug level. -s, --show-versions Show current and preferred versions of all recipes. -e, --environment Show the global or per-recipe environment complete with information about where variables were set/changed. -g, --graphviz Save dependency tree information for the specified targets in the dot syntax. -I EXTRA_ASSUME_PROVIDED, --ignore-deps EXTRA_ASSUME_PROVIDED Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing -l DEBUG_DOMAINS, --log-domains DEBUG_DOMAINS Show debug logging for the specified logging domains -P, --profile Profile the command and save reports. Internal options: -S SIGNATURE_HANDLER, --dump-signatures SIGNATURE_HANDLER Dump out the signature construction information, with no task execution. The SIGNATURE_HANDLER parameter is passed to the handler. Two common values are none and printdiff but the handler may define more/less. none means only dump the signature, printdiff means compare the dumped signature with the cached one. -p, --parse-only Quit after parsing the BB recipes. --revisions-changed Set the exit code depending on whether upstream floating revisions have changed or not. --server-only Run bitbake without a UI, only starting a server (cooker) process. --no-setscene Do not run any setscene tasks. sstate will be ignored and everything needed, built. --skip-setscene Skip setscene tasks if they would be executed. Tasks previously restored from sstate will be kept, unlike --no-setscene --setscene-only Only run setscene tasks, don't run any real tasks. -w WRITEEVENTLOG, --write-log WRITEEVENTLOG Writes the event log of the build to a bitbake event json file. Use '' (empty string) to assign the name automatically.
Thanks, that example did get me thinking! I like the server group, I think that makes a lot of sense (--server-only needed to be there too). I had a bit of a think and realised we probably need "execution control" and "task" groups of options, and probably a "configuration" group. The configuration are some of the least used options so can be 'lost' at the end. We do need to try and keep bitbake's "modes of operation" together too (-b, -e, -s, -u, -g and kind of -S). I played around a bit and wondered about something like what I've included below. As for sorting, I think it we shouldn't as a hard requirement as it may be more helpful to have the commonly used options first in each section. General options: recipename/target Execute the specified task (default is 'build') for these target recipes (.bb files). -b BUILDFILE, --buildfile BUILDFILE Execute tasks from a specific .bb recipe directly. WARNING: Does not handle any dependencies from other recipes. -s, --show-versions Show current and preferred versions of all recipes. -e, --environment Show the global or per-recipe environment complete with information about where variables were set/changed. -g, --graphviz Save dependency tree information for the specified targets in the dot syntax. -u UI, --ui UI The user interface to use (knotty, ncurses, taskexp or teamcity - default knotty). --version Show programs version and exit -h, --help Show this help message and exit Task control options: -f, --force Force the specified targets/task to run (invalidating any existing stamp file). -c CMD, --cmd CMD Specify the task to execute. The exact options available depend on the metadata. Some examples might be 'compile' or 'populate_sysroot' or 'listtasks' may give a list of the tasks available. -C INVALIDATE_STAMP, --clear-stamp INVALIDATE_STAMP Invalidate the stamp for the specified task such as 'compile' and then run the default task for the specified target(s). --runall RUNALL Run the specified task for any recipe in the taskgraph of the specified target (even if it wouldn't otherwise have run). --runonly RUNONLY Run only the specified task within the taskgraph of the specified targets (and any task dependencies those tasks may have). --no-setscene Do not run any setscene tasks. sstate will be ignored and everything needed, built. --skip-setscene Skip setscene tasks if they would be executed. Tasks previously restored from sstate will be kept, unlike --no-setscene --setscene-only Only run setscene tasks, don't run any real tasks. Execution control options: -n, --dry-run Don't execute, just go through the motions. -p, --parse-only Quit after parsing the BB recipes. -k, --continue Continue as much as possible after an error. While the target that failed and anything depending on it cannot be built, as much as possible will be built before stopping. -P, --profile Profile the command and save reports. -S SIGNATURE_HANDLER, --dump-signatures SIGNATURE_HANDLER Dump out the signature construction information, with no task execution. The SIGNATURE_HANDLER parameter is passed to the handler. Two common values are none and printdiff but the handler may define more/less. none means only dump the signature, printdiff means compare the dumped signature with the cached one. --revisions-changed Set the exit code depending on whether upstream floating revisions have changed or not. Logging/output control options: -D, --debug Increase the debug level. You can specify this more than once. -D sets the debug level to 1, where only bb.debug(1, ...) messages are printed to stdout; -DD sets the debug level to 2, where both bb.debug(1, ...) and bb.debug(2, ...) messages are printed; etc. Without -D, no debug messages are printed. Note that -D only affects output to stdout. All debug messages are written to ${T}/log.do_taskname, regardless of the debug level. -l DEBUG_DOMAINS, --log-domains DEBUG_DOMAINS Show debug logging for the specified logging domains -v, --verbose Enable tracing of shell tasks (with 'set -x'). Also print bb.note(...) messages to stdout (in addition to writing them to ${T}/log.do_<task>). -q, --quiet Output less log message data to the terminal. You can specify this more than once. -w WRITEEVENTLOG, --write-log WRITEEVENTLOG Writes the event log of the build to a bitbake event json file. Use '' (empty string) to assign the name automatically. Server options: -B BIND, --bind BIND The name/address for the bitbake xmlrpc server to bind to. -T SERVER_TIMEOUT, --idle-timeout SERVER_TIMEOUT Set timeout to unload bitbake server due to inactivity, set to -1 means no unload, default: Environment variable BB_SERVER_TIMEOUT. --remote-server REMOTE_SERVER Connect to the specified server. -m, --kill-server Terminate any running bitbake server. --token XMLRPCTOKEN Specify the connection token to be used when connecting to a remote server. --observe-only Connect to a server as an observing-only client. --status-only Check the status of the remote bitbake server. --server-only Run bitbake without a UI, only starting a server (cooker) process. Configuration options: -r PREFILE, --read PREFILE Read the specified file before bitbake.conf. -R POSTFILE, --postread POSTFILE Read the specified file after bitbake.conf. -I EXTRA_ASSUME_PROVIDED, --ignore-deps EXTRA_ASSUME_PROVIDED Assume these dependencies don't exist and are already provided (equivalent to ASSUME_PROVIDED). Useful to make dependency graphs more appealing
(In reply to comment #6) > Thanks, that example did get me thinking! I suspected that was gonna happen... Hence the discussion on the output instead of the patch ;) > I like the server group, I think > that makes a lot of sense (--server-only needed to be there too). Changed. > I had a bit of a think and realised we probably need "execution control" and > "task" groups of options, and probably a "configuration" group. The > configuration are some of the least used options so can be 'lost' at the > end. We do need to try and keep bitbake's "modes of operation" together too > (-b, -e, -s, -u, -g and kind of -S). > > I played around a bit and wondered about something like what I've included > below. I'll send a patch implementing your grouping. > As for sorting, I think it we shouldn't as a hard requirement as it may be > more helpful to have the commonly used options first in each section. I was thinking of : * Most used options first * other options in alphabetical order ... but the distinction between the two was not clear. And, with groups limited to a handful of options, this does not make a big difference. I'll leave the order as your output example.
It looks much better, at least to me, thanks! We'll need to run it through the automated testing and so on but I think it should be good to go in.
My main suggestion would be to move -b out of General Options as it's almost never the right thing to do, but people keep on using it as they see "bitbake -b [recipename.bb]" and think that's how to call bitbake. Maybe at the bottom of task control options?
(In reply to comment #9) > My main suggestion would be to move -b out of General Options as it's almost > never the right thing to do, but people keep on using it as they see > "bitbake -b [recipename.bb]" and think that's how to call bitbake. > > Maybe at the bottom of task control options? I think this is a good idea. Richard, what do you think about this?
Minus Ross's suggestion, the re-ordering has been merged : https://git.openembedded.org/bitbake/commit/?id=9257c48c3e36daaecb5e15da22d0bed24865f02c
(In reply to comment #9) > My main suggestion would be to move -b out of General Options as it's almost > never the right thing to do, but people keep on using it as they see > "bitbake -b [recipename.bb]" and think that's how to call bitbake. > > Maybe at the bottom of task control options? It isn't really a task control option. Perhaps a separate "Expert options:" section? Or maybe at the end of "Execution control options"?
Minus Ross's suggestion, the re-ordering has been merged : https://git.openembedded.org/bitbake/commit/?id=9257c48c3e36daaecb5e15da22d0bed24865f02c(In reply to comment #12) > (In reply to comment #9) > > My main suggestion would be to move -b out of General Options as it's almost > > never the right thing to do, but people keep on using it as they see > > "bitbake -b [recipename.bb]" and think that's how to call bitbake. > > > > Maybe at the bottom of task control options? > > It isn't really a task control option. Perhaps a separate "Expert options:" > section? Or maybe at the end of "Execution control options"? I like 'the end of "Execution control options"' : * It will look weird to have only one "Expert options" * I often use '-b' to mean "Do not execute anything but this recipe"
Fixed by : * https://git.yoctoproject.org/poky/commit/?id=654380885a167fb11d3918dd737ff02ce8716a71 * https://git.yoctoproject.org/poky/commit/?id=29cfebf4e0312ec3cffcea08a7b7cbfecbc7d4e2