dropbear fails to correctly generate a key during startup, it hangs the system. Initial debug shows that a value passed into the mp_div_3() function is invalid. It should be a reasonable value, but instead is a full 64bit integer. This may be due to the way this number is initially stored. I did not that the mp_int "used" value is different between x86-64 and x32, the xa86-64 value was 9 while the x32 value was 19.
I tried creating x32 qemux86-64 image. And it built booted fine. Then I tried to run this command: dropbearkey -tdss -f private and it takes forever to finish it.
Created attachment 1222 [details] A patch to fix x32 dropbear should use unsigned long long for unsigned 64-bit integer.
I check the HJ's fix with yocto project. And it does fix the issue I was seeing before. Thanks HJ for the quick fix. I will cook a commit for posting to YP mailing list. Nitin
Fixed by this commit: commit 676fd3f394a0c0576b63c77ed2184bf5f42c29b6 Author: Nitin A Kamble <nitin.a.kamble@intel.com> Date: Thu May 23 16:09:20 2013 +0000 dropbear: a fix for hang in dropbearkey, built for x32 This commit fixes runtime hang of 'dropbearkey' utility, built for a x32 target abi system. The hang was observed while generating ssh keys, with this command: dropbearkey -t dss -f private The issue is fixed by changing the code, where 'long' in x86_64 mode is assumed as 64bit quantity. With the x32 abi, the processor is in x86_64 mode, but the 'long' is a 32bit quantity. Hence the fix uses 'long long' instead of 'long' to define/access 64bit data variables. Fixes bug: [YOCTO #4496] (From OE-Core rev: 8f5bc47729edb8cb051d81e9ff1680cb8d2eca25) Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com> Signed-off-by: Saul Wold <sgw@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>