Bug 10621

Summary: make bmaptool a recommended tool for flashing images
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Ed Bartosh <eduard.bartosh>
Component: deploymentAssignee: Ed Bartosh <eduard.bartosh>
Status: VERIFIED FIXED QA Contact: oscar lopez arandas <oscarx.lopez.arandas>
Severity: enhancement    
Priority: Medium+ CC: bluelightning, jose.perez.carranza, srifenbark
Version: unspecified   
Target Milestone: 2.3 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: 05 Jan 2017: IN PROGRESS REVIEW
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)

Description Ed Bartosh 2016-11-08 10:36:17 UTC
bmaptool is integrated into oe-core image building system, but not many people are aware of what this tool is and how to use it.

We need to make changes that would simplify it's usage:
- generate .bmap files by default for oe images
- make it easier to use than currently
- update Yocto documentation and README.hardware providing usage info and explaining its benefits
Comment 1 Ed Bartosh 2016-12-07 13:50:51 UTC
Enabled generation of wic.bmap files for genericx86 and genericx86-64 machines: https://lists.yoctoproject.org/pipermail/poky/2016-December/010824.html
Comment 2 Ed Bartosh 2016-12-09 17:44:34 UTC
Enabled generation of wic.bmap files for Beaglebone: https://lists.yoctoproject.org/pipermail/poky/2016-December/010830.html
Comment 3 Ed Bartosh 2016-12-09 17:48:17 UTC
Enabled generation of wic.bmap for Edgerouter: http://lists.openembedded.org/pipermail/openembedded-core/2016-December/129984.html
Comment 4 Ed Bartosh 2016-12-12 12:44:11 UTC
Scott, I need your help in documenting bmaptool and suggesting it as a recommended way of flashing images.

My idea so far is to add new section "2.4 Flashing images" to Development manual and describe how to use bmaptool there.

Another place where it may make sense to mention bmaptool is "5.9.6.1. Generate an Image using an Existing Kickstart File'. Now only dd is mentioned there. Bmaptool should be mentioned first.

I'd appreciate your suggestions on how to better put this in the documentation.

Here is a brief example of bmaptool usage for wic images (note that bmaptool can be used for other image types too):
 
1.       Put IMAGE_FSTYPES += "wic wic.bmap" into your local.conf
2.       Build core-image-minimal or any other image: bitbake <image>
3.       Flash image using bmap-tool to the media:
-          Easy way(if you have write access): oe-run-native bmap-tool copy ./tmp/deploy/images/qemux86-64/core-image-minimal-<machine>.wic /dev/sdX
-          A bit more complex way(if you don’t have write access):
o    sudo bash
o    PATH=tmp/sysroots/x86_64-linux/usr/bin/ bmaptool copy ./tmp/deploy/images/qemux86-64/core-image-minimal-<machine>.wic /dev/sdX
-          If you’re lucky user of Ubuntu 16.10 or Debian you can install bmaptool with apt-get install bmap-tools and use it without specifying PATH even from root account.

You can see comparison of flashing times for dd and bmaptool here:
http://lists.openembedded.org/pipermail/openembedded-core/2016-May/120834.html
 
And a feedback from happy bmaptool user:
http://lists.openembedded.org/pipermail/openembedded-core/2016-May/120934.html


I'd be happy to answer any questions regarding this.
Comment 5 Scott Rifenbark 2017-01-03 22:18:09 UTC
Hi Ed, 

See http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#generate-an-image-using-a-provided-kickstart-file for a small additional example on flashing an image to some media.

Also, see http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#flashing-images-using-bmap-tool for the new section on flashing an image using the bmap-tool.

One thing I am not sure of is the actual command... is it bmap-tool or bmaptool?

Anyway... give the changes a look and let me know how we can improve or correct them. 

Thanks, 
Scott

PS - I set the bug to IN PROGRESS REVIEW.  I am assuming it is just down to the docs now?  If not, then please reset the status appropriately.
Comment 6 Scott Rifenbark 2017-01-04 00:38:49 UTC
Ed, 

Applied your 'real-time' edits over chat... Here are the links: 

http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#generate-an-image-using-a-provided-kickstart-file 

http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#flashing-images-using-bmap-tool

Since this is your bug Ed... you can mark it as RESOLVED when you want to  --- or if you find something else.. note it here. 

Thanks, 
Scott
Comment 7 Ed Bartosh 2017-01-04 20:55:56 UTC
(In reply to comment #6)
Scott, thank you for documenting this!

Couple of suggestions:

1. I'd change this to show that bmaptools is just one of two possible ways of flashing image to USB stick:

Continuing with the example, you can now write the image to a USB stick, or whatever media for which you built your image, and boot the resulting media.
This can be done using either bmaptool or dd:  

     $ oe-run-native bmaptool copy /var/tmp/wic/build/mkefidisk-201310230946-sda.direct /dev/sdX
                        
Note
For more information on how to use the bmaptool to flash an image, see the "Flashing Images Using bmaptool" section.

     $ sudo dd if=/var/tmp/wic/build/mkefidisk-201310230946-sda.direct of=/dev/sdX


Note: I changed /dev/sdb to /dev/sdX in dd command line and image path in bmfptool command line.

2. It'd be better to change 'target device' to 'media' to avoid confusing USB media with a target embedded device:

"If you have write access to the target device, use this command form:" -> "If you have write access to the media, use this command form:"
Comment 8 Scott Rifenbark 2017-01-05 17:28:14 UTC
Ed, 

thanks for the additional comments:

http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#generate-an-image-using-a-provided-kickstart-file

http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#flashing-images-using-bmaptool

If these are acceptable, you can move the bug into RESOLVED and turn the doc flag to "done."  Or, you just let me know and I will do that. 

Thanks, 
Scott
Comment 9 Ed Bartosh 2017-01-09 11:02:04 UTC
Scott,

Thank you for updating the docs.

I still need to get couple of patches merged before I can close the bug. Most of them are already under test. I'll try to push maintainers to merge them fast.
Comment 10 Ed Bartosh 2017-01-17 09:54:07 UTC
All patches have been accepted.
Comment 11 oscar lopez arandas 2017-08-10 22:02:05 UTC
using same steps from http://www.yoctoproject.org/docs/2.3/dev-manual/dev-manual.html#flashing-images-using-bmaptool

If you are using Ubuntu or Debian distributions, you can install the bmap-tools package using the following command and then use the tool without specifying PATH even from the root account:

     $ sudo apt-get install bmap-tools

BUILD: 2.4 M2: rc3-9ed748a542b520c1cb763d981969233c0f5efd4e
ENVIRONMENT:ubuntu 16.04 lts
STEPS:                            
To use bmaptool to burn .wic
1. first download the two .wic images and the .wic.map in the same folder
2. make sure the usb is dismounting / dev / sdb
3.querem using bmaptool, with the following command.
 $ sudo bmaptool copy ./Downloads/nuc/core-image-sato-sdk-genericx86-64.wic / dev / sdb
4. use the command to make sure that all information is synchronized
 ~ $ sync

============================================================================
oscar@oscar:~$ umount /media/oscar/9415-B91F 
oscar@oscar:~$ sudo bmaptool copy ./Downloads/nuc/core-image-sato-sdk-genericx86-64.wic /dev/sdb
bmaptool: info: discovered bmap file './Downloads/nuc/core-image-sato-sdk-genericx86-64.wic.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 1115443 blocks of size 4096 (4.3 GiB), mapped 633712 blocks (2.4 GiB or 56.8%)
bmaptool: info: copying image 'core-image-sato-sdk-genericx86-64.wic' to block device '/dev/sdb' using bmap file 'core-image-sato-sdk-genericx86-64.wic.bmap'
bmaptool: info: 100% copied
bmaptool: info: synchronizing '/dev/sdb'
bmaptool: info: copying time: 4m 4.5s, copying speed 10.1 MiB/sec
oscar@oscar:~$ sync
================================================================================