9099 2016-02-11 11:51:04 +0000 Create sparse images with wic 2016-05-06 10:22:54 +0000 1 1 7 Build System, Metadata & Runtime OE-Core Scripts and Tools 2.2 x86 Multiple RESOLVED FIXED Medium enhancement 2.2 9413 1 joshuagloe eduard.bartosh alexander.kanevskiy eduard.bartosh kergoth randy.macleod ross.burton --- --- Yes (doc changes required) oldest_to_newest 58873 0 joshuagloe 2016-02-11 11:51:04 +0000 Sparse images use less disk space and can be written more efficiently with a utility like bmap-tools. I wrote a patch for wic[1] and a testcase[2] but the option the sparse conversion symbol isn't available in the version of dd shipped in Debian 7. These changes can't be merged until we no longer support Debian 7, so this bug is to ensure the changes don't get lost. 1. http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114228.html 2. http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114309.html 58881 1 eduard.bartosh 2016-02-11 13:03:31 +0000 Can we use native dd? Generally I'd suggest to use native tools in wic as much as possible. This should make it less dependent of the host system. There was some work done already and adding dd to the list of already used native tools would be good. 58883 2 ross.burton 2016-02-11 13:52:52 +0000 Dropping Debian 7 is something we want to do for master anyway, as it's very old now. 59586 3 joshuagloe 2016-03-01 14:12:14 +0000 Patch was submitted[1][2] and a proposal was made to drop support for Debian 7[3][4], though there hasn't been a clear resolution. 1. http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114228.html 2. http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114309.html 3. http://lists.openembedded.org/pipermail/openembedded-core/2016-February/117387.html 4. https://lists.yoctoproject.org/pipermail/yocto/2016-February/028500.html 60607 4 eduard.bartosh 2016-03-31 15:15:21 +0000 There is interesting piece of work done in ostroproject that may be related to this: https://github.com/ostroproject/meta-ostro/pull/70/files This commit contains the code which we can use instead of dd: https://github.com/ostroproject/meta-ostro/pull/70/commits/ccb4f0ca6b36f365bddaa9367ac1e95e0b8c3b5b 60967 5 joshuagloe 2016-04-08 09:17:25 +0000 After discussion in the triage meeting on 7th April we want to try and get my simple patches that use dd into 2.1 and Ed will work on improving the sparseness support in the 2.2 cycle. To that end I resubmitted the patches to the list: http://lists.openembedded.org/pipermail/openembedded-core/2016-April/120034.html http://lists.openembedded.org/pipermail/openembedded-core/2016-April/120035.html 60993 6 alexander.kanevskiy 2016-04-09 10:39:17 +0000 Replied to list. Proposed patch might be dangerous, as dd would not handle sparse blocks correctly. 61007 7 3111 alexander.kanevskiy 2016-04-11 11:43:37 +0000 Created attachment 3111 bmap-tools recipe bmap-tools recipe adopted from Ostro Project to OE 61008 8 3112 alexander.kanevskiy 2016-04-11 11:44:33 +0000 Created attachment 3112 bmap image conversion type bmap image coversion type. 61128 9 eduard.bartosh 2016-04-13 08:39:05 +0000 I'll take care of this bug as I'll be working on bmap-tools support for wic in 2.2 time frame. 61767 10 eduard.bartosh 2016-04-28 09:48:51 +0000 Used APIs from bmap-tools and ostro projects to make wic images sparse. Agreed with Alexander to add required functionality to the bmaptool that wic can use it. The patchset has been sent for review: http://lists.openembedded.org/pipermail/openembedded-core/2016-April/120735.html 61794 11 eduard.bartosh 2016-04-28 13:21:06 +0000 sent v2: http://lists.openembedded.org/pipermail/openembedded-core/2016-April/120747.html 62013 12 eduard.bartosh 2016-05-06 10:22:54 +0000 The patchset has been merged to master: https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=7669219652bcc10fe957114952861d450f825274 3111 2016-04-11 11:43:37 +0000 2016-04-11 11:43:37 +0000 bmap-tools recipe 0001-bmap-tools-utility-to-create-block-map-files.patch text/plain 2544 alexander.kanevskiy RnJvbSA5YTJmMjhmNjQ0ZmI2ZWZjYjQ3YzZkNDdiMTU2YWY4NWZjY2RlNDdlIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiAiQWxleGFuZGVyIEQuIEthbmV2c2tpeSIgPGthZEBrYWQubmFt ZT4KRGF0ZTogTW9uLCAxMSBBcHIgMjAxNiAxNDozNzo0NyArMDMwMApTdWJqZWN0OiBbUEFUQ0gg MS8yXSBibWFwLXRvb2xzOiB1dGlsaXR5IHRvIGNyZWF0ZSBibG9jayBtYXAgZmlsZXMKClN0YW5k YWxvbmUgdmVyc2lvbiBvZiB0aGUgdG9vbCBwcm92aWRlZCBpbgpkZXBsb3kvdG9vbHMgZGlyZWN0 b3J5IGFuZCBjYW4gYmUgZWFzaWx5IHVzZWQgb24KYW55IExpbnV4IHN5c3RlbSB3aXRoIFB5dGhv biAyLjcrIGluc3RhbGxlZAoKU2lnbmVkLW9mZi1ieTogQWxleGFuZGVyIEQuIEthbmV2c2tpeSA8 a2FkQGthZC5uYW1lPgotLS0KIG1ldGEvcmVjaXBlcy1zdXBwb3J0L2JtYXAtdG9vbHMvYm1hcC10 b29sc18zLjIuYmIgfCA0MSArKysrKysrKysrKysrKysrKysrKysrKwogMSBmaWxlIGNoYW5nZWQs IDQxIGluc2VydGlvbnMoKykKIGNyZWF0ZSBtb2RlIDEwMDY0NCBtZXRhL3JlY2lwZXMtc3VwcG9y dC9ibWFwLXRvb2xzL2JtYXAtdG9vbHNfMy4yLmJiCgpkaWZmIC0tZ2l0IGEvbWV0YS9yZWNpcGVz LXN1cHBvcnQvYm1hcC10b29scy9ibWFwLXRvb2xzXzMuMi5iYiBiL21ldGEvcmVjaXBlcy1zdXBw b3J0L2JtYXAtdG9vbHMvYm1hcC10b29sc18zLjIuYmIKbmV3IGZpbGUgbW9kZSAxMDA2NDQKaW5k ZXggMDAwMDAwMC4uNDY0MDgyMAotLS0gL2Rldi9udWxsCisrKyBiL21ldGEvcmVjaXBlcy1zdXBw b3J0L2JtYXAtdG9vbHMvYm1hcC10b29sc18zLjIuYmIKQEAgLTAsMCArMSw0MSBAQAorU1VNTUFS WSA9ICJUb29scyB0byBnZW5lcmF0ZSBibG9jayBtYXAgKEFLQSBibWFwKSBhbmQgZmxhc2ggaW1h Z2VzIHVzaW5nIGJtYXAiCitERVNDUklQVElPTiA9ICJCbWFwLXRvb2xzIC0gdG9vbHMgdG8gZ2Vu ZXJhdGUgYmxvY2sgbWFwIChBS0EgYm1hcCkgYW5kIGZsYXNoIGltYWdlcyB1c2luZyBcCitibWFw LiBCbWFwdG9vbCBpcyBhIGdlbmVyaWMgdG9vbCBmb3IgY3JlYXRpbmcgdGhlIGJsb2NrIG1hcCAo Ym1hcCkgZm9yIGEgZmlsZSwgXAorYW5kIGNvcHlpbmcgZmlsZXMgdXNpbmcgdGhlIGJsb2NrIG1h cC4gVGhlIGlkZWEgaXMgdGhhdCBsYXJnZSBmaWxlIGNvbnRhaW5pbmcgXAordW51c2VkIGJsb2Nr cywgbGlrZSByYXcgc3lzdGVtIGltYWdlIGZpbGVzLCBjYW4gYmUgY29waWVkIG9yIGZsYXNoZWQg YSBsb3QgXAorZmFzdGVyIHdpdGggYm1hcHRvb2wgdGhhbiB3aXRoIHRyYWRpdGlvbmFsIHRvb2xz IGxpa2UgImRkIiBvciAiY3AiLiIKK0hPTUVQQUdFID0gImh0dHA6Ly9naXQuaW5mcmFkZWFkLm9y Zy91c2Vycy9kZWRla2luZC9ibWFwLXRvb2xzLmdpdCIKK1NFQ1RJT04gPSAiZGV2ZWwvcHl0aG9u IgorTElDRU5TRSA9ICJHUEx2MiIKK0xJQ19GSUxFU19DSEtTVU0gPSAiZmlsZTovL0NPUFlJTkc7 bWQ1PWIyMzRlZTRkNjlmNWZjZTQ0ODZhODBmZGFmNGE0MjYzIgorCitTUkNfVVJJID0gImZ0cDov L2Z0cC5pbmZyYWRlYWQub3JnL3B1Yi9ibWFwLXRvb2xzL2JtYXAtdG9vbHMtJHtQVn0udGd6Igor U1JDX1VSSVttZDVzdW1dID0gIjkyY2RhZDFjYjRkZmEwY2NhNzE3NmM4ZTIyNzUyNjE2IgorU1JD X1VSSVtzaGEyNTZzdW1dID0gImNjNmM3ZjdkYzBhMzdlMmEzMmRlYjEyNzMwOGUyNGU2YzRiODBi ZmI1NGYzODAzYzMwOGVmYWIwMmJmMmQ0MzQiCisKK2luaGVyaXQgc2V0dXB0b29scyBkZXBsb3kK KworUkRFUEVORFNfJHtQTn0gKz0gInB5dGhvbi1jb21wcmVzc2lvbiBweXRob24tY29yZSBweXRo b24tdW5pdHRlc3QiCisKK0JCQ0xBU1NFWFRFTkQgPSAibmF0aXZlc2RrIG5hdGl2ZSIKKworZG9f aW5zdGFsbF9hcHBlbmRfY2xhc3MtbmF0aXZlKCkgeworICAgIHNlZCAtaSAtZSAnc3xeIyEuKi91 c3IvYmluL2VudiBweXRob258IyEgL3Vzci9iaW4vZW52IG5hdGl2ZXB5dGhvbnwnICR7RH0ke2Jp bmRpcn0vYm1hcHRvb2wKK30KKworZG9fZGVwbG95W3NzdGF0ZS1vdXRwdXRkaXJzXSA9ICIke0RF UExPWV9ESVJfVE9PTFN9IgorZG9fZGVwbG95X2NsYXNzLW5hdGl2ZSgpIHsKKyAgICBjcCBibWFw dG9vbCBfX21haW5fXy5weQorICAgIHB5dGhvbiAtbSB6aXBmaWxlIC1jIGJtYXB0b29sLnppcCBi bWFwdG9vbHMgX19tYWluX18ucHkKKyAgICBlY2hvICcjIS91c3IvYmluL2VudiBweXRob24nIHwg Y2F0IC0gYm1hcHRvb2wuemlwID4gYm1hcHRvb2wtc3RhbmRhbG9uZQorICAgIGluc3RhbGwgLWQg JHtERVBMT1lESVJ9CisgICAgaW5zdGFsbCAtbSAwNzU1IGJtYXB0b29sLXN0YW5kYWxvbmUgJHtE RVBMT1lESVJ9L2JtYXB0b29sLSR7UFZ9CisgICAgcm0gLWYgJHtERVBMT1lESVJ9L2JtYXB0b29s CisgICAgbG4gLXNmIC4vYm1hcHRvb2wtJHtQVn0gJHtERVBMT1lESVJ9L2JtYXB0b29sCit9CisK K2RvX2RlcGxveSgpIHsKKyAgICAgICAgOgorfQorCithZGR0YXNrIGRlcGxveSBiZWZvcmUgZG9f cGFja2FnZSBhZnRlciBkb19pbnN0YWxsCi0tIAoyLjguMQoK 3112 2016-04-11 11:44:33 +0000 2016-04-11 11:44:33 +0000 bmap image conversion type 0002-image-types-adding-bmap-generation-option.patch text/plain 2311 alexander.kanevskiy RnJvbSA4NjRkZDVmZmRiMzhiOGRjNmFkNDVkZmMzYWI4N2UzNzBjYWM2MjhkIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiAiQWxleGFuZGVyIEQuIEthbmV2c2tpeSIgPGthZEBrYWQubmFt ZT4KRGF0ZTogTW9uLCAxMSBBcHIgMjAxNiAxNDo0MDoyMyArMDMwMApTdWJqZWN0OiBbUEFUQ0gg Mi8yXSBpbWFnZSB0eXBlczogYWRkaW5nIGJtYXAgZ2VuZXJhdGlvbiBvcHRpb24KCmJtYXAgaW1h Z2UgY29udmVyc2lvbiB0eXBlIGFsbG93cyB0byBjcmVhdGUgdG8gY3JlYXRlIGJsb2NrIG1hcCBm aWxlcwpmb3Igc3BhcnNlIGltYWdlcy4gQm1hcCBmaWxlIGNhbiBiZSB1c2VkIHRvZ2V0aGVyIHdp dGggYm1hcC10b29scwpmb3IgZWZmaWNpZW50bHkgZmxhc2ggaW1hZ2VzIHRvIHJhdyBkZXZpY2Vz IChoZGQgb3IgdXNiIGRyaXZlKQoKU2lnbmVkLW9mZi1ieTogQWxleGFuZGVyIEQuIEthbmV2c2tp eSA8a2FkQGthZC5uYW1lPgotLS0KIG1ldGEvY2xhc3Nlcy9pbWFnZV90eXBlcy5iYmNsYXNzIHwg NCArKystCiAxIGZpbGUgY2hhbmdlZCwgMyBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCgpk aWZmIC0tZ2l0IGEvbWV0YS9jbGFzc2VzL2ltYWdlX3R5cGVzLmJiY2xhc3MgYi9tZXRhL2NsYXNz ZXMvaW1hZ2VfdHlwZXMuYmJjbGFzcwppbmRleCBiY2IyNDk3Li4xMmQzMWUzIDEwMDY0NAotLS0g YS9tZXRhL2NsYXNzZXMvaW1hZ2VfdHlwZXMuYmJjbGFzcworKysgYi9tZXRhL2NsYXNzZXMvaW1h Z2VfdHlwZXMuYmJjbGFzcwpAQCAtMjYwLDcgKzI2MCw3IEBAIElNQUdFX1RZUEVTID0gIiBcCiAg ICAgd2ljIHdpYy5neiB3aWMuYnoyIHdpYy5sem1hIFwKICIKIAotQ09NUFJFU1NJT05UWVBFUyA9 ICJneiBiejIgbHptYSB4eiBsejQgc3VtIG1kNXN1bSBzaGExc3VtIHNoYTIyNHN1bSBzaGEyNTZz dW0gc2hhMzg0c3VtIHNoYTUxMnN1bSIKK0NPTVBSRVNTSU9OVFlQRVMgPSAiZ3ogYnoyIGx6bWEg eHogbHo0IHN1bSBtZDVzdW0gc2hhMXN1bSBzaGEyMjRzdW0gc2hhMjU2c3VtIHNoYTM4NHN1bSBz aGE1MTJzdW0gYm1hcCIKIENPTVBSRVNTX0NNRF9sem1hID0gImx6bWEgLWsgLWYgLTcgJHtJTUFH RV9OQU1FfSR7SU1BR0VfTkFNRV9TVUZGSVh9LiR7dHlwZX0iCiBDT01QUkVTU19DTURfZ3ogPSAi Z3ppcCAtZiAtOSAtYyAke0lNQUdFX05BTUV9JHtJTUFHRV9OQU1FX1NVRkZJWH0uJHt0eXBlfSA+ ICR7SU1BR0VfTkFNRX0ke0lNQUdFX05BTUVfU1VGRklYfS4ke3R5cGV9Lmd6IgogQ09NUFJFU1Nf Q01EX2J6MiA9ICJwYnppcDIgLWYgLWsgJHtJTUFHRV9OQU1FfSR7SU1BR0VfTkFNRV9TVUZGSVh9 LiR7dHlwZX0iCkBAIC0yNzMsMTIgKzI3MywxNCBAQCBDT01QUkVTU19DTURfc2hhMjI0c3VtID0g InNoYTIyNHN1bSAke0lNQUdFX05BTUV9JHtJTUFHRV9OQU1FX1NVRkZJWH0uJHt0eXBlfSA+CiBD T01QUkVTU19DTURfc2hhMjU2c3VtID0gInNoYTI1NnN1bSAke0lNQUdFX05BTUV9JHtJTUFHRV9O QU1FX1NVRkZJWH0uJHt0eXBlfSA+ICR7SU1BR0VfTkFNRX0ke0lNQUdFX05BTUVfU1VGRklYfS4k e3R5cGV9LnNoYTI1NnN1bSIKIENPTVBSRVNTX0NNRF9zaGEzODRzdW0gPSAic2hhMzg0c3VtICR7 SU1BR0VfTkFNRX0ke0lNQUdFX05BTUVfU1VGRklYfS4ke3R5cGV9ID4gJHtJTUFHRV9OQU1FfSR7 SU1BR0VfTkFNRV9TVUZGSVh9LiR7dHlwZX0uc2hhMzg0c3VtIgogQ09NUFJFU1NfQ01EX3NoYTUx MnN1bSA9ICJzaGE1MTJzdW0gJHtJTUFHRV9OQU1FfSR7SU1BR0VfTkFNRV9TVUZGSVh9LiR7dHlw ZX0gPiAke0lNQUdFX05BTUV9JHtJTUFHRV9OQU1FX1NVRkZJWH0uJHt0eXBlfS5zaGE1MTJzdW0i CitDT01QUkVTU19DTURfYm1hcCA9ICJibWFwdG9vbCBjcmVhdGUgJHtJTUFHRV9OQU1FfSR7SU1B R0VfTkFNRV9TVUZGSVh9LiR7dHlwZX0gLW8gJHtJTUFHRV9OQU1FfSR7SU1BR0VfTkFNRV9TVUZG SVh9LiR7dHlwZX0uYm1hcCIKIENPTVBSRVNTX0RFUEVORFNfbHptYSA9ICJ4ei1uYXRpdmUiCiBD T01QUkVTU19ERVBFTkRTX2d6ID0gIiIKIENPTVBSRVNTX0RFUEVORFNfYnoyID0gInBiemlwMi1u YXRpdmUiCiBDT01QUkVTU19ERVBFTkRTX3h6ID0gInh6LW5hdGl2ZSIKIENPTVBSRVNTX0RFUEVO RFNfbHo0ID0gImx6NC1uYXRpdmUiCiBDT01QUkVTU19ERVBFTkRTX3N1bSA9ICJtdGQtdXRpbHMt bmF0aXZlIgorQ09NUFJFU1NfREVQRU5EU19ibWFwID0gImJtYXAtdG9vbHMtbmF0aXZlIgogCiBS VU5OQUJMRV9JTUFHRV9UWVBFUyA/PSAiZXh0MiBleHQzIGV4dDQiCiBSVU5OQUJMRV9NQUNISU5F X1BBVFRFUk5TID89ICJxZW11IgotLSAKMi44LjEKCg==