12723 2018-05-01 10:04:46 +0000 mysql requires unicode and char length filtering 2025-11-06 15:40:18 +0000 1 1 7 Build System, Metadata & Runtime Toaster toaster 2.5 x86 Multiple IN PROGRESS IMPLEMENTATION Medium+ normal 6.0 1 david.reyna david.reyna jessica.zhang mingli.yu randy.macleod stanciux.mihail toaster --- --- No (bug/feature does not impact docs) oldest_to_newest 80459 0 david.reyna 2018-05-01 10:04:46 +0000 The MySQL database is more sensitive to content exceptions that Sqlite. Toaster needs additional filtering on problematic data to be correct for both database and not fail for MySql. Observed issues causing fails in MySql: (a) Unicode characters are appearing in layer descriptions ('\\xC2\\x96', "meta-crosswalk") (b) Some layer URL's exceed 200 characters (204 characters, "meta-nexell") (c) Some recipe licenses exceed 200 characters (1451 characters, "linux-firmware") NOTE: the MySQL docs imply a max character length of 256, but 204 characters fail in reality. The solution is: (a) Filter unicode, e.g. li['description'].encode('ascii','replace') (b) Add proactive filter for char fields, e.g. generally enforce char limits, truncate then append a " ..." for extra long fields (c) Promote some Char fields to un-limited Text fields 80460 1 4294 david.reyna 2018-05-01 10:12:18 +0000 Created attachment 4294 "lsupdates.py", patched to support MySQL This patched version of "bitbake/lib/toaster/orm/management/commands/lsupdates.py" allows integration with MySql. 101605 2 randy.macleod 2025-04-10 13:53:17 +0000 David, Can Mingli help with this bug? She has worked on MariaDB and is available to work on YP bugs now so I thought of her. 4294 2018-05-01 10:12:18 +0000 2018-05-01 10:12:18 +0000 "lsupdates.py", patched to support MySQL lsupdates.py application/octet-stream 12955 david.reyna IwojIGV4OnRzPTQ6c3c9NDpzdHM9NDpldAojIC0qLSB0YWItd2lkdGg6IDQ7IGMtYmFzaWMtb2Zm c2V0OiA0OyBpbmRlbnQtdGFicy1tb2RlOiBuaWwgLSotCiMKIyBCaXRCYWtlIFRvYXN0ZXIgSW1w bGVtZW50YXRpb24KIwojIENvcHlyaWdodCAoQykgMjAxNi0yMDE3ICAgSW50ZWwgQ29ycG9yYXRp b24KIwojIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOyB5b3UgY2FuIHJlZGlzdHJpYnV0 ZSBpdCBhbmQvb3IgbW9kaWZ5CiMgaXQgdW5kZXIgdGhlIHRlcm1zIG9mIHRoZSBHTlUgR2VuZXJh bCBQdWJsaWMgTGljZW5zZSB2ZXJzaW9uIDIgYXMKIyBwdWJsaXNoZWQgYnkgdGhlIEZyZWUgU29m dHdhcmUgRm91bmRhdGlvbi4KIwojIFRoaXMgcHJvZ3JhbSBpcyBkaXN0cmlidXRlZCBpbiB0aGUg aG9wZSB0aGF0IGl0IHdpbGwgYmUgdXNlZnVsLAojIGJ1dCBXSVRIT1VUIEFOWSBXQVJSQU5UWTsg d2l0aG91dCBldmVuIHRoZSBpbXBsaWVkIHdhcnJhbnR5IG9mCiMgTUVSQ0hBTlRBQklMSVRZIG9y IEZJVE5FU1MgRk9SIEEgUEFSVElDVUxBUiBQVVJQT1NFLiAgU2VlIHRoZQojIEdOVSBHZW5lcmFs IFB1YmxpYyBMaWNlbnNlIGZvciBtb3JlIGRldGFpbHMuCiMKIyBZb3Ugc2hvdWxkIGhhdmUgcmVj ZWl2ZWQgYSBjb3B5IG9mIHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBhbG9uZwojIHdp dGggdGhpcyBwcm9ncmFtOyBpZiBub3QsIHdyaXRlIHRvIHRoZSBGcmVlIFNvZnR3YXJlIEZvdW5k YXRpb24sIEluYy4sCiMgNTEgRnJhbmtsaW4gU3RyZWV0LCBGaWZ0aCBGbG9vciwgQm9zdG9uLCBN QSAwMjExMC0xMzAxIFVTQS4KCmZyb20gZGphbmdvLmNvcmUubWFuYWdlbWVudC5iYXNlIGltcG9y dCBCYXNlQ29tbWFuZAoKZnJvbSBvcm0ubW9kZWxzIGltcG9ydCBMYXllclNvdXJjZSwgTGF5ZXIs IFJlbGVhc2UsIExheWVyX1ZlcnNpb24KZnJvbSBvcm0ubW9kZWxzIGltcG9ydCBMYXllclZlcnNp b25EZXBlbmRlbmN5LCBNYWNoaW5lLCBSZWNpcGUKZnJvbSBvcm0ubW9kZWxzIGltcG9ydCBEaXN0 cm8KZnJvbSBvcm0ubW9kZWxzIGltcG9ydCBUb2FzdGVyU2V0dGluZwoKaW1wb3J0IG9zCmltcG9y dCBzeXMKCmltcG9ydCBqc29uCmltcG9ydCBsb2dnaW5nCmltcG9ydCB0aHJlYWRpbmcKaW1wb3J0 IHRpbWUKbG9nZ2VyID0gbG9nZ2luZy5nZXRMb2dnZXIoInRvYXN0ZXIiKQoKREVGQVVMVF9MQVlF UklOREVYX1NFUlZFUiA9ICJodHRwOi8vbGF5ZXJzLm9wZW5lbWJlZGRlZC5vcmcvbGF5ZXJpbmRl eC9hcGkvIgoKY2xhc3MgU3Bpbm5lcih0aHJlYWRpbmcuVGhyZWFkKToKICAgICIiIiBBIHNpbXBs ZSBwcm9ncmVzcyBzcGlubmVyIHRvIGluZGljYXRlIGRvd25sb2FkL3BhcnNpbmcgaXMgaGFwcGVu aW5nIiIiCiAgICBkZWYgX19pbml0X18oc2VsZiwgKmFyZ3MsICoqa3dhcmdzKToKICAgICAgICBz dXBlcihTcGlubmVyLCBzZWxmKS5fX2luaXRfXygqYXJncywgKiprd2FyZ3MpCiAgICAgICAgc2Vs Zi5zZXREYWVtb24oVHJ1ZSkKICAgICAgICBzZWxmLnNpZ25hbCA9IFRydWUKCiAgICBkZWYgcnVu KHNlbGYpOgogICAgICAgIG9zLnN5c3RlbSgnc2V0dGVybSAtY3Vyc29yIG9mZicpCiAgICAgICAg d2hpbGUgc2VsZi5zaWduYWw6CiAgICAgICAgICAgIGZvciBjaGFyIGluIFsiLyIsICItIiwgIlxc IiwgInwiXToKICAgICAgICAgICAgICAgIHN5cy5zdGRvdXQud3JpdGUoIlxyIiArIGNoYXIpCiAg ICAgICAgICAgICAgICBzeXMuc3Rkb3V0LmZsdXNoKCkKICAgICAgICAgICAgICAgIHRpbWUuc2xl ZXAoMC4yNSkKICAgICAgICBvcy5zeXN0ZW0oJ3NldHRlcm0gLWN1cnNvciBvbicpCgogICAgZGVm IHN0b3Aoc2VsZik6CiAgICAgICAgc2VsZi5zaWduYWwgPSBGYWxzZQoKCmNsYXNzIENvbW1hbmQo QmFzZUNvbW1hbmQpOgogICAgYXJncyA9ICIiCiAgICBoZWxwID0gIlVwZGF0ZXMgbG9jYWxseSBj YWNoZWQgaW5mb3JtYXRpb24gZnJvbSBhIGxheWVyaW5kZXggc2VydmVyIgoKICAgIGRlZiBtaW5p X3Byb2dyZXNzKHNlbGYsIHdoYXQsIGksIHRvdGFsKToKICAgICAgICBpID0gaSArIDEKICAgICAg ICBwZWMgPSAoZmxvYXQoaSkvZmxvYXQodG90YWwpKSoxMDAKCiAgICAgICAgc3lzLnN0ZG91dC53 cml0ZSgiXHJVcGRhdGluZyAlcyAlZCUlIiAlCiAgICAgICAgICAgICAgICAgICAgICAgICAod2hh dCwKICAgICAgICAgICAgICAgICAgICAgICAgICBwZWMpKQogICAgICAgIHN5cy5zdGRvdXQuZmx1 c2goKQogICAgICAgIGlmIGludChwZWMpIGlzIDEwMDoKICAgICAgICAgICAgc3lzLnN0ZG91dC53 cml0ZSgiXG4iKQogICAgICAgICAgICBzeXMuc3Rkb3V0LmZsdXNoKCkKCiAgICBkZWYgdXBkYXRl KHNlbGYpOgogICAgICAgICIiIgogICAgICAgICAgICBGZXRjaGVzIGxheWVyLCByZWNpcGUgYW5k IG1hY2hpbmUgaW5mb3JtYXRpb24gZnJvbSBhIGxheWVyaW5kZXgKICAgICAgICAgICAgc2VydmVy CiAgICAgICAgIiIiCiAgICAgICAgb3Muc3lzdGVtKCdzZXR0ZXJtIC1jdXJzb3Igb2ZmJykKCiAg ICAgICAgc2VsZi5hcGl1cmwgPSBERUZBVUxUX0xBWUVSSU5ERVhfU0VSVkVSCiAgICAgICAgaWYg VG9hc3RlclNldHRpbmcub2JqZWN0cy5maWx0ZXIobmFtZT0nQ1VTVE9NX0xBWUVSSU5ERVhfU0VS VkVSJykuY291bnQoKSA9PSAxOgogICAgICAgICAgICBzZWxmLmFwaXVybCA9IFRvYXN0ZXJTZXR0 aW5nLm9iamVjdHMuZ2V0KG5hbWUgPSAnQ1VTVE9NX0xBWUVSSU5ERVhfU0VSVkVSJykudmFsdWUK CiAgICAgICAgYXNzZXJ0IHNlbGYuYXBpdXJsIGlzIG5vdCBOb25lCiAgICAgICAgdHJ5OgogICAg ICAgICAgICBmcm9tIHVybGxpYi5yZXF1ZXN0IGltcG9ydCB1cmxvcGVuLCBVUkxFcnJvcgogICAg ICAgICAgICBmcm9tIHVybGxpYi5wYXJzZSBpbXBvcnQgdXJscGFyc2UKICAgICAgICBleGNlcHQg SW1wb3J0RXJyb3I6CiAgICAgICAgICAgIGZyb20gdXJsbGliMiBpbXBvcnQgdXJsb3BlbiwgVVJM RXJyb3IKICAgICAgICAgICAgZnJvbSB1cmxwYXJzZSBpbXBvcnQgdXJscGFyc2UKCiAgICAgICAg cHJveHlfc2V0dGluZ3MgPSBvcy5lbnZpcm9uLmdldCgiaHR0cF9wcm94eSIsIE5vbmUpCgogICAg ICAgIGRlZiBfZ2V0X2pzb25fcmVzcG9uc2UoYXBpdXJsPU5vbmUpOgogICAgICAgICAgICBpZiBO b25lID09IGFwaXVybDoKICAgICAgICAgICAgICAgIGFwaXVybD1zZWxmLmFwaXVybAogICAgICAg ICAgICBodHRwX3Byb2dyZXNzID0gU3Bpbm5lcigpCiAgICAgICAgICAgIGh0dHBfcHJvZ3Jlc3Mu c3RhcnQoKQoKICAgICAgICAgICAgX3BhcnNlZHVybCA9IHVybHBhcnNlKGFwaXVybCkKICAgICAg ICAgICAgcGF0aCA9IF9wYXJzZWR1cmwucGF0aAoKICAgICAgICAgICAgIyBsb2dnZXIuZGVidWco IkZldGNoaW5nICVzIiwgYXBpdXJsKQogICAgICAgICAgICB0cnk6CiAgICAgICAgICAgICAgICBy ZXMgPSB1cmxvcGVuKGFwaXVybCkKICAgICAgICAgICAgZXhjZXB0IFVSTEVycm9yIGFzIGU6CiAg ICAgICAgICAgICAgICByYWlzZSBFeGNlcHRpb24oIkZhaWxlZCB0byByZWFkICVzOiAlcyIgJSAo cGF0aCwgZS5yZWFzb24pKQoKICAgICAgICAgICAgcGFyc2VkID0ganNvbi5sb2FkcyhyZXMucmVh ZCgpLmRlY29kZSgndXRmLTgnKSkKCiAgICAgICAgICAgIGh0dHBfcHJvZ3Jlc3Muc3RvcCgpCiAg ICAgICAgICAgIHJldHVybiBwYXJzZWQKCiAgICAgICAgIyB2ZXJpZnkgd2UgY2FuIGdldCB0aGUg YmFzaWMgYXBpCiAgICAgICAgdHJ5OgogICAgICAgICAgICBhcGlsaW5rcyA9IF9nZXRfanNvbl9y ZXNwb25zZSgpCiAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbiBhcyBlOgogICAgICAgICAgICBpbXBv cnQgdHJhY2ViYWNrCiAgICAgICAgICAgIGlmIHByb3h5X3NldHRpbmdzIGlzIG5vdCBOb25lOgog ICAgICAgICAgICAgICAgbG9nZ2VyLmluZm8oIkVFOiBVc2luZyBwcm94eSAlcyIgJSBwcm94eV9z ZXR0aW5ncykKICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmcoIkVFOiBjb3VsZCBub3QgY29ubmVj dCB0byAlcywgc2tpcHBpbmcgdXBkYXRlOiIKICAgICAgICAgICAgICAgICAgICAgICAgICAgIiVz XG4lcyIgJSAoc2VsZi5hcGl1cmwsIGUsIHRyYWNlYmFjay5mb3JtYXRfZXhjKCkpKQogICAgICAg ICAgICByZXR1cm4KCiAgICAgICAgIyB1cGRhdGUgYnJhbmNoZXM7IG9ubHkgdGhvc2UgdGhhdCB3 ZSBhbHJlYWR5IGhhdmUgbmFtZXMgbGlzdGVkIGluIHRoZQogICAgICAgICMgUmVsZWFzZXMgdGFi bGUKICAgICAgICB3aGl0ZWxpc3RfYnJhbmNoX25hbWVzID0gW3JlbC5icmFuY2hfbmFtZQogICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZm9yIHJlbCBpbiBSZWxlYXNlLm9iamVjdHMu YWxsKCldCiAgICAgICAgaWYgbGVuKHdoaXRlbGlzdF9icmFuY2hfbmFtZXMpID09IDA6CiAgICAg ICAgICAgIHJhaXNlIEV4Y2VwdGlvbigiRmFpbGVkIHRvIG1ha2UgbGlzdCBvZiBicmFuY2hlcyB0 byBmZXRjaCIpCgogICAgICAgIGxvZ2dlci5pbmZvKCJGZXRjaGluZyBtZXRhZGF0YSByZWxlYXNl cyBmb3IgJXMiLAogICAgICAgICAgICAgICAgICAgICIgIi5qb2luKHdoaXRlbGlzdF9icmFuY2hf bmFtZXMpKQoKICAgICAgICBicmFuY2hlc19pbmZvID0gX2dldF9qc29uX3Jlc3BvbnNlKGFwaWxp bmtzWydicmFuY2hlcyddICsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICI/ZmlsdGVyPW5hbWU6JXMiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAlICJPUiIuam9pbih3aGl0ZWxpc3RfYnJhbmNoX25hbWVzKSkKCiAgICAgICAgIyBN YXAgdGhlIGxheWVyIGluZGV4IGJyYW5jaGVzIHRvIHRvYXN0ZXIgcmVsZWFzZXMKICAgICAgICBs aV9icmFuY2hfaWRfdG9fdG9hc3Rlcl9yZWxlYXNlID0ge30KCiAgICAgICAgdG90YWwgPSBsZW4o YnJhbmNoZXNfaW5mbykKICAgICAgICBmb3IgaSwgYnJhbmNoIGluIGVudW1lcmF0ZShicmFuY2hl c19pbmZvKToKICAgICAgICAgICAgbGlfYnJhbmNoX2lkX3RvX3RvYXN0ZXJfcmVsZWFzZVticmFu Y2hbJ2lkJ11dID0gXAogICAgICAgICAgICAgICAgICAgIFJlbGVhc2Uub2JqZWN0cy5nZXQobmFt ZT1icmFuY2hbJ25hbWUnXSkKICAgICAgICAgICAgc2VsZi5taW5pX3Byb2dyZXNzKCJSZWxlYXNl cyIsIGksIHRvdGFsKQoKICAgICAgICAjIGtlZXAgYSB0cmFjayBvZiB0aGUgbGF5ZXJpbmRleCAo bGkpIGlkIG1hcHBpbmdzIHNvIHRoYXQKICAgICAgICAjIGxheWVyX3ZlcnNpb25zIGNhbiBiZSBj cmVhdGVkIGZvciB0aGVzZSBsYXllcnMgbGF0ZXIgb24KICAgICAgICBsaV9sYXllcl9pZF90b190 b2FzdGVyX2xheWVyX2lkID0ge30KCiAgICAgICAgbG9nZ2VyLmluZm8oIkZldGNoaW5nIGxheWVy cyIpCgogICAgICAgIGxheWVyc19pbmZvID0gX2dldF9qc29uX3Jlc3BvbnNlKGFwaWxpbmtzWyds YXllckl0ZW1zJ10pCgogICAgICAgIHRvdGFsID0gbGVuKGxheWVyc19pbmZvKQogICAgICAgIGZv ciBpLCBsaSBpbiBlbnVtZXJhdGUobGF5ZXJzX2luZm8pOgogICAgICAgICAgICB0cnk6CiAgICAg ICAgICAgICAgICBsLCBjcmVhdGVkID0gTGF5ZXIub2JqZWN0cy5nZXRfb3JfY3JlYXRlKG5hbWU9 bGlbJ25hbWUnXSkKICAgICAgICAgICAgICAgIGwudXBfZGF0ZSA9IGxpWyd1cGRhdGVkJ10KICAg ICAgICAgICAgICAgIGwuc3VtbWFyeSA9IGxpWydzdW1tYXJ5J10uZW5jb2RlKCdhc2NpaScsJ3Jl cGxhY2UnKQogICAgICAgICAgICAgICAgbC5kZXNjcmlwdGlvbiA9IGxpWydkZXNjcmlwdGlvbidd LmVuY29kZSgnYXNjaWknLCdyZXBsYWNlJykKCiAgICAgICAgICAgICAgICBpZiBjcmVhdGVkOgog ICAgICAgICAgICAgICAgICAgICMgcHJlZGVmaW5lZCBsYXllcnMgaW4gdGhlIGZpeHR1cmVzIChm b3IgZXhhbXBsZSBwb2t5LnhtbCkKICAgICAgICAgICAgICAgICAgICAjIGFsd2F5cyBwcmVlbXB0 IHRoZSBMYXllciBJbmRleCBmb3IgdGhlc2UgdmFsdWVzCiAgICAgICAgICAgICAgICAgICAgbC52 Y3NfdXJsID0gbGlbJ3Zjc191cmwnXQogICAgICAgICAgICAgICAgICAgIGwudmNzX3dlYl91cmwg PSBsaVsndmNzX3dlYl91cmwnXQoKICAgICAgICAgICAgICAgICAgICBsLnZjc193ZWJfdHJlZV9i YXNlX3VybCA9IGxpWyd2Y3Nfd2ViX3RyZWVfYmFzZV91cmwnXVs6MjAwXQogICAgICAgICAgICAg ICAgICAgIGwudmNzX3dlYl9maWxlX2Jhc2VfdXJsID0gbGlbJ3Zjc193ZWJfZmlsZV9iYXNlX3Vy bCddWzoyMDBdCiAgICAgICAgICAgICAgICBsLnNhdmUoKQogICAgICAgICAgICBleGNlcHQgTGF5 ZXIuTXVsdGlwbGVPYmplY3RzUmV0dXJuZWQ6CiAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygi U2tpcHBlZCAlcyBhcyB3ZSBmb3VuZCBtdWx0aXBsZSBsYXllcnMgYW5kICIKICAgICAgICAgICAg ICAgICAgICAgICAgICAgICJkb24ndCBrbm93IHdoaWNoIHRvIHVwZGF0ZSIgJQogICAgICAgICAg ICAgICAgICAgICAgICAgICAgbGlbJ25hbWUnXSkKCiAgICAgICAgICAgIGxpX2xheWVyX2lkX3Rv X3RvYXN0ZXJfbGF5ZXJfaWRbbGlbJ2lkJ11dID0gbC5wawoKICAgICAgICAgICAgc2VsZi5taW5p X3Byb2dyZXNzKCJsYXllcnMiLCBpLCB0b3RhbCkKCiAgICAgICAgIyB1cGRhdGUgbGF5ZXJfdmVy c2lvbnMKICAgICAgICBsb2dnZXIuaW5mbygiRmV0Y2hpbmcgbGF5ZXIgdmVyc2lvbnMiKQogICAg ICAgIGxheWVyYnJhbmNoZXNfaW5mbyA9IF9nZXRfanNvbl9yZXNwb25zZSgKICAgICAgICAgICAg YXBpbGlua3NbJ2xheWVyQnJhbmNoZXMnXSArICI/ZmlsdGVyPWJyYW5jaF9fbmFtZTolcyIgJQog ICAgICAgICAgICAiT1IiLmpvaW4od2hpdGVsaXN0X2JyYW5jaF9uYW1lcykpCgogICAgICAgICMg TWFwIExheWVyIGluZGV4IGxheWVyX2JyYW5jaCBvYmplY3QgaWQgdG8KICAgICAgICAjIGxheWVy X3ZlcnNpb24gdG9hc3RlciBvYmplY3QgaWQKICAgICAgICBsaV9sYXllcl9icmFuY2hfaWRfdG9f dG9hc3Rlcl9sdl9pZCA9IHt9CgogICAgICAgIHRvdGFsID0gbGVuKGxheWVyYnJhbmNoZXNfaW5m bykKICAgICAgICBmb3IgaSwgbGJpIGluIGVudW1lcmF0ZShsYXllcmJyYW5jaGVzX2luZm8pOgog ICAgICAgICAgICAjIHJlbGVhc2UgYXMgZGVmaW5lZCBieSB0b2FzdGVyIG1hcCB0byBsYXllcmlu ZGV4IGJyYW5jaAogICAgICAgICAgICByZWxlYXNlID0gbGlfYnJhbmNoX2lkX3RvX3RvYXN0ZXJf cmVsZWFzZVtsYmlbJ2JyYW5jaCddXQoKICAgICAgICAgICAgdHJ5OgogICAgICAgICAgICAgICAg bHYsIGNyZWF0ZWQgPSBMYXllcl9WZXJzaW9uLm9iamVjdHMuZ2V0X29yX2NyZWF0ZSgKICAgICAg ICAgICAgICAgICAgICBsYXllcj1MYXllci5vYmplY3RzLmdldCgKICAgICAgICAgICAgICAgICAg ICAgICAgcGs9bGlfbGF5ZXJfaWRfdG9fdG9hc3Rlcl9sYXllcl9pZFtsYmlbJ2xheWVyJ11dKSwK ICAgICAgICAgICAgICAgICAgICByZWxlYXNlPXJlbGVhc2UKICAgICAgICAgICAgICAgICkKICAg ICAgICAgICAgZXhjZXB0IEtleUVycm9yOgogICAgICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmco CiAgICAgICAgICAgICAgICAgICAgIk5vIHN1Y2ggbGF5ZXJpbmRleCBsYXllciByZWZlcmVuY2Vk IGJ5IGxheWVyYnJhbmNoICVkIiAlCiAgICAgICAgICAgICAgICAgICAgbGJpWydsYXllciddKQog ICAgICAgICAgICAgICAgY29udGludWUKCiAgICAgICAgICAgIGlmIGNyZWF0ZWQ6CiAgICAgICAg ICAgICAgICBsdi5yZWxlYXNlID0gbGlfYnJhbmNoX2lkX3RvX3RvYXN0ZXJfcmVsZWFzZVtsYmlb J2JyYW5jaCddXQogICAgICAgICAgICAgICAgbHYudXBfZGF0ZSA9IGxiaVsndXBkYXRlZCddCiAg ICAgICAgICAgICAgICBsdi5jb21taXQgPSBsYmlbJ2FjdHVhbF9icmFuY2gnXQogICAgICAgICAg ICAgICAgbHYuZGlycGF0aCA9IGxiaVsndmNzX3N1YmRpciddCiAgICAgICAgICAgICAgICBsdi5z YXZlKCkKCiAgICAgICAgICAgIGxpX2xheWVyX2JyYW5jaF9pZF90b190b2FzdGVyX2x2X2lkW2xi aVsnaWQnXV0gPVwKICAgICAgICAgICAgICAgIGx2LnBrCiAgICAgICAgICAgIHNlbGYubWluaV9w cm9ncmVzcygibGF5ZXIgdmVyc2lvbnMiLCBpLCB0b3RhbCkKCiAgICAgICAgbG9nZ2VyLmluZm8o IkZldGNoaW5nIGxheWVyIHZlcnNpb24gZGVwZW5kZW5jaWVzIikKICAgICAgICAjIHVwZGF0ZSBs YXllciBkZXBlbmRlbmNpZXMKICAgICAgICBsYXllcmRlcGVuZGVuY2llc19pbmZvID0gX2dldF9q c29uX3Jlc3BvbnNlKAogICAgICAgICAgICBhcGlsaW5rc1snbGF5ZXJEZXBlbmRlbmNpZXMnXSAr CiAgICAgICAgICAgICI/ZmlsdGVyPWxheWVyYnJhbmNoX19icmFuY2hfX25hbWU6JXMiICUKICAg ICAgICAgICAgIk9SIi5qb2luKHdoaXRlbGlzdF9icmFuY2hfbmFtZXMpKQoKICAgICAgICBkZXBl bmRsaXN0ID0ge30KICAgICAgICBmb3IgbGRpIGluIGxheWVyZGVwZW5kZW5jaWVzX2luZm86CiAg ICAgICAgICAgIHRyeToKICAgICAgICAgICAgICAgIGx2ID0gTGF5ZXJfVmVyc2lvbi5vYmplY3Rz LmdldCgKICAgICAgICAgICAgICAgICAgICBwaz1saV9sYXllcl9icmFuY2hfaWRfdG9fdG9hc3Rl cl9sdl9pZFtsZGlbJ2xheWVyYnJhbmNoJ11dKQogICAgICAgICAgICBleGNlcHQgTGF5ZXJfVmVy c2lvbi5Eb2VzTm90RXhpc3QgYXMgZToKICAgICAgICAgICAgICAgIGNvbnRpbnVlCgogICAgICAg ICAgICBpZiBsdiBub3QgaW4gZGVwZW5kbGlzdDoKICAgICAgICAgICAgICAgIGRlcGVuZGxpc3Rb bHZdID0gW10KICAgICAgICAgICAgdHJ5OgogICAgICAgICAgICAgICAgbGF5ZXJfaWQgPSBsaV9s YXllcl9pZF90b190b2FzdGVyX2xheWVyX2lkW2xkaVsnZGVwZW5kZW5jeSddXQoKICAgICAgICAg ICAgICAgIGRlcGVuZGxpc3RbbHZdLmFwcGVuZCgKICAgICAgICAgICAgICAgICAgICBMYXllcl9W ZXJzaW9uLm9iamVjdHMuZ2V0KGxheWVyX19waz1sYXllcl9pZCwKICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIHJlbGVhc2U9bHYucmVsZWFzZSkpCgogICAgICAg ICAgICBleGNlcHQgTGF5ZXJfVmVyc2lvbi5Eb2VzTm90RXhpc3Q6CiAgICAgICAgICAgICAgICBs b2dnZXIud2FybmluZygiQ2Fubm90IGZpbmQgbGF5ZXIgdmVyc2lvbiAobHM6JXMpLCIKICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICJ1cF9pZDolcyBsdjolcyIgJQogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgKHNlbGYsIGxkaVsnZGVwZW5kZW5jeSddLCBsdikpCgogICAgICAg IHRvdGFsID0gbGVuKGRlcGVuZGxpc3QpCiAgICAgICAgZm9yIGksIGx2IGluIGVudW1lcmF0ZShk ZXBlbmRsaXN0KToKICAgICAgICAgICAgTGF5ZXJWZXJzaW9uRGVwZW5kZW5jeS5vYmplY3RzLmZp bHRlcihsYXllcl92ZXJzaW9uPWx2KS5kZWxldGUoKQogICAgICAgICAgICBmb3IgbHZkIGluIGRl cGVuZGxpc3RbbHZdOgogICAgICAgICAgICAgICAgTGF5ZXJWZXJzaW9uRGVwZW5kZW5jeS5vYmpl Y3RzLmdldF9vcl9jcmVhdGUobGF5ZXJfdmVyc2lvbj1sdiwKICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRlcGVuZHNfb249bHZkKQog ICAgICAgICAgICBzZWxmLm1pbmlfcHJvZ3Jlc3MoIkxheWVyIHZlcnNpb24gZGVwZW5kZW5jaWVz IiwgaSwgdG90YWwpCgogICAgICAgICMgdXBkYXRlIERpc3Ryb3MKICAgICAgICBsb2dnZXIuaW5m bygiRmV0Y2hpbmcgZGlzdHJvIGluZm9ybWF0aW9uIikKICAgICAgICBkaXN0cm9zX2luZm8gPSBf Z2V0X2pzb25fcmVzcG9uc2UoCiAgICAgICAgICAgIGFwaWxpbmtzWydkaXN0cm9zJ10gKyAiP2Zp bHRlcj1sYXllcmJyYW5jaF9fYnJhbmNoX19uYW1lOiVzIiAlCiAgICAgICAgICAgICJPUiIuam9p bih3aGl0ZWxpc3RfYnJhbmNoX25hbWVzKSkKCiAgICAgICAgdG90YWwgPSBsZW4oZGlzdHJvc19p bmZvKQogICAgICAgIGZvciBpLCBkaSBpbiBlbnVtZXJhdGUoZGlzdHJvc19pbmZvKToKICAgICAg ICAgICAgZGlzdHJvLCBjcmVhdGVkID0gRGlzdHJvLm9iamVjdHMuZ2V0X29yX2NyZWF0ZSgKICAg ICAgICAgICAgICAgIG5hbWU9ZGlbJ25hbWUnXSwKICAgICAgICAgICAgICAgIGxheWVyX3ZlcnNp b249TGF5ZXJfVmVyc2lvbi5vYmplY3RzLmdldCgKICAgICAgICAgICAgICAgICAgICBwaz1saV9s YXllcl9icmFuY2hfaWRfdG9fdG9hc3Rlcl9sdl9pZFtkaVsnbGF5ZXJicmFuY2gnXV0pKQogICAg ICAgICAgICBkaXN0cm8udXBfZGF0ZSA9IGRpWyd1cGRhdGVkJ10KICAgICAgICAgICAgZGlzdHJv Lm5hbWUgPSBkaVsnbmFtZSddCiAgICAgICAgICAgIGRpc3Ryby5kZXNjcmlwdGlvbiA9IGRpWydk ZXNjcmlwdGlvbiddLmVuY29kZSgnYXNjaWknLCdyZXBsYWNlJykKICAgICAgICAgICAgZGlzdHJv LnNhdmUoKQogICAgICAgICAgICBzZWxmLm1pbmlfcHJvZ3Jlc3MoImRpc3Ryb3MiLCBpLCB0b3Rh bCkKCiAgICAgICAgIyB1cGRhdGUgbWFjaGluZXMKICAgICAgICBsb2dnZXIuaW5mbygiRmV0Y2hp bmcgbWFjaGluZSBpbmZvcm1hdGlvbiIpCiAgICAgICAgbWFjaGluZXNfaW5mbyA9IF9nZXRfanNv bl9yZXNwb25zZSgKICAgICAgICAgICAgYXBpbGlua3NbJ21hY2hpbmVzJ10gKyAiP2ZpbHRlcj1s YXllcmJyYW5jaF9fYnJhbmNoX19uYW1lOiVzIiAlCiAgICAgICAgICAgICJPUiIuam9pbih3aGl0 ZWxpc3RfYnJhbmNoX25hbWVzKSkKCiAgICAgICAgdG90YWwgPSBsZW4obWFjaGluZXNfaW5mbykK ICAgICAgICBmb3IgaSwgbWkgaW4gZW51bWVyYXRlKG1hY2hpbmVzX2luZm8pOgogICAgICAgICAg ICBtbywgY3JlYXRlZCA9IE1hY2hpbmUub2JqZWN0cy5nZXRfb3JfY3JlYXRlKAogICAgICAgICAg ICAgICAgbmFtZT1taVsnbmFtZSddLAogICAgICAgICAgICAgICAgbGF5ZXJfdmVyc2lvbj1MYXll cl9WZXJzaW9uLm9iamVjdHMuZ2V0KAogICAgICAgICAgICAgICAgICAgIHBrPWxpX2xheWVyX2Jy YW5jaF9pZF90b190b2FzdGVyX2x2X2lkW21pWydsYXllcmJyYW5jaCddXSkpCiAgICAgICAgICAg IG1vLnVwX2RhdGUgPSBtaVsndXBkYXRlZCddCiAgICAgICAgICAgIG1vLm5hbWUgPSBtaVsnbmFt ZSddCiAgICAgICAgICAgIG1vLmRlc2NyaXB0aW9uID0gbWlbJ2Rlc2NyaXB0aW9uJ10uZW5jb2Rl KCdhc2NpaScsJ3JlcGxhY2UnKQogICAgICAgICAgICBtby5zYXZlKCkKICAgICAgICAgICAgc2Vs Zi5taW5pX3Byb2dyZXNzKCJtYWNoaW5lcyIsIGksIHRvdGFsKQoKICAgICAgICAjIHVwZGF0ZSBy ZWNpcGVzOyBwYWdpbmF0ZSBieSBsYXllciB2ZXJzaW9uIC8gbGF5ZXIgYnJhbmNoCiAgICAgICAg bG9nZ2VyLmluZm8oIkZldGNoaW5nIHJlY2lwZSBpbmZvcm1hdGlvbiIpCiAgICAgICAgcmVjaXBl c19pbmZvID0gX2dldF9qc29uX3Jlc3BvbnNlKAogICAgICAgICAgICBhcGlsaW5rc1sncmVjaXBl cyddICsgIj9maWx0ZXI9bGF5ZXJicmFuY2hfX2JyYW5jaF9fbmFtZTolcyIgJQogICAgICAgICAg ICAiT1IiLmpvaW4od2hpdGVsaXN0X2JyYW5jaF9uYW1lcykpCgogICAgICAgIHRvdGFsID0gbGVu KHJlY2lwZXNfaW5mbykKICAgICAgICBmb3IgaSwgcmkgaW4gZW51bWVyYXRlKHJlY2lwZXNfaW5m byk6CiAgICAgICAgICAgIHRyeToKICAgICAgICAgICAgICAgIGx2X2lkID0gbGlfbGF5ZXJfYnJh bmNoX2lkX3RvX3RvYXN0ZXJfbHZfaWRbcmlbJ2xheWVyYnJhbmNoJ11dCiAgICAgICAgICAgICAg ICBsdiA9IExheWVyX1ZlcnNpb24ub2JqZWN0cy5nZXQocGs9bHZfaWQpCgogICAgICAgICAgICAg ICAgcm8sIGNyZWF0ZWQgPSBSZWNpcGUub2JqZWN0cy5nZXRfb3JfY3JlYXRlKAogICAgICAgICAg ICAgICAgICAgIGxheWVyX3ZlcnNpb249bHYsCiAgICAgICAgICAgICAgICAgICAgbmFtZT1yaVsn cG4nXQogICAgICAgICAgICAgICAgKQoKICAgICAgICAgICAgICAgIHJvLmxheWVyX3ZlcnNpb24g PSBsdgogICAgICAgICAgICAgICAgcm8udXBfZGF0ZSA9IHJpWyd1cGRhdGVkJ10KICAgICAgICAg ICAgICAgIHJvLm5hbWUgPSByaVsncG4nXQogICAgICAgICAgICAgICAgcm8udmVyc2lvbiA9IHJp WydwdiddCiAgICAgICAgICAgICAgICByby5zdW1tYXJ5ID0gcmlbJ3N1bW1hcnknXS5lbmNvZGUo J2FzY2lpJywncmVwbGFjZScpCiAgICAgICAgICAgICAgICByby5kZXNjcmlwdGlvbiA9IHJpWydk ZXNjcmlwdGlvbiddLmVuY29kZSgnYXNjaWknLCdyZXBsYWNlJykKICAgICAgICAgICAgICAgIHJv LnNlY3Rpb24gPSByaVsnc2VjdGlvbiddCiAgICAgICAgICAgICAgICByby5saWNlbnNlID0gcmlb J2xpY2Vuc2UnXVs6MjAwXQogICAgICAgICAgICAgICAgcm8uaG9tZXBhZ2UgPSByaVsnaG9tZXBh Z2UnXQogICAgICAgICAgICAgICAgcm8uYnVndHJhY2tlciA9IHJpWydidWd0cmFja2VyJ10KICAg ICAgICAgICAgICAgIHJvLmZpbGVfcGF0aCA9IHJpWydmaWxlcGF0aCddICsgIi8iICsgcmlbJ2Zp bGVuYW1lJ10KICAgICAgICAgICAgICAgIGlmICdpbmhlcml0cycgaW4gcmk6CiAgICAgICAgICAg ICAgICAgICAgcm8uaXNfaW1hZ2UgPSAnaW1hZ2UnIGluIHJpWydpbmhlcml0cyddLnNwbGl0KCkK ICAgICAgICAgICAgICAgIGVsc2U6ICAjIHdvcmthcm91bmQgZm9yIG9sZCBzdHlsZSBsYXllciBp bmRleAogICAgICAgICAgICAgICAgICAgIHJvLmlzX2ltYWdlID0gIi1pbWFnZS0iIGluIHJpWydw biddCiAgICAgICAgICAgICAgICByby5zYXZlKCkKICAgICAgICAgICAgZXhjZXB0IEV4Y2VwdGlv biBhcyBlOgogICAgICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmcoIkZhaWxlZCBzYXZpbmcgcmVj aXBlICVzIiwgZSkKICAgICAgICAgICAgICAgIHJldHVybigpCgogICAgICAgICAgICBzZWxmLm1p bmlfcHJvZ3Jlc3MoInJlY2lwZXMiLCBpLCB0b3RhbCkKCiAgICAgICAgb3Muc3lzdGVtKCdzZXR0 ZXJtIC1jdXJzb3Igb24nKQoKICAgIGRlZiBoYW5kbGUoc2VsZiwgKipvcHRpb25zKToKICAgICAg ICBzZWxmLnVwZGF0ZSgpCg==