Bug 12937

Summary: Consistent naming scheme for deployed artifacts
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Martin Jansa <Martin.Jansa>
Component: coreAssignee: Unassigned <unassigned>
Status: NEW --- QA Contact:
Severity: enhancement    
Priority: Medium CC: meta.mr.watcher, meta.watcher, randy.macleod, richard.purdie
Version: 2.7   
Target Milestone: 6.99   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Yes (doc changes required)

Description Martin Jansa 2018-09-27 09:08:01 UTC 
I've started working on this in 2.6 and couple supporting changes were already merged, (e.g. https://patchwork.openembedded.org/series/9882/) but unfortunately I haven't finished the most intrusive one before 2.6 freeze.

The missing pieces as described in commit message of my WIP change:

bitbake.conf, kernel*.bbclass: include IMAGE_VERSION_SUFFIX only in the _LINK_NAME variables and change it to hard link
* move IMAGE_VERSION_SUFFIX from _NAME variables to _LINK_NAME
  that way e.g. kernel.do_deploy can be reused from sstate to
  provide "version-less" artifacts and then very fast
  do_deploy_links task just adds links with consistent suffixes
* create hard links instead of symlinks, so that whatever version
  the filename says is really there
* some IMAGE_FSTYPES might need the "version-less" IMAGE_NAME file
  to be removed first or they might either append or update the
  content of the image instead of creating new image file from
  scratch - I have seen this only with one proprietary format we
  generate with our own tool, so hopefully this isn't very common
* this is basically the mechanism are using in webOS with
  WEBOS_IMAGE_NAME_SUFFIX which is for official builds set from
  jenkins job and then all artifacts (images as well as corresponding
  kernel files) have the same version string, the implementation
  "from outside" is a bit tricky as shown in webOS OSE, the variable
  modifications:
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/conf/distro/include/webos.inc#L65
  and then various bbclasses to hook do_webos_deploy_fixup task creating
  the hardlinks for possible artifacts:
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/webos_deploy.bbclass
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/kernel.bbclass
  https://github.com/webosose/meta-webosose/blob/a35e81622aae1066591e44a132d01297ff478248/meta-webos/classes/image.bbclass
  so hopefully with all these changes in oe-core other project can
  achieve the same just by setting one variable IMAGE_VERSION_SUFFIX
Comment 1 Richard Purdie 2019-08-15 15:04:29 UTC 
Hi Martin,

Any update on what we should do with this?

Cheers,

Richard
Comment 2 Martin Jansa 2019-08-15 15:13:46 UTC 
The implementation is here:
http://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts
top 6 commits.

It seems to work fine for our internal builds, but I still haven't tested it properly in various other scenarios to send it to ML for review.

Also want to document more examples for people to better understand what will change and why this is useful.
Comment 3 Martin Jansa 2019-08-16 18:27:22 UTC 
RFC with hopefully better description sent to oe-core ML:
http://lists.openembedded.org/pipermail/openembedded-core/2019-August/285822.html

This isn't the final version, just after sending it I've noticed that the branch was missing one more fix, which is now in the jansa/artifacts branch, but I haven't sent v2 with it yet (waiting for some comments from others).
Comment 4 Martin Jansa 2019-09-02 17:17:28 UTC 
Implementation sent to ML:
http://lists.openembedded.org/pipermail/openembedded-core/2019-August/286026.html
Comment 5 Randy MacLeod 2021-03-18 15:17:23 UTC 
3.3 development is closed, move to 3.4
Comment 6 Randy MacLeod 2022-11-03 15:17:55 UTC 
Some day...
Comment 7 Martin Jansa 2023-03-13 12:29:25 UTC 
Another set of "preparation" changes sent to ML:
https://lists.openembedded.org/g/openembedded-core/message/178432

Only 2 commits are left in:
https://git.openembedded.org/openembedded-core-contrib/log/?h=jansa/artifacts

1st changes the naming scheme a bit (used to verify the selftest changes - runqemu script still need few fixes for this to pass completely).

2nd is the important one which switches the _NAME and _LINK_NAME, from symlinks to hardlinks and introduces deploy_links task.

Both will be sent for Nanbield once it's open as they might need some adaptation from other layers which use own IMAGE_FSTYPEs etc.
Comment 8 Randy MacLeod 2023-10-24 14:24:50 UTC 
Bulk move from 4.99 or 0.00 to 5.99
Comment 10 Martin Jansa 2024-03-10 15:32:32 UTC 
The whole series was rejected in:
https://lists.openembedded.org/g/openembedded-core/topic/102747718#191102
(some patches from it might be useful even without the switch to hardlinks).

I can make it more configurable (to allow using symlinks instead of hardlinks) or to create versioned links in subdirectory with the same version, so it's easier to copy just one hardlink either with version or without in filename).

But I don't want to spend even more time on this if there is no agreement to fix this (I though we had agreement to use hardlinks as it was mentioned in all previous patch series and in this ticket).
Comment 11 Randy MacLeod 2026-08-13 15:20:37 UTC 
We could review the work done and try to get agreement again.