Bug 10350

Summary: devtool: deploy-target: deploy run time dependencies
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Henry Bruce <henry.bruce>
Component: devtools / tool chainAssignee: Scott Rifenbark <srifenbark>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: bluelightning, brian.avery, meta.mr.watcher, meta.watcher, srifenbark, tim.orling
Version: 2.2   
Target Milestone: 2.3 M4   
Hardware: x86   
OS: Multiple   
Whiteboard: 12 April 2017: RESOLVED
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Done (doc changes complete)
Bug Depends on:    
Bug Blocks: 10653    

Description Henry Bruce 2016-09-29 23:05:40 UTC
When running a 'devtool add' for new project 'foo' that introduces new runtime dependencies, the dependent packages are not deployed. So running 'devtool deploy-target foo' will succeed but running 'foo' on target will fail due to missing dependencies.

An example would be creating npm packages with devtool for an image that does already include the node runtime. When you deploy the package, the node runtime is not present so the module cannot run.
Comment 1 Paul Eggleton 2016-09-30 09:30:56 UTC
To a degree this is intentional, because solving this is not straightforward. We know the runtime dependencies, and deploying those at the same time is pretty easy - but what we don't necessarily know is which of those dependencies are already satisfied on the target (worst case, it's an image we don't have metadata for with no runtime package management). If we can figure out a practical way of addressing that then I'm all for fixing it.
Comment 2 Henry Bruce 2016-09-30 16:31:10 UTC
We should make sure this limitation is documented while we work out a solution. Documentation should help developer work out which runtime dependencies need to be deployed ahead of time.
Comment 3 brian avery 2017-02-16 21:29:48 UTC
Henry could you document this as is?

I'd also add that we could add a -f or 0a flag to deploy target that would copy all the runtime dependencies. so you could do a workflow like
deploy-target -a cow
modify/build cow
deploy-target cow 
iterate....
basically let the user override to say, push all and then push just the target
Comment 4 Tim Orling 2017-02-19 06:08:35 UTC
+1 for 'devtool deploy-target -a cow'

In upgrading libmodule-build-tiny-perl, I found that I could not test it on qemu without adding all the RDEPENDS to the image (there are 23 of them). I think it is acceptable to allow the developer to have a hammer which installs all the RDEPENDS (including overwriting those already on the target). What to do with undeploy becomes messier... some kind of uninstall manifest which tracks what was replaced?

Another idea would be to perhaps provide a mechanism to bbappend the image package group and then performing 'devtool build-image core-image-foo-bar-baz' in the workspace. In this way we can help the developer understand what will be required to actually deploy the recipe in question.
Comment 5 Henry Bruce 2017-03-08 19:32:39 UTC
I agree with Brian's -a option to deploy all dependencies. However this will require further discussion with Paul and should should be a 2.4 feature.

In the meantime, I assign to Scott so docs are updated to make sure developers know that devtool will not deploy dependencies.

Once that's done, we can set target milestone to 2.4.
Comment 6 Scott Rifenbark 2017-03-13 20:13:27 UTC
Accepting - I will put the appropriate warnings and behaviors in the existing doc for now. 

Scott
Comment 7 Scott Rifenbark 2017-03-13 20:37:43 UTC
Look at this section in the sdk-manual - http://www.yoctoproject.org/docs/2.3/sdk-manual/sdk-manual.html#sdk-dependency-detection-and-mapping.  It is titled "Dependency Detection and Mapping" and is a subsection of the "http://www.yoctoproject.org/docs/2.3/sdk-manual/sdk-manual.html#sdk-a-closer-look-at-devtool-add" section.  It talks about dependencies.  Does the information in this section apply to the problem here?
Comment 8 Henry Bruce 2017-03-13 20:43:45 UTC
The section you found refers to the devtool "add" sub-command and how to manually edit a recipe if it fails to automatically determine run-time dependencies. 

This bug refers to the "deploy-target" sub-command, where we assume runtime dependencies have been correctly defined in recipe but the tool does not deploy them to the target. Make sense?
Comment 9 Scott Rifenbark 2017-03-14 16:21:57 UTC
Hey Henry, 

I get it that the issue is more with deploy.  However, that section about dependencies probably needs a pointer to the explanation as devtool add apparently does not work as expected right in setting up and knowing about all dependencies.  Isn't it just the action of deploying that unveils this?  So a solution I think would be to describe the issue in a devtool deploy area but have some sort of pointer or reference from devtool add section whose topic matter is indeed about dependencies.  

In the ref-manual, there is a quick reference on devtool (http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#ref-devtool-reference).  Further down in that section is a subsection named "Deploying Your Software on the Target Machine" at http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#devtool-deploying-your-software-on-the-target-machine.  Right now, this section has nothing of sort about software not running correctly due to missing dependencies.  It seems this would be the place to detail the issue in this bug.  Correct?

Thanks,
Scott
Comment 10 Scott Rifenbark 2017-03-17 17:56:23 UTC
Changing back to Scott and setting to IN PROGRESS DESIGN.
Comment 11 Scott Rifenbark 2017-03-18 19:35:25 UTC
Henry, 

See the additional information at the bottom of the http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#devtool-deploying-your-software-on-the-target-machine section to describe this situation.  Make sure I am covering the bases here.

Also, see the note added to the http://www.yoctoproject.org/docs/2.3/ref-manual/ref-manual.html#devtool-adding-a-new-recipe-to-the-workspace section to cross-reference the situation. 

Let me know of any tweaks, changes, etc. 

Scott
Comment 12 Scott Rifenbark 2017-03-20 17:50:15 UTC
Setting to 2.3 M4.
Comment 13 Henry Bruce 2017-03-27 19:09:28 UTC
If feel the wording could be clearer but it gets the message across.
I recommend you move the new paragraph closer to the the existing description of deploy-target operation "This command deploys all files installed during the do_install task" as this will help clarify that no dependent packages will be deployed.
Comment 14 Scott Rifenbark 2017-03-27 21:25:05 UTC
Henry, 

I reworded that first bullet of the new information after the note.  I don't want to move that block of information though.  The reason I don't is because the section introduces the command basics and then the new information is additional information about some finer points of using the command.  I am going to leave that structure in place.  

I am open to any rewordings you think might make the text clearer.

Thanks, 
Scott
Comment 15 Henry Bruce 2017-03-27 21:37:28 UTC
Thanks Scott. Looks good. That should do it for warning user of current behavior. 

Question: Do we leave this bug open for implementation in 2.4 or close it and open a new one?
Comment 16 Scott Rifenbark 2017-03-27 22:06:58 UTC
Hi Henry, 

I would say we close this one and open a new one.  If they fix the behavior for 2.4 then the documentation component of the bug would be to update the manual back to "normal" (or however it will behave for 2.4).

Scott
Comment 17 Scott Rifenbark 2017-04-12 17:44:44 UTC
Closing... the doc part is done.

Scott