Bug 2645 - [DOC] The ADT documentation should be improved to include additional command line instructions
Summary: [DOC] The ADT documentation should be improved to include additional command ...
Status: RESOLVED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: devtools / tool chain (show other bugs)
Version: unspecified
Hardware: x86 Multiple
: Medium enhancement
Target Milestone: 1.4
Assignee: Scott Rifenbark
QA Contact:
URL: http://git.yoctoproject.org/cgit.cgi/...
Whiteboard: 1 March 2013: In Progress Review
Depends on:
Blocks:
 
Reported: 2012-06-21 20:08 UTC by Mark Hatle
Modified: 2013-04-25 15:10 UTC (History)
8 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: ---
Verified:
Documentation change: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Hatle 2012-06-21 20:08:43 UTC
There are two primary use cases that I am familiar with for the ADT (SDK) generation.

The first is a simple how do I compile something case.  This seems not be present in the ADT documentation.

What I'd like to be able to do is [on the command line]:

build an SDK
extract/install the SDK
create a small hello-world.c program
compile the hello-world.c program
copy the item to my target, execute it and verify it works

The second case seems to be partially covered today, but likely needs to some clearer step by step information:

build an SDK
extract/install the SDK
configure/compile an autotools based project
copy a resulting executable to the target, execute it and verify it works

(GNU used to have a hello-world like autotools project, this might be a reasonable candidate.)
Comment 1 Jessica 2012-06-21 20:19:15 UTC
Hi Hongna,

Does QA already have some test cases cover these 2 scenarios, can you just copy the steps in the comments then I'll re-assign it to ScottR for capture in ADT manual?

Thanks,
Jessica
Comment 2 Hongna Xu 2012-06-25 06:09:08 UTC
hi jessica
  these two use cases are covered by current QA Test plan. we test it with eclipse IED which covers the command line mode, the steps are:
1.install toolchianSDK.(there are three methods to install it, please reference to www.yoctoproject.org/docs/current/adt-manual/adt-manual.html#installing-the-adt)
2.Installl the Eclipse Yocto Plug-in(http://www.yoctoproject.org/docs/current/adt-manual/adt-manual.html#installing-the-eclipse-yocto-plug-in) or install from Eclipse "Help"-> "Install New Software" -> Add -> set a location where you download the plug-in (the file name end with archive.zip) from autobuilder.
3.Configuring the Eclipse Yocto Plug-in.(http://www.yoctoproject.org/docs/current/adt-manual/adt-manual.html#configuring-the-eclipse-yocto-plug-in)
4.In eclipse, select File -> New -> Project.
5.Double click C/C++.
6.Click C or C++ Project to create the project.
7.Expand Yocto ADT Project.
8.Select Hello World ANSI C Autotools Project.
9.Put a name in the Project name. Do not use hyphens as part of the name.
10.Click Next.
11.Add information in the Author and Copyright notice fields.
12.Click Finish.
13.If the "open perspective" prompt appears, click "Yes" so that you in the C/C++ perspective.
14.Select the project, then click Project-> Change Yocto Project Settings to set toochian and target options, then click Project -> Reconfigure Project(In Juno Eclipse release "Reconfigure Project" will be removed)
15.Right click the  project -> Build project.
16.After build successfully, right click the project-> Run configuration, in C/C++ Remote Application, it will generate a configuration named ${ProjectName}-gbd-${TargetArch}-poky-linux, click it then go to its main tab.
17.In main tab, new a connection based TCF, just input the IP address of the remote target, enter the "Remote Absolute File Path for C/C++ Application", for example /home/root/hello, others we use default settings, clicke [Run] button. 
18.you will see "Hello World" in eclipse's console. you can also verify it in remote target machine, there should be a file named hello in /home/root/, file it by command "file /home/root/hello" to confirm it and run it in target machine by "./hello", it will print "hello world".
Comment 3 Scott Rifenbark 2012-06-25 17:38:52 UTC
the steps from Comment 2 are already covered in the ADT manual.  I need some detailed command-line steps to document the scenario that Mark is calling for.  Can someone provide those command-line steps?

ScottR
Comment 4 Song Liu 2012-08-20 20:35:30 UTC
Hi Shane,

Scott asked some help in this bug for documentation. Do you think Hongna can still help or we need to find someone else?

Thanks,
Song
Comment 5 Hongna Xu 2012-08-21 10:55:25 UTC
the case is being reviewed.
Comment 6 Hongna Xu 2012-08-22 09:42:16 UTC
The ADT manual has described how to build an SDK and extract/install it at  http://www.yoctoproject.org/docs/current/adt-manual/adt-manual.html#installing-the-adt, now I organized them step by step clearly in command line (I provided these steps cause it is more clearly step by step to do it,  users don’t have to find them in the detail introduction, so it is Mark’s call whether use them or not)

 5. Using the Command Line

  5.1.Build an SKD

   There are three methods to generate the ADT SDK, they are using the ADT installer script, using an existing toolchain tarball and using the toolchain from a Yocto project build tree.

   5.1.1 Building the ADT Installer

     1.clone the source code, for example poky tree.

         $:git clone git://git.yoctoproject.org/poky

     2.check the right branch you want to build.

     3.source the environment file.

         $:source oe-init-build-env

     4.run command bitbake adt-installer,after the building , you will find the adt_installer.tar.bz2 in tmp/deploy/sdk in the Yocto project build tree.

        $: bitbake adt-installer


   5.1.2 Generating a Cross-Toolchian Tarball

    1.clone the source code, for example poky tree.

        $: git clone git://git.yoctoproject.org/poky.

    2.check the right branch you want to build the SDK.

    3.source the environment file.

        $:source oe-init-build-env

    4.modify the file conf/local.conf to set SDKMACHINE ?= “x86_64” and  MACHINE ?= “qemux86”, it will generate toolchain tarball with 64bit host and 32bit target.

    5.run bitbake meta-toolchain,

       $: bitbake meta-toolchain

when the bitbake command completes, the tarball will be in tmp/deploy/sdk of the Yocto project build tree, the name likes poky-eglibc-x86_64-i586-toolchain-${version}.tar.bz2

 
   5.1.3 Bitbake with the Yocto Project Tree

      1.clone the source code, for example poky tree.

          $: git clone git://git.yoctoproject.org/poky

      2.check the right branch you want to build.

      3.source the environment file located in the Yocto Project files.

           $: source oe-init-build-env

      4. modify the variable MACHINE in file conf/local.conf to set for the target architecture, for example I set  MACHINE ?=”qemux86”, and run

          $: bitbake meta-ide-support

it will generate a file named environment-setup-i586-poky-linux in tmp folder in Yocto Project build tree.

 

  5.2 extract/install SDK


     5.2.1  Using the ADT installer script

         1.prepare the ADT Installer, you can get it from the autobuilder, also you may generate by a Yocto project build tree.

         2. unpack the tarball and it creates a directory adt-installer.

             $:tar xjf adt_installer.tar.bz2

         3.modify adt-installer/adt-installer.conf to configurate it, follow the section of “2.1.1.2. Configuring and Running the ADT Installer Script” in this wiki, for example I set YOCTOADT_REPO="http://adtrepo.yoctoproject.org/1.2.1"

         4.another configuration file is adt-installer/opkg/conf/opkg-sdk-${HOST-ARCH}.conf, you should point the location of the SDK ipk packages, for example, my machine is 64bit, so I set : src yp-i686-nativesdk http://adtrepo.yoctoproject.org/1.2.1/adt-ipk/x86_64-nativesdk/ in opkg-sdk-x86_64.conf.

         5.then run the script adt_installer

             $:./adt_installer

         6.after installation, you can find the SDK in /opt/poky/.

              

    5.2.2  using an existing toolchain tarball

         1.prepare the cross-toolchain tarball, for example I download one,named  poky-eglibc-x86_64-i586-toolchain-gmae-1.2.1.tar.bz2 from http://downloads.yoctoproject.org/releases/yocto/yocto-1.2.1/toolchain/

         2. Make sure you are in the root directory with root privileges and then expand the tarball

           #:tar xjf poky-eglibc-x86_64-i586-toolchain-gmae-1.2.1.tar.bz2

the tarball expands into /opt/poky/1.2.1. Once the tarball is expanded, the cross-toolchain is installed.

            

   5.2.3  using the toolchain from a Yocto project build tree

        1.just source the environment file generated by section 5.1.3 Bitbake with the Yocto Project Tree, but without target’s roofs.

        $:source environment-setup-i586-poky-linux

 

 5.3 Autotools-Based Project

    5.3.1 create a project based on GNU autotools

         1.new a directory for the GNU autotool helloworld project.
            $:mkdir helloworld

         2.go into the the helloworld folder
           $:cd helloworld

         3.new a file named hello.c, add the lines to it

           ****************

                 #include <stdio.h>

                  main()

                        {

                         printf("Hello World!\n");

                       }

           *******************


        4.new Makefile.am , add the lines to it:
         ************

         bin_PROGRAMS = hello 

         hello_SOURCES = hello.c

         ************

        5.new configure.in, the file like this:

          *************

          AC_INIT(hello.c)

          AM_INIT_AUTOMAKE(hello,0.1)

          AC_PROG_CC

          AC_PROG_INSTALL

          AC_OUTPUT(Makefile)

          ****************
     6.source the environment file in /opt/poky or yocto project build tree.

     7.then run aclocal and autoconf.
       
        $:aclocal
  
        $:autoconf

     8.Run “touch NEWS README AUTHORS ChangeLog ” to generate some files which are required by the GNU coding standards

     9.Run "automake –a" to generate the configure file

  5.3.2  cross compile the project

     1.go into the helloworld folder, source the cross compile environment in /opt/poky or in yocto project build tree.

     2.cross configure the project.

      $:./configure ${CONFIGURE_FLAGS}

    3.after configuration without error, run make and make install DESTDIR=./tmp
     
      $:make 
  
      #:make isntall DESTDIR=./tmp

   4.a simple way to verify the installation, just run "file ./tmp/usr/local/bin/hello", it will print which arch the binary file could run on, the arch should be same with  the toolchain’s target arch you installed, you can also copy this file to a target machine and run it
  
  $:./hello
 
  it will print “Hello world!”.
Comment 7 Scott Rifenbark 2012-08-23 22:41:03 UTC
I have studied Hongna Xu detailed steps in his comment and they are pretty much in line with what is in the current YP Application Developer's Guide (formerly ADT Manual).  You can find it at http://www.yoctoproject.org/docs/latest/adt-manual/adt-manual.html. 

The guide covers the three ways to install and extract what Hongna refers to as the ADT SDK.  The manual currently covers all the exact command lines needed to install the ADT and configure it with the exception of setting the SDKMACHINE variable and the MACHINE variable in local.conf in the case of generating a cross-toolchain tarball before running bitbake meta-toolchain (Hongna's section 5.1.2 and step 4).  I will add this to the appropriate section in the manual to close that gap.  

There is another point I can add to cover steps 3 and 4 in Hongna's section 5.2.1 (Using the ADT Installer Script).  This has to do with setting YOCTOADT_REPO and the other configuration file mentioned:  adt-installer/opkg/conf/opkg-sdk-${HOST-ARCH}.conf.  I will find out more about this and modify the existing sections in the manual to close that gap.  

Other than these two gaps, I feel the existing documentation and its organization are fine.  The user should be able to download/build the SDK, configure it, and install it.  Granted, Hongna's approach is not to use existing tarballs that are downloaded from the YP respositories, but rather, he uses Git repositories and builds from there.  If that gap is a big concern I can mention that alternative as a method along with the instruction that points the user to the release areas to get tarballs. 

Hongna's section 5.3 is concerned with Autotools-based projects.  Specifically, how to create and compile one.  This section in the manual does exist but is sparse.  I can take Hongna's specific steps and significantly improve the information in that section based on his information in the comments section 5.3.

Summary:  My proposal is to close the two small gaps in Section 2.1 of the manual and to incorporate the detailed steps of the Autotools-based example, which is in section 4.1 of the manual (http://www.yoctoproject.org/docs/latest/adt-manual/adt-manual.html#autotools-based-projects).  I don't want to reorganize the general flow of the material so that it matches Hongna's steps.  Hopefully, this solution will satisfy Mark's original concern.  

Scott
Comment 8 Scott Rifenbark 2012-09-07 19:16:41 UTC
Hongna, 

Are there significant differences between Juno and Indigo when building and running the project in the IDE?  If so, I need to account for that in the docs.  Your instructions look to be Juno.  Right now all I am encountering is the "Reconfigure Project" difference noted in Comment 2 Step 14.
Comment 9 Hongna Xu 2012-09-08 01:53:24 UTC
hi ScottR
 There are not too much different between Juno and Indigo in IDE options, Comment 2 described with Indigo version, if you want to perfect the docs, i think Laurentiu or Georgescu, Alexandru C could help.
Comment 10 Scott Rifenbark 2012-09-10 15:38:19 UTC
Hongna, 

I am trying to duplicate the steps in 5.3 in Comment 6 and am failing when I follow your step 7.  Here is what happens:

* I create a new directory named "helloworld"
* I cd into that directory 
* I create the three files hello.c, Makefile.am, and configure.in
* I source the setup file by running 'source ~/poky/oe-init-build-env' while still in the helloworld directory.
* I attempt to run :aclocal

Here is the transcript:

srifenbark@scottrif-desktop:~$ cd helloworld
srifenbark@scottrif-desktop:~/helloworld$ source ~/poky/oe-init-build-env

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

Common targets are:
    core-image-minimal
    core-image-sato
    meta-toolchain
    meta-toolchain-sdk
    adt-installer
    meta-ide-support

You can also run generated qemu images with a command like 'runqemu qemux86'

srifenbark@scottrif-desktop:~/helloworld/build$ :aclocal
No command ':aclocal' found, did you mean:
 Command 'aclocal' from package 'automake1.10' (main)
 Command 'aclocal' from package 'automake' (main)
 Command 'aclocal' from package 'automake1.4' (main)
 Command 'aclocal' from package 'automake1.9' (main)
 Command 'aclocal' from package 'automake1.7' (universe)
:aclocal: command not found
srifenbark@scottrif-desktop:~/helloworld/build$ 


I also tried to run aclocal without the leading colon character and this is what I get:

srifenbark@scottrif-desktop:~/helloworld/build$ aclocal
aclocal: `configure.ac' or `configure.in' is required
srifenbark@scottrif-desktop:~/helloworld/build$ 

Are there missing steps?  Am I supposed to be in a different directory?  The steps are not explicit enough for me to duplicate and understand what is going on.  Can you please provide missing or corrected pieces. 

Thanks, 
Scott
Comment 11 Mark Hatle 2012-09-10 16:03:35 UTC
Typos in the original documentation..  there are no colon (:) before the aclocal or other calls..

--Mark
Comment 12 Hongna Xu 2012-09-11 00:56:49 UTC
Scott R, I am so sorry that all commands should put the colon (:) off. :(
Comment 13 Scott Rifenbark 2012-11-14 01:12:18 UTC
The two gaps I identified in my Comment 7 below are being addressed.  I am waiting for some more information from Jessica on the SDKMACHINE issue.  I am trying to understand if it is really necessary to mess with this commented out variable in the local.conf file.  If so, I will alter the manual.  The other gap had to do with the adt-installer/opkg directory and the two configuration files in there.  I have updated the danny version (1.3.1) of the adt-manual where it talks about setting YOCTOADT_REPO and included a bit about needing to edit one of these files if you are using your own adt repo for the ipkg stuff.  Jessica informed me that the 1.4 version of the adt will be sstate based and the adt-installer/opkg directory with the config files goes away.  So I making these changes on the tip of danny in yocto-docs only.  

I have been able to duplicate the simple helloworld example provided by Hongna with one variation.  I had to include the following statement in my configure.in file:

AC_CONFIG_HEADERS(config.h)

The absence of this statement did not prevent the project from compiling and running but it did prevent me from directly passing --with options to the configure script.  The solution I was trying to verify was the note in the section about autotools-based projects.

More later when I complete the work on this stuff.

Scott
Comment 14 Scott Rifenbark 2012-11-14 18:46:42 UTC
All changes are in to satisfy this bug.  I need an approval on the new text before I mark it resolved.  Please review and send feedback.

You can see the new text here:

http://www.yoctoproject.org/docs/1.4/adt-manual/adt-manual.html#using-an-existing-toolchain-tarball for the note about MACHINE and SDKMACHINE when generating a cross-toolchain using bitbake meta-toolchain

http://www.yoctoproject.org/docs/1.4/adt-manual/adt-manual.html#autotools-based-projects for the new Hello World example.

Scott
Comment 15 Jessica 2012-11-29 22:41:49 UTC
Scott,

The content that you've added is correct, but since we're still changing ADT installation for 1.4 (make installer based on sstate), so I'd suggest we hold closing on this bug.  Also, for 1.3 Mark added a new way to generate toolchain to match a specific sysroot, e.g. bitbake -c populate_sdk core-image-sato, then the sysroot directory of the toolchain will match your target, and we probably should advocate more of this usage compare to just bitbake meta-toolchain and meta-toolchain-sdk/gmae which are prefixed.
Comment 16 Jessica 2013-02-28 21:37:24 UTC
Hi Scott,

Since the ADT installation using sstate won't happen in 1.4, I think we probably can move on and toward fixing and close this bug.  I took a look of the ADT manual and have following comments:

1. In the doc, we still using /opt/poky across board referring to where toolchain is installed.  I'd like us to change that to a user specified location since this will show a big improvement in 1.3 that our toolchain now are relocatable

2. section 2.3.2, I'd like to not refering the files under user's build directory as much as possible so please change this:

   $ source $HOME/poky/build/tmp/environment-setup-i586-poky-linux
     $ runqemu-extract-sdk \
        tmp/deploy/images/core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2 \
        $HOME/qemux86-sato

to 
$ source $HOME/toolchain_dir/environment-setup-i586-poky-linux
$ runqemu-extract-sdk ~/Downloads/core-image-sato-sdk-qemux86-2011091411831.rootfs.tar.bz2 \
        $HOME/qemux86-sato

3. 2.1.2 Using a cross toolchain tarball.  There's a big feature improvements in 1.3 that allows user to build a toolchain tarball that contains sysroot that matching target rootfs contents by doing: bitbake image -c populate_sdk.  Please make sure it is metioned and specifically call out in the manual.
Comment 17 Scott Rifenbark 2013-03-01 20:33:46 UTC
Went through the manual and located all occurences of "/opt/poky".  I changed all instances to read appropriately to be sure the reader understands that this directory is a default directory and the user has clear control as to where to install the ADT.

I also updated the note in Section 2.1.2 "Using a Cross-Toolchain Tarball" (http://www.yoctoproject.org/docs/1.4/adt-manual/adt-manual.html#using-an-existing-toolchain-tarball) so that it calls out the method of using bitbake image -c populate_sdk as a way to build the tarball and have the tarball contain a matching sysroot.  

I am marking the bug as in review.  If I don't hear back in a reasonable amount of time then I will mark it closed before the release. 

Thanks, 
Scott