Bug 8684

Summary: Missing python-ast package in generate-menifest-2.7.py
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Kei Nohguchi <kei>
Component: devtools / tool chainAssignee: Ross Burton <ross.burton>
Status: RESOLVED FIXED QA Contact:
Severity: major    
Priority: Medium CC: benjamin.esquivel, kei, laurent.bonnans, meta.mr.watcher, meta.watcher, ross.burton
Version: 2.0   
Target Milestone: 2.1 M3   
Hardware: Other   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Kei Nohguchi 2015-11-12 00:27:40 UTC
scripts/contrib/python/generate-manifest-2.7.py doesn't expose the python-ast, Abstruct Syntax Trees[1], package.  One of our OpenSwitch[2] component, ops-ansible[3], requires that package to make the ansible python scripts run on our OpenSwitch target.

We mirror the poky environment and want to upstream the changes as much as possible, so that we can easily downstream it into our build system.

[1] https://docs.python.org/2/library/ast.html
[2] http://www.openswitch.net/
[3] https://review.openswitch.net/#/c/2260/1/yocto/openswitch/meta-distro-openswitch/recipes-ops/mgmt/ops-ansible.bb
Comment 1 Kei Nohguchi 2015-11-12 01:10:25 UTC
Here is the actual change request against our oe/poky mirror.

https://review.openswitch.net/#/c/2251/

As indicated in one of the review comment, I'm now preparing to send the email to the OE maintainer but please let me know if I'm not the right track.

Cheers,

Kei
Comment 2 Kei Nohguchi 2015-11-12 02:12:57 UTC
Added myself in Cc list.
Comment 3 Ross Burton 2015-11-12 10:05:18 UTC
That's certainly the right approach to splitting it out, but it is currently packaged in the "everything else" package, python-misc.
Comment 4 Kei Nohguchi 2015-11-12 19:09:50 UTC
Thank you, Ross, for confirming the change.

I've also sent a patch to Richard Purdie <richard.purdie@linuxfoundation.org> of OE.  I'll wait for his reply.

Kei Nohguchi <kei@nohguchi.com>
To: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Kei Nohguchi <kei@nohguchi.com>
[PATCH] Fixes [YOCTO #8684] Add python-ast, Python Abstruct Syntax Tree package

In a meanwhile, I'll play with python-misc to get things done. :)

Cheers and have a wonderful day today!

(In reply to comment #3)
> That's certainly the right approach to splitting it out, but it is currently
> packaged in the "everything else" package, python-misc.
Comment 5 Ross Burton 2015-11-12 19:16:23 UTC
Patches need to go do the list to be reviewed in public, so can you resend that mail to the mailing list?
Comment 6 Kei Nohguchi 2015-11-12 22:09:01 UTC
Thank you Ross!

I've sent the patch to the OE-core, as below.

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/112694.html

Cheers,

(In reply to comment #5)
> Patches need to go do the list to be reviewed in public, so can you resend
> that mail to the mailing list?
Comment 7 Kei Nohguchi 2015-11-13 02:10:33 UTC
Hi Ross,

I've tried this python-misc but when I made a image with python-misc package, instead of the python-ast package, I got different error saying "ImportError: No module named compiler"...

Since this is specific some ansible module issue, I think I'll work on OE-core and make the python-ast officially supported by the OE.

The patch is updated, as suggested by Chris Larson, and hopefully will be approved by OE-core.

http://patchwork.openembedded.org/patch/107551/
http://patchwork.openembedded.org/patch/107499/

Just FYI. 

debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Sending command: /bin/sh -c 'LANG=C LC_CTYPE=C python'
debug1: mux_client_request_session: master session id: 2
Traceback (most recent call last):
  File "<stdin>", line 242, in <module>
ImportError: No module named compiler
debug1: client_input_channel_req: channel 2 rtype exit-status reply 0
debug1: channel 2: free: client-session, nchannels 3
debug1: channel 1: free: mux-control, nchannels 2


(In reply to comment #3)
> That's certainly the right approach to splitting it out, but it is currently
> packaged in the "everything else" package, python-misc.
Comment 8 Laurent Bonnans 2016-03-04 13:42:34 UTC
Hello,

As the 'ast' module contains a small number of helpers for the low level '_ast' which is a builtin in python (https://docs.python.org/2/library/ast.html), perhaps it would suffice to add it to the core modules ('python-core') instead of adding another small standalone package.

Also, it should probably be added to python3.x as well.
Comment 9 Ross Burton 2016-03-08 17:02:28 UTC
A package for a single small source file that is basically required to use a builtin function doesn't seem very sensible.  I agree that it should be moved to core, patch incoming.
Comment 10 Ross Burton 2016-03-16 21:58:48 UTC
oe-core 48bc643a4ebe74d0fa49fedbe7b0fd63fd0003e8 moves the ast libraries into -core.
Comment 11 Kei Nohguchi 2016-03-18 17:40:37 UTC
Thank you for taking care of this!

Really appreciated!

Cheers,