Bug 7515

Summary: Add CVE testing tool
Product: [Yocto Project Subprojects] Patchwork/Patchtest Reporter: Ross Burton <ross.burton>
Component: Patchtest TestsuiteAssignee: Mariano Lopez <mariano.lopez>
Status: RESOLVED FIXED QA Contact: Lucian Musat <georgex.l.musat>
Severity: enhancement    
Priority: Medium+ CC: akuster, alex.kanavin, bluelightning, jose.a.lamego, joshuagloe, leonardo.sandoval.gonzalez, mariano.lopez, mhalstead, richard.purdie, stanciux.mihail, zhenfeng.zhao
Version: unspecified   
Target Milestone: 2.2 M4   
Hardware: All   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=10771
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)
Bug Depends on: 8119    
Bug Blocks:    

Description Ross Burton 2015-03-25 15:22:06 UTC
Clear Linux has released their CVE checking tool (https://github.com/ikeydoherty/cve-check-tool) which verifies that all known CVEs are fixed in their distribution.  We should extend that tool to read bitbake metadata.
Comment 1 Saul Wold 2015-03-26 16:09:17 UTC
Please also refer to our existing tool: 
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=mulhern/cvechecker-final2, 
which uses an upstream tool and set of scripts in the 3rd commit from the top.

We want to consider integrating this in with recipetool and AUH.

This may be something given to one of the othe GDC, but leaving with Anibal as 
starting point.
Comment 2 Ross Burton 2015-04-16 17:03:57 UTC
The author of the Clear tool isn't very complementary of the upstream tool.  Integration into other tooling such as AUH is useful but the ability to run standalone is fairly important I'd say: the ability to run a full CVE check when you're getting ready to release is key.
Comment 3 Alexandru Georgescu 2015-05-04 11:15:39 UTC
Lucian,
Maybe we can dry run this before it's implemented by Ross.
Comment 4 Aníbal Limón 2015-08-06 20:19:47 UTC
Talking with RP it have a dependency in upstream patching format that need to specify if patch is CVE.
Comment 5 Armin Kuster 2015-10-28 21:16:39 UTC
What is the intent of such a framework or tool?

What will the Yocto Project do with the data is gathers from such a tool?

Is the AB ok with the Yocto project exposing security vulnerabilities to AB member customers before the AB members can get their responses / fixes distributed? Or the AB member customers may not approve if they them selves can get the fixes in their products.

- Armin
Comment 6 Armin Kuster 2015-10-28 22:22:44 UTC
I have been avoiding commenting on this as I was hoping an unbiased individual would have run this to see what it takes to accurately get a listing of vulnerabilities. 

I have not run this myself but the description is similar in nature to OpenVas which I have run.  They both need databases to be able to provide notification if a package is vulnerable. The first issue is the data provided in security notifications are not always accurate. A common statement is Version X and below are affected. Now someone needs to look at the code to see if this is true. Kernel tends to be worst offender in this. Who has the time to do this?

So we use the prserver.. all packages are 'R0' and we don't rev no mater how many security patches have been applied. If the scanning s/w is run, it will produce false positives. Do we update the DB? Or a copy of the prserve data needs to be provided and synced with a commit so to minimize false positives. any false positives need to be  updated in the database. Who has the time to do this? This comparison tends to  works if the package itself is updated to a newer version.

I would rather spend the time fixing something than creating and maintaining a database to tell me something is possible affected.
Comment 7 Ross Burton 2015-11-03 16:24:21 UTC
As far as a tool goes, we've recently open-sourced something that was being developed privately by a sister team: https://github.com/01org/isafw

It's a framework of security-related tools but the relevant bit for this is ISA_cve_plugin which uses cve-check-tools (https://github.com/ikeydoherty/cve-check-tool) for the real work.

I'm building this now to see how well it works.
Comment 8 Ross Burton 2015-11-03 16:33:20 UTC
Erm, make that https://github.com/01org/meta-security-isafw for the layer.
Comment 9 Aníbal Limón 2015-11-03 16:46:22 UTC
(In reply to comment #8)
> Erm, make that https://github.com/01org/meta-security-isafw for the layer.

I tried to use the layer but fails when try to INHERIT, 

Traceback (most recent call last):
  File "/home/alimon/repos/poky/bitbake/lib/bb/parse/parse_py/ConfHandler.py", line 100, in include(parentfn='configuration INHERITs', fn='classes/isafw.bbclass', lineno=0, data=<bb.data_smart.DataSmart object at 0x7f0f426eb950>, error_out='inherit'):
                 if error_out:
    >                raise ParseError("Could not %s file %s" % (error_out, fn), parentfn, lineno)
                 logger.debug(2, "CONF file '%s' not found", fn)
ParseError: ParseError in configuration INHERITs: Could not inherit file classes/isafw.bbclass
Comment 10 Ross Burton 2015-11-03 17:59:23 UTC
Anibal, did you add the layer to bblayers.conf?
Comment 11 Aníbal Limón 2015-11-03 18:08:41 UTC
(In reply to comment #10)
> Anibal, did you add the layer to bblayers.conf?

Yes, without it in bblayers.conf don't fail.
Comment 12 Ross Burton 2015-11-03 18:29:42 UTC
As an experiment into using cve-check-tool without the isafw surrounding it I wrote a ten-line python class and added it to meta-ross (github:rossburton/meta-ross).  With current jethro against core-image-sato I get this:

WARNING: iptables: Unpatched CVEs: CVE-2012-2663
WARNING: readline: Unpatched CVEs: CVE-2014-2524
WARNING: expat: Unpatched CVEs: CVE-2013-0340
WARNING: subversion: Unpatched CVEs: CVE-2015-3184 CVE-2015-3187
WARNING: vte: Unpatched CVEs: CVE-2012-2738
WARNING: iptables: Unpatched CVEs: CVE-2012-2663
WARNING: unzip: Unpatched CVEs: CVE-2014-9636
WARNING: expat: Unpatched CVEs: CVE-2013-0340
WARNING: expat: Unpatched CVEs: CVE-2013-0340
WARNING: readline: Unpatched CVEs: CVE-2014-2524
WARNING: qemu: Unpatched CVEs: CVE-2015-5279
WARNING: readline: Unpatched CVEs: CVE-2014-2524

Doubtless there's recipes missed due to PN mismatch (I'm thinking gcc) and false-positives from bad patch names (libarchive is fixed in my local branch, should be reporting there) but this is an interesting start.  I don't see any problems with using something like this on release branches, as long as CVE patches have the CVE ID in the filename.

I guess an important question is do we a) endorse meta-security-isafw or b) split out the bits we like into something more focused in oe-core.
Comment 13 Armin Kuster 2015-11-03 20:07:52 UTC
(In reply to comment #12)
> As an experiment into using cve-check-tool without the isafw surrounding it
> I wrote a ten-line python class and added it to meta-ross
> (github:rossburton/meta-ross).  With current jethro against core-image-sato
> I get this:
> 
> WARNING: iptables: Unpatched CVEs: CVE-2012-2663
> WARNING: readline: Unpatched CVEs: CVE-2014-2524
> WARNING: expat: Unpatched CVEs: CVE-2013-0340
> WARNING: subversion: Unpatched CVEs: CVE-2015-3184 CVE-2015-3187
> WARNING: vte: Unpatched CVEs: CVE-2012-2738
> WARNING: iptables: Unpatched CVEs: CVE-2012-2663
> WARNING: unzip: Unpatched CVEs: CVE-2014-9636
> WARNING: expat: Unpatched CVEs: CVE-2013-0340
> WARNING: expat: Unpatched CVEs: CVE-2013-0340
> WARNING: readline: Unpatched CVEs: CVE-2014-2524
> WARNING: qemu: Unpatched CVEs: CVE-2015-5279
> WARNING: readline: Unpatched CVEs: CVE-2014-2524

Does it queue the kernel?

its missing the libxml2 and libxslt.

(patches in queue for libxslt)

> 
> Doubtless there's recipes missed due to PN mismatch (I'm thinking gcc) and
> false-positives from bad patch names (libarchive is fixed in my local
> branch, should be reporting there) but this is an interesting start.  I
> don't see any problems with using something like this on release branches,

Ask Fray.. he had concerns.

> as long as CVE patches have the CVE ID in the filename.

kernel wont.

> 
> I guess an important question is do we a) endorse meta-security-isafw or b)
> split out the bits we like into something more focused in oe-core.

why would we want another security layer? there are 4 currently
Comment 14 Mihail Stanciu 2015-12-10 13:51:57 UTC
(In reply to comment #3)
> Lucian,
> Maybe we can dry run this before it's implemented by Ross.

I've added a buildstep on our local AB that runs the cve-check-tool against a nightly-world build and sends and email with the results.
For now it sends the e-mail just to me until I'm sure everything is working properly.
Comment 15 Mariano Lopez 2015-12-15 16:08:02 UTC
I started a RFC thread in OE-core, OE-devel, so we get more input. Feel free to comment.

http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114166.html

http://lists.openembedded.org/pipermail/openembedded-core/2015-December/114166.html
Comment 16 Mariano Lopez 2015-12-29 20:24:02 UTC
Will wait after the holidays to reach a conclusion
Comment 17 Ross Burton 2016-01-04 11:28:33 UTC
WaitForUpstream is only valid if there's an upstream and generally means a bug disappears forever.  Changing back to accepted now the holiday is over. :)
Comment 18 Mariano Lopez 2016-02-19 22:38:43 UTC
I have started to implement this class using cve-check-tool base on Ross' class
Comment 19 Armin Kuster 2016-02-19 23:17:47 UTC
(In reply to comment #18)
> I have started to implement this class using cve-check-tool base on Ross'
> class

Please answer my questions in commit #5.

in addition. where will this class reside?
Comment 20 Mariano Lopez 2016-02-25 17:20:42 UTC
The idea is just to provide the tool, what people do with is up to them.

I don't think we are going to expose the vulnerabilities more than they already exposed. For example, you can download the layer that Ross mentioned and get what CVEs haven't been patched.

The idea is to include the tool into oe-core
Comment 21 Armin Kuster 2016-02-25 21:51:30 UTC
(In reply to comment #20)
> The idea is just to provide the tool, what people do with is up to them.
> 
> I don't think we are going to expose the vulnerabilities more than they
> already exposed. For example, you can download the layer that Ross mentioned
> and get what CVEs haven't been patched.
> 
> The idea is to include the tool into oe-core

thanks for the reply
Comment 22 Mariano Lopez 2016-02-29 14:43:45 UTC
Sent patches to the list:

http://lists.openembedded.org/pipermail/openembedded-core/2016-February/118117.html
Comment 23 Mariano Lopez 2016-03-08 22:30:19 UTC
Moving to 2.2, this enhancement didn't make it to 2.1
Comment 24 Ross Burton 2016-09-19 12:41:37 UTC
Merged in oe-core d98338075ec3a66acb8828e74711550d53b4d91b.