Bug 4394

Summary: Intel Data Plane Development Kit (DPDK) support
Product: [Build System, Metadata & Runtime] BSPs Reporter: Terence Chiang <terence.chiang>
Component: bsps-meta-intelAssignee: Wei Sern Chan <wei.sern.chan>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: Medium CC: boon.leong.ong, dvhart, gshastri, jessica.zhang, rebecca.swee.fun.chang, ross.burton, tom.zanussi, wei.sern.chan
Version: 1.5   
Target Milestone: 1.7   
Hardware: x86   
OS: x86   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: Don't know
Attachments:
Description Flags
dpdk-firstdraft-recipe none

Description Terence Chiang 2013-04-23 08:24:33 UTC
Packet processing on the latest Intel processors is now an increasingly viable option due to continued improvements in multi-core architectures combined with the latest packet processing software enhancements provided by the Intel® Data Plane Development Kit (Intel® DPDK). Huge performance boosts achieved by this hardware/software combination is making Intel® architecture increasingly attractive as a packet processing solution. Additionally, by consolidating packet processing with other workloads on a Intel® multi-core processor, it is possible to reduce hardware costs, simplify the application development environment, and reduce time to market—with all the factors combining to reduce overall total cost of ownership. There is a demanding request for DPDK support in Yocto Project Intel BSP release. With that support, it will be easier for system integrator and device maker to create a POC for network devices.
for more information: http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/packet-processing-is-enhanced-with-software-from-intel-dpdk.html
Comment 1 Tom Zanussi 2013-04-23 13:30:22 UTC
DPDK recipes are already available from ISG - see the romley and crystalforest BSPs READMEs:

III. Enabling Intel DPDK Technology
===================================

There are recipes available for the 'Intel Data Plane Development Kit
Technology components for use with this BSP.  Please contact your
Intel Field Sales Representative or visit Intel Embedded Design
Center (www.intel.com/p/en_US/embedded/designcenter) for instructions on how
to locate and use these recipes.

Also, WindRiver was demo'ing DPDK at the Yocto booth at ELC 2013.

So, the support is already implemented somewhere, perhaps we can leverage it.
Comment 2 Tom Zanussi 2013-04-23 14:45:16 UTC
Though there doesn't seem to be any obvious location to find the Yocto DPDK support at the www.intel.com/p/en_US/embedded/designcenter link - I guess you really do have to contact your Intel Field Sales representative to get it.
Comment 3 Terence Chiang 2013-04-23 15:00:49 UTC
Intel DPDK download link is http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/dpdk-source-code.html 
After unzip that, there is no YP recipes but source only. I will check with field team on this.
Comment 4 kishore 2013-04-23 17:26:59 UTC
Terence,

There is no location for the DPDK Yocto recipes.
The location mentioned here is only for the DPDK sources.
You need to contact Lynch, Michael A to get the Yocto DPDK recipes.

Tom,

What should be the status for these kind of bug?
Since the recipes are not yet public and is only under NDA, can I make this bug status as resolved?

Thanks
Kishore.
Comment 5 Tom Zanussi 2013-04-23 18:13:32 UTC
(In reply to comment #4)
> Terence,
> 
> There is no location for the DPDK Yocto recipes.
> The location mentioned here is only for the DPDK sources.
> You need to contact Lynch, Michael A to get the Yocto DPDK recipes.
> 
> Tom,
> 
> What should be the status for these kind of bug?
> Since the recipes are not yet public and is only under NDA, can I make this
> bug status as resolved?
> 

I'm not sure I understand why the recipes would be under NDA if the DPDK itself is publicly available.
Comment 6 kishore 2013-04-23 18:16:17 UTC
The latest DPDK is 1.3 which is not public yet and so the Yocto recipes.
Comment 7 Tom Zanussi 2013-04-23 18:30:17 UTC
(In reply to comment #6)
> The latest DPDK is 1.3 which is not public yet and so the Yocto recipes.

The current romley and crystalforest BSPs explicitly state that there is Yocto support for the DPDK, and that the recipes are available if you 'visit Intel Embedded Design Center (www.intel.com/p/en_US/embedded/designcenter) for instructions on how to locate and use these recipes.'.  So Terence or anyone else should be able to do just that, but there doesn't seem to be enough information to do that, so at minimum please supply that information.  It would also make sense to add that more detailed information to the BSP READMEs.

As for the recipes for the new 1.3 version that you mention, I guess it would be up to Terence as the one requesting the feature to decide what to do about it i.e. is the currently-released DPDK enough for his needs.  If so, he might want to close this as resolved, or if not, he might want to wait for the 1.3 support from ISG, or get it on the schedule for someone else to implement, or get it from WindRiver, if they have what he needs.
Comment 8 Terence Chiang 2013-04-24 07:29:41 UTC
Before I closed this bug. Let me check with Lynch, Michael A on the release cretira whether it is under NDA or public avaialbe. We need make it clear in the README.
For DPDK 1.3 release, i can understand it might be under CNDA.
But for the DPDK 1.2.3 (http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/dpdk-source-code.html) which is already public avaiable in source code format but the YP recipes is not avaialbe.
I will provide my feedback after checking with Michael. 
Having said that we might still need to have DPDK 1.3 release in YP recipes format for YP 1.5 release as a feature. I will work with Michael on the release format (CNDA or other license).

thanks.
Comment 9 Darren Hart 2013-04-25 14:39:30 UTC
Just categorizing to get it off the bug triage list, leaving it to Terence to close per his final comment.
Comment 10 Tom Zanussi 2013-05-03 13:04:34 UTC
So the current status of this is that Terence is OK waiting for 1.3 support, which will become available when 1.4 releases, which will not become publicly available until 1.5 releases.

There actually is currently no 1.2 recipe so the BSP READMEs are simply wrong and misleading to users, and 1.3 isn't expected tomorrow or even next week, so we need to remove that text from the READMEs until there is something available.  Kishore, can you please submit a patch for that?
 
The DPDK group uses a previous-to-current methodology for releasing, which the Yocto recipes should have no problem accommodating - we can keep the last several iterations of the DPDK recipes available like we do for the kernel, and users can select them using PREFERRED_VERSION.

We can also make the recipe for the currently closed version available (1.4 in the current scenario) since the recipe itself isn't under NDA; the recipe can define a LICENSE_FLAGS string that if not set will simply skip the recipe as fif it wasn't available.  The customer can then download the NDA binaries and enable the recipe locally using the LICENSE_FLAGS_WHITELIST setting.

We should also have a validation framework in place for each release, and the Yocto team should also have at least one hardware platform we can independently test and verify.  Currently we have jasperforest - will that be good enough for testing, or should we order new hardware?

Also, Terence, in addition to having a validation framework, it would also be very helpful to have as many details as possible as to how the customers are actually using the DPDK - it would be nice to be able to have a setup that we could demonstrate similar use-cases, for both testing and demo purposes.
Comment 11 Terence Chiang 2013-05-06 06:33:12 UTC
(In reply to comment #10)
We will have a meeting with DPDK team to understand DPDK release plan and its scheule. So for the README, we need to update that information.
DPDK is released with sample applications that OxM use those applications for valdation and testing performance.

> Also, Terence, in addition to having a validation framework, it would also
> be very helpful to have as many details as possible as to how the customers
> are actually using the DPDK - it would be nice to be able to have a setup
> that we could demonstrate similar use-cases, for both testing and demo
> purposes.
Comment 12 kishore 2013-05-14 17:45:09 UTC
CY and his team are having discussions with the DPDK team.

Since Boon Leong is leading this effort, I will let him decide and update the README to reflect the outcome of the meeting.


Thanks
Kishore.
Comment 13 Ong Boon Leong 2014-04-08 15:45:28 UTC
Following 2014/03 discussion with Intel DPDK team, Intel is contributing its Intel DPDK work into git://dpdk.org/dpdk. ISG target to create recipe to support move in YP1.7 time frame.

So, I am moving this to YP1.7.
Comment 14 Ross Burton 2014-05-22 16:13:00 UTC
Taking this out of NEEDINFO as Boon seems to have this under control.
Comment 15 Gaurang Shastri 2014-05-23 08:49:42 UTC
Created attachment 1974 [details]
dpdk-firstdraft-recipe
Comment 16 Gaurang Shastri 2014-05-23 08:51:19 UTC
Hi,

I have created first draft recipe and verified in my environment.

It is working fine and giving me desired output.

Please find it attached here (https://bugzilla.yoctoproject.org/attachment.cgi?id=1974)

Let me know if it is good to go in main line.
Comment 17 Ross Burton 2014-05-23 08:55:19 UTC
Please can you send that as a patch to the meta-intel list?
Comment 18 Gaurang Shastri 2014-05-23 09:28:44 UTC
Dear Ross,

(In reply to comment #17)
> Please can you send that as a patch to the meta-intel list?

Done the same as a patch format and sent to meta-intel mailing list.

Thank you.
Comment 19 Ong Boon Leong 2014-05-29 02:46:16 UTC
Assigned to Wei Sern for review. He will start the review process next week
Comment 20 Wei Sern Chan 2014-06-27 01:57:09 UTC
Hi Gaurang Shastri,

Reviewed meta-intel mailing list, you sent the patch file as attachment to  meta-intel@yoctoproject.org. Then Tom asked you to re-send your patch in text format. Can you re-send those patch in text format ?

By the way, since Intel DPDK development team is contributing to git://dpdk.org/dpdk. So ISG team decided to create Yocto recipe and which is targeting on DPDK 1.6.0 and finally upstream it to meta-intel. 

We are almost there so hopefully you can see the recipe creation next week or so.

Since then we are doing the similiar stuff, if you're comfortable with your schedule, I would suggest is better to wait for our recipe landed on meta-intel git and then you can do upgrade/enhance from there.
Comment 21 Ross Burton 2014-07-22 12:53:11 UTC
Merged into meta-intel as of c249763d3cb92bd06742b7986f08cf844ddb1d51.