Bug 15660

Summary: [kirkstone, scarthgap] AB-INT: cve-update-nvd2-native-1.0-r0 do_fetch: CVE database update failed
Product: [QA/Testing] Functional (self) Testing Reporter: Mathieu Dubois-Briand <mathieu.dubois-briand>
Component: generalAssignee: Yoann Congal <yoann.congal>
Status: NEW --- QA Contact:
Severity: normal    
Priority: Medium+ CC: paul, randy.macleod, richard.purdie, yoann.congal
Version: unspecified   
Target Milestone: 5.0.22   
Hardware: x86   
OS: Multiple   
See Also: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15560
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Mathieu Dubois-Briand 2024-11-25 16:41:00 UTC
Seems to be a new AB-INT, we can see it a lot in the metrics builder on the last days. Still, it seems to be working correctly on some builds. Impacts about 75% of the builds of the last 3 days.

Only this warning appears in the log:

WARNING: cve-update-nvd2-native-1.0-r0 do_fetch: CVE database update failed
Comment 1 Mathieu Dubois-Briand 2024-11-25 17:51:03 UTC
metrics alma8-vk-2 dunfell completed at 2024-11-22T06:21:11Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/238/steps/14/logs/stdio

metrics alma9-vk-1 kirkstone completed at 2024-11-22T06:23:34Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/239/steps/13/logs/stdio

metrics debian12-vk-3 scarthgap completed at 2024-11-22T06:41:28Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/240/steps/15/logs/stdio

metrics debian11-vk-1 styhead completed at 2024-11-22T06:52:56Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/241/steps/15/logs/stdio

metrics stream9-vk-1 master completed at 2024-11-22T07:17:49Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/242/steps/15/logs/stdio

metrics debian11-vk-2 dunfell completed at 2024-11-23T07:03:19Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/243/steps/13/logs/stdio

metrics alma8-vk-2 kirkstone completed at 2024-11-23T06:23:54Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/244/steps/14/logs/stdio

metrics debian12-vk-5 scarthgap completed at 2024-11-23T06:41:54Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/245/steps/15/logs/stdio

metrics rocky9-vk-3 styhead completed at 2024-11-23T06:55:13Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/246/steps/15/logs/stdio

metrics debian12-vk-3 master completed at 2024-11-23T07:12:12Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/247/steps/15/logs/stdio

metrics alma8-vk-2 dunfell completed at 2024-11-24T06:45:37Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/248/steps/14/logs/stdio

metrics debian12-vk-2 styhead completed at 2024-11-24T07:14:38Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/251/steps/15/logs/stdio

metrics opensuse155-vk-2 master completed at 2024-11-24T07:24:57Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/252/steps/16/logs/stdio

metrics debian11-vk-3 dunfell completed at 2024-11-25T07:44:19Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/253/steps/13/logs/stdio

metrics debian12-vk-4 styhead completed at 2024-11-25T06:52:09Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/256/steps/15/logs/stdio

metrics opensuse155-vk-2 master completed at 2024-11-25T07:18:26Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/257/steps/16/logs/stdio
Comment 2 Mathieu Dubois-Briand 2024-11-26 14:08:13 UTC
metrics alma8-vk-2 dunfell completed at 2024-11-26T06:35:07Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/258/steps/14/logs/stdio

metrics debian12-vk-4 styhead completed at 2024-11-26T06:58:03Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/261/steps/15/logs/stdio

metrics alma8-vk-2 master completed at 2024-11-26T07:16:30Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/262/steps/16/logs/stdio
Comment 3 Mathieu Dubois-Briand 2024-11-27 09:40:41 UTC
metrics debian11-vk-1 dunfell completed at 2024-11-27T07:25:35Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/263/steps/13/logs/stdio

metrics alma8-vk-2 master completed at 2024-11-27T07:13:52Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/267/steps/16/logs/stdio
Comment 4 Mathieu Dubois-Briand 2024-11-28 15:05:22 UTC
metrics debian11-vk-1 dunfell completed at 2024-11-28T07:45:14Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/268/steps/13/logs/stdio

metrics alma9-vk-1 kirkstone completed at 2024-11-28T06:21:07Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/269/steps/13/logs/stdio

metrics debian11-vk-3 scarthgap completed at 2024-11-28T07:09:02Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/270/steps/15/logs/stdio

metrics alma8-vk-2 styhead completed at 2024-11-28T07:11:12Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/271/steps/16/logs/stdio
Comment 5 Randy MacLeod 2024-11-28 15:41:23 UTC
We need a new strategy for how we deal with NVD data.
Comment 6 Yoann Congal 2024-11-28 15:54:17 UTC
Recent mail on OE-Core ML : https://lists.openembedded.org/g/openembedded-core/topic/109438548#msg207851

With the idea to follow buildroot in switching to 
https://github.com/fkie-cad/nvd-json-data-feeds (which use the NVD API to push files (in the legacy format) to a git repo)
Comment 7 Mathieu Dubois-Briand 2024-12-03 15:21:14 UTC
metrics debian11-vk-1 dunfell completed at 2024-11-29T07:33:54Z
https://valkyrie.yoctoproject.org/#/builders/103/builds/273/steps/13/logs/stdio
Comment 8 Richard Purdie 2024-12-19 16:01:23 UTC
Not being seen frequently now so lowering priority
Comment 9 Randy MacLeod 2025-04-10 14:55:46 UTC
Added a work-around on the autobuilder and that seems to have made it go away...

The cause seems to have been NFS issues and regularily cutting the DB file down to a smaller size avoids the problem.
Comment 10 Mathieu Dubois-Briand 2025-11-27 09:01:11 UTC
oe-selftest-fedora alma9-vk-2 master completed at 2025-11-27 02:49:58+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/48/builds/2657/steps/15/logs/stdio
Comment 11 Mathieu Dubois-Briand 2026-06-18 10:32:34 UTC
metrics debian12-vk-3 2.0&kirkstone completed at 2026-06-18 07:17:06+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/103/builds/3521/steps/14/logs/stdio

metrics debian12-vk-8 scarthgap&2.8 completed at 2026-06-18 07:47:12+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/103/builds/3522/steps/16/logs/stdio
Comment 12 Mathieu Dubois-Briand 2026-06-18 10:36:22 UTC
Reopening.
Comment 13 Paul Barker 2026-06-18 15:13:52 UTC
@Yoann, please backport f63622bbec1c ("cve-update: Avoid NFS caching issues") to the affected branches.
Comment 14 Mathieu Dubois-Briand 2026-07-20 12:05:15 UTC
metrics opensuse156-vk-1 whinlatter&2.16 completed at 2026-07-20 06:52:06+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/103/builds/3715/steps/17/logs/stdio
Comment 15 Yoann Congal 2026-07-30 23:45:21 UTC
(In reply to Mathieu Dubois-Briand from comment #14)
> metrics opensuse156-vk-1 whinlatter&2.16 completed at 2026-07-20 06:52:06+00:00
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/103/builds/3715/steps/17/logs/stdio

While whinlatter has the fixes, the hypothesis is that another metric running branch corrupted the database in DL_DIR:

See related: https://bugzilla.yoctoproject.org/show_bug.cgi?id=15560
Comment 16 Mathieu Dubois-Briand 2026-10-12 08:57:42 UTC
metrics alma8-vk-1 2.16&whinlatter completed at 2026-10-04 06:52:54+00:00
https://autobuilder.yoctoproject.org/valkyrie/#/builders/103/builds/4184/steps/19/logs/stdio