Bug 15144

Summary: AB-INT: cve-update-db-native do_fetch failure, error 503
Product: [Infrastructure] AutoBuilder Reporter: Alexandre Belloni <alexandre.belloni>
Component: autobuilderAssignee: Steve Sakoman <steve>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: alexandre.belloni, infras.ab.watcher, Infras.watcher, pidge, randy.macleod, tomas.volka
Version: unspecified   
Target Milestone: 4.3 M2   
Hardware: x86   
OS: Multiple   
Whiteboard: AB-INT
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Alexandre Belloni 2023-06-14 08:22:35 UTC 
WARNING: cve-update-db-native-1.0-r0 do_fetch: Failed to fetch CVE data (HTTP Error 503: Service Unavailable)
WARNING: cve-update-db-native-1.0-r0 do_fetch: Host IPs are 2600:1f18:268d:1d01:f609:5e91:8a48:f546, 18.235.227.114

https://autobuilder.yoctoproject.org/typhoon/#/builders/138/builds/425/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/138/builds/423/steps/11/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/138/builds/417/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/138/builds/411/steps/11/logs/stdio
Comment 1 Tomas Volka 2023-07-11 19:58:27 UTC 
We've been fighting with this too. With cve-check enabled in the build we've been hitting:

[18:20:45]W:	 [Step 1/1] ERROR: cve-update-nvd2-native-1.0-r0 do_fetch: Error executing a python function in exec_func_python() autogenerated:
...
[18:20:45]W:	 [Step 1/1] Exception: urllib.error.HTTPError: HTTP Error 503: Service Unavailable

The issue here is that there's only one remote URL defined in the recipe: https://services.nvd.nist.gov/rest/json/cves/2.0

Sometimes it spits out 503 - but no wonder, it's serving paginated 4MB JSONs.

And while it works on a good day, it's been broken lately and we had to disable cve-check functionality altogether.

That being said, i conclude that the service is currently not reliable enough. The python script needs either to retry and wait or it should use some other source for the CVEs.