Bug 12559

Summary: buildhistory-diff: directory rename detection sometimes goes crazy
Product: [Build System, Metadata & Runtime] OE-Core Reporter: Ross Burton <ross.burton>
Component: Scripts and ToolsAssignee: Anuj Mittal <anuj.mittal>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium+ CC: apoorv.sangal, ross.burton, stephano
Version: 2.5   
Target Milestone: 2.5 M4   
Hardware: x86   
OS: Multiple   
Whiteboard:
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: No (bug/feature does not impact docs)

Description Ross Burton 2018-02-19 23:54:25 UTC 
Sometimes and not always, and I'm yet to spot the pattern, the directory rename detection goes wrong:

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /usr/src/debug/nativesdk-qemu/2.11.1-r0/build/hw/misc

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /usr/src/debug/nativesdk-qemu/2.11.1-r0/build/hw/i386

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /
usr/src/debug/nativesdk-qemu/2.11.1-r0/build/audio

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /usr/src/debug/nativesdk-qemu/2.11.1-r0/build/hw/char

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /usr/src/debug/nativesdk-qemu/2.11.1-r0/build/util

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /usr/src/debug/nativesdk-qemu/2.11.1-r0/build/net

directory renamed /usr/src/debug/nativesdk-qemu/2.11.0-r0/build/accel/kvm -> /usr/src/debug/nativesdk-qemu/2.11.1-r0/build/chardev

Repeat this pattern for another hundred directories as the target, and then again for every source directory, and you end up with hundreds of pages of garbage.
Comment 1 Anuj Mittal 2018-02-23 07:07:34 UTC 
I was able to reproduce this by updating qemu to 2.11.1 from 2.11.0. So this will happen if there are files with same names in different folders if a package is upgraded.

In case of qemu, there're a lot of trace.c/h files in the folders listed here. Directory rename logic thinks that if file names are same and directory is different, it's a directory rename and prints it.
Comment 2 Ross Burton 2018-02-23 09:48:45 UTC 
I can see how it's getting confused, but it definitely needs to be trying harder to find the minimal change (numbers change only would deal with every -dbg rename) and more importantly only report a rename pair once.
Comment 3 Anuj Mittal 2018-03-01 09:32:26 UTC 
Since this is all using metadata only, using file names is probably the only way here to check if two directories are same. But, we should be able to skip the dirs that have been detected as a rename once. If the entire list is sorted, the first hit is actually the closest ...

Right now, the entire list is dumped twice and in a nested for loop, that means 2*n (n=number of dirs) changes displayed for every dir even when a rename has already been processed. I will send a change after doing some more testing.