Bug 2329

Summary: Forwarding only sometimes works from qemu connman-based images
Product: [Runtime] Installation Reporter: Paul Eggleton <bluelightning>
Component: InstallationAssignee: Scott Garman <scott.a.garman>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: Medium CC: dexuan.cui, kevin.tian, scott.a.garman, sgw
Version: 1.2   
Target Milestone: 1.2.1   
Hardware: All   
OS: Multiple   
Whiteboard: Under patch review on oe-core ML
OS type for building Yocto: --- Type of Regression: ---
Verified: Documentation change: ---

Description Paul Eggleton 2012-04-18 15:25:04 UTC
Sometimes on a core-image-sato qemu image (which uses connman) I cannot ping/connect to any network address other than the host 192.168.7.1, thus forwarding seems to be broken sometimes. core-image-minimal does not seem to be affected, nor does core-image-sato on atom-pc.

I've reproduced this intermittently on both a qemumips, qemux86-64 and qemux86 images, however it seems more likely to occur on the qemumips image (which incidentally I am running remotely over ssh from another machine, thus it is slower). The behaviour leads me to believe it is a timing issue. Possibly unrelated, but it does seem that connman behaves slightly differently on the emulated network interface which is set up from boot as opposed to a real ethernet adapter on atom-pc - e.g. within qemu the connman applet doesn't show the network as connected until quite a while (up to 30s) after it starts, whereas on atom-pc it immediately shows as connected.
Comment 1 Saul Wold 2012-04-19 23:06:37 UTC
Is this tap/tun or something else, please update
Comment 2 Paul Eggleton 2012-04-19 23:28:00 UTC
This would be via tun/tap yes, when running an ext3 image via runqemu. My first plan of attack was going to be to compare the outputs of "route -n" when working and not working; I haven't got to that yet.
Comment 3 Dexuan Cui 2012-04-23 11:14:57 UTC
(In reply to comment #0)
> Sometimes on a core-image-sato qemu image (which uses connman) I cannot
> ping/connect to any network address other than the host 192.168.7.1, thus
> forwarding seems to be broken sometimes. core-image-minimal does not seem to be
> affected, nor does core-image-sato on atom-pc.
> 
> I've reproduced this intermittently on both a qemumips, qemux86-64 and qemux86
> images, however it seems more likely to occur on the qemumips image (which
> incidentally I am running remotely over ssh from another machine, thus it is
> slower). The behaviour leads me to believe it is a timing issue. Possibly
> unrelated, but it does seem that connman behaves slightly differently on the
> emulated network interface which is set up from boot as opposed to a real
> ethernet adapter on atom-pc - e.g. within qemu the connman applet doesn't show
> the network as connected until quite a while (up to 30s) after it starts,
> whereas on atom-pc it immediately shows as connected.
Hi Paul,
When you see the issue, can you please run
"echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp" (change tap0 to the actual one if it's not tap0) in the host with root privilege, and please use IP address directly to exclude any possible DNS issue?

I recently noticed the similar issue (I suppose it's similar) -- I can always reproduce the issue on qemux86 and I work it around by enabling the proxy_arp and cat my host's resolv.conf into the qemu guest's one.

I remember vaguely long ago I didn't have such an issue.
Comment 4 Dexuan Cui 2012-04-27 00:06:16 UTC
(In reply to comment #3)
> (In reply to comment #0)
> > Sometimes on a core-image-sato qemu image (which uses connman) I cannot
> > ping/connect to any network address other than the host 192.168.7.1, thus
> > forwarding seems to be broken sometimes. core-image-minimal does not seem to be
> > affected, nor does core-image-sato on atom-pc.
> > 
> > I've reproduced this intermittently on both a qemumips, qemux86-64 and qemux86
> > images, however it seems more likely to occur on the qemumips image (which
> > incidentally I am running remotely over ssh from another machine, thus it is
> > slower). The behaviour leads me to believe it is a timing issue. Possibly
> > unrelated, but it does seem that connman behaves slightly differently on the
> > emulated network interface which is set up from boot as opposed to a real
> > ethernet adapter on atom-pc - e.g. within qemu the connman applet doesn't show
> > the network as connected until quite a while (up to 30s) after it starts,
> > whereas on atom-pc it immediately shows as connected.
> Hi Paul,
> When you see the issue, can you please run
> "echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp" (change tap0 to the actual
> one if it's not tap0) in the host with root privilege, and please use IP
> address directly to exclude any possible DNS issue?
> 
> I recently noticed the similar issue (I suppose it's similar) -- I can always
> reproduce the issue on qemux86 and I work it around by enabling the proxy_arp
> and cat my host's resolv.conf into the qemu guest's one.
> 
> I remember vaguely long ago I didn't have such an issue.


I suppose the patch in poky master has fixed this bug?
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=0f95925b4cdc39a40969f7e496c3924fa91f61c6
Comment 5 Scott Garman 2012-04-27 02:07:04 UTC
No, that patch only fixed the issue in core-image-minimal, which doesn't use conman. What I found is that external network access works for about the first 30 seconds or so after a -sato image boots up, then external network access stops working. So I suspect that there is something conman is doing to change things - though I have yet to confirm this.
Comment 6 Dexuan Cui 2012-04-27 02:19:41 UTC
(In reply to comment #5)
> No, that patch only fixed the issue in core-image-minimal, which doesn't use
> conman. What I found is that external network access works for about the first
> 30 seconds or so after a -sato image boots up, then external network access
> stops working. So I suspect that there is something conman is doing to change
> things - though I have yet to confirm this.
Though I'm not familiar with iptables and the tap device, with the following patch aplied, I do make the network in qemu sato image work . :-)
BTW, however the DNS resolver in qemu sato still doesn't work as expected -- I have to use IP address directly, or scp my host's /etc/resolv.conf into the qemu target.
Maybe you can try this when you get the issue with connman in qemu sato.
Hope this can be a clue to the final fix.

diff --git a/scripts/runqemu-ifup b/scripts/runqemu-ifup
index f80538f..1f1b4c8 100755
--- a/scripts/runqemu-ifup
+++ b/scripts/runqemu-ifup
@@ -103,6 +103,7 @@ $ROUTE add -host 192.168.7.$dest $TAP
 $IPTABLES -A POSTROUTING -t nat -j MASQUERADE -s 192.168.7.$n/32
 $IPTABLES -A POSTROUTING -t nat -j MASQUERADE -s 192.168.7.$dest/32
 echo 1 > /proc/sys/net/ipv4/ip_forward
+echo 1 > /proc/sys/net/ipv4/conf/$TAP/proxy_arp
 $IPTABLES -P FORWARD ACCEPT

 echo $TAP
Comment 7 Scott Garman 2012-04-27 02:44:37 UTC
Thanks! I'll check that out.