Bug 8448 - Audio via HDMI doesn't work
Summary: Audio via HDMI doesn't work
Status: VERIFIED FIXED
Alias: None
Product: OE-Core
Classification: Build System, Metadata & Runtime
Component: multimedia (show other bugs)
Version: 2.0
Hardware: NUC x86_64
: Medium normal
Target Milestone: 2.1
Assignee: Jussi Kukkonen
QA Contact: Cristina Agurida
URL:
Whiteboard:
Depends on:
Blocks:
 
Reported: 2015-10-06 11:17 UTC by Cristina Agurida
Modified: 2015-12-15 13:26 UTC (History)
7 users (show)

See Also:
OS type for building Yocto: ---
Type of Regression: New (Never tested)
Verified:
Documentation change: No (bug/feature does not impact docs)


Attachments
"pactl list" output with HDMI connected (15.32 KB, text/plain)
2015-10-13 07:03 UTC, Jussi Kukkonen
no flags Details
"pactl list" output with no HDMI connected (15.31 KB, text/plain)
2015-10-13 07:04 UTC, Jussi Kukkonen
no flags Details
pa log (195.28 KB, text/plain)
2015-10-20 12:51 UTC, Jussi Kukkonen
no flags Details
[PATCH 1/4] card: add pa_card_profile.ports (8.12 KB, patch)
2015-10-23 11:26 UTC, Tanu Kaskinen
no flags Details | Diff
[PATCH 2/4] alsa, bluetooth: fail if user-requested profile doesn't exist (2.12 KB, patch)
2015-10-23 11:27 UTC, Tanu Kaskinen
no flags Details | Diff
[PATCH 3/4] card: move profile selection after pa_card_new() (13.75 KB, patch)
2015-10-23 11:27 UTC, Tanu Kaskinen
no flags Details | Diff
[PATCH 4/4] alsa: set availability for (some) unavailable profiles (2.65 KB, patch)
2015-10-23 11:28 UTC, Tanu Kaskinen
no flags Details | Diff
active_profile init fix (as last patch) (718 bytes, patch)
2015-10-26 14:48 UTC, Jussi Kukkonen
no flags Details | Diff
clear port/profile hashmaps on free (1.63 KB, patch)
2015-11-23 12:30 UTC, Jussi Kukkonen
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Cristina Agurida 2015-10-06 11:17:17 UTC
BUILD: 2.0: eac61f37e36099f74485dab398b57f3812826d17
ENVIRONMENT: genericx86-64 on NUC
IMAGE: core-image-sato-sdk

STEPS TO REPRODUCE: 

1. Successfully install a core-image-sato-sdk image.
2. Copy an audio file(ogg/wav) to system.
2. Connect system with a HDMI monitor.
3. Launch media player and play the audio file.

EXPECTED OUTCOME: 

The audio file can be played without problems with HDMI.

Actual outcome:

There is no sound in the headphones.

I tried the sound on both D54250WYK and NUC5i5RYH NUC models using LG FLATRON W2261VT Monitor.
Comment 1 Cristina Agurida 2015-10-08 15:35:04 UTC
On MinnowMax with latest firmware the sound works, we should make more investigations on this issue also on NUC.
Comment 2 Cristina Agurida 2015-10-09 14:14:34 UTC
I tried the sound via HDMI with NUC5i5RYH NUC model and Ubuntu 14.04 on 64 bits and it works, but with core-image-sato-sdk image for genericx86-64 on the same hardware the sound doesn't work.
Comment 3 Jussi Kukkonen 2015-10-12 12:22:09 UTC
I've taken a look at this on my (different model) NUC and I believe I'm seeing the same thing: playback does not work out of the box through HDMI.

The problem is that there are 3 devices on the NUC:
* card 0, device 3 (HDMI 0)
* card 0, device 7 (HDMI 1)
* card 1, device 0 (headphones)



Pulseaudio is able to detect the headphone plug events (I checked) but it doesn't seem to be able to figure out that "HDMI 0" does not work, only "HDMI 1" does.

Playback works fine if I force the correct device:
    aplay -D plughw:0,7 Front_Center.wav

or alternatively in pulseaudio configuration:
    load-module module-alsa-sink device=hw:0,7
    set-default-sink 0


I'll continue looking into this.
Comment 4 Jussi Kukkonen 2015-10-12 13:34:55 UTC
Possibly related: there are no pulseaudio events when HDMI is unplugged/plugged.

Cristina: if you have that ubuntu image still easily available, could you check if the events are visible there:
 * run "pactl subscribe"
 * unplug and replug the HDMI
 * see if any events appeared
Comment 5 Jussi Kukkonen 2015-10-12 14:08:50 UTC
(In reply to comment #4)
> Possibly related: there are no pulseaudio events when HDMI is
> unplugged/plugged.

The corresponding udev events are there (or at least I can see some events in there when (un)plugging) so this is a bit suspicious.
Comment 6 Tanu Kaskinen 2015-10-12 17:01:33 UTC
Can you attach "pactl list" output for both cases, monitor plugged and unplugged?

What does "amixer -c0 controls | grep Jack" print?
Comment 7 Jussi Kukkonen 2015-10-13 07:03:24 UTC
Created attachment 2793 [details]
"pactl list" output with HDMI connected
Comment 8 Jussi Kukkonen 2015-10-13 07:04:23 UTC
Created attachment 2794 [details]
"pactl list" output with no HDMI connected
Comment 9 Jussi Kukkonen 2015-10-13 07:08:35 UTC
Hi Tanu, thanks for looking at this.

I've attached the full outputs, the diff is below. I don't have amixer on the image right now, will report on that later.

--- list-with-hdmi	2015-10-13 10:00:58.552118120 +0300
+++ list-without-hdmi	2015-10-13 10:00:58.556118033 +0300
@@ -168,7 +168,7 @@
 		module.version = "6.0"
 
 Sink #0
-	State: IDLE
+	State: SUSPENDED
 	Name: alsa_output.pci-0000_00_03.0.hdmi-stereo
 	Description: Built-in Audio Digital Stereo (HDMI)
 	Driver: module-alsa-card.c
@@ -180,7 +180,7 @@
 	        balance 0.00
 	Base Volume: 65536 / 100% / 0.00 dB
 	Monitor Source: alsa_output.pci-0000_00_03.0.hdmi-stereo.monitor
-	Latency: 352388 usec, configured 371519 usec
+	Latency: 0 usec, configured 0 usec
 	Flags: HARDWARE DECIBEL_VOLUME LATENCY SET_FORMATS 
 	Properties:
 		alsa.resolution_bits = "16"
@@ -220,7 +220,7 @@
 		pcm
 
 Sink #1
-	State: IDLE
+	State: SUSPENDED
 	Name: alsa_output.pci-0000_00_1b.0.analog-stereo
 	Description: Built-in Audio Analog Stereo
 	Driver: module-alsa-card.c
@@ -232,7 +232,7 @@
 	        balance 0.00
 	Base Volume: 65536 / 100% / 0.00 dB
 	Monitor Source: alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
-	Latency: 366691 usec, configured 371519 usec
+	Latency: 0 usec, configured 0 usec
 	Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 
 	Properties:
 		alsa.resolution_bits = "16"
@@ -272,7 +272,7 @@
 		pcm
 
 Source #0
-	State: IDLE
+	State: SUSPENDED
 	Name: alsa_output.pci-0000_00_03.0.hdmi-stereo.monitor
 	Description: Monitor of Built-in Audio Digital Stereo (HDMI)
 	Driver: module-alsa-card.c
@@ -284,7 +284,7 @@
 	        balance 0.00
 	Base Volume: 65536 / 100% / 0.00 dB
 	Monitor of Sink: alsa_output.pci-0000_00_03.0.hdmi-stereo
-	Latency: 0 usec, configured 371519 usec
+	Latency: 0 usec, configured 0 usec
 	Flags: DECIBEL_VOLUME LATENCY 
 	Properties:
 		device.description = "Monitor of Built-in Audio Digital Stereo (HDMI)"
@@ -305,7 +305,7 @@
 		pcm
 
 Source #1
-	State: IDLE
+	State: SUSPENDED
 	Name: alsa_output.pci-0000_00_1b.0.analog-stereo.monitor
 	Description: Monitor of Built-in Audio Analog Stereo
 	Driver: module-alsa-card.c
@@ -317,7 +317,7 @@
 	        balance 0.00
 	Base Volume: 65536 / 100% / 0.00 dB
 	Monitor of Sink: alsa_output.pci-0000_00_1b.0.analog-stereo
-	Latency: 0 usec, configured 371519 usec
+	Latency: 0 usec, configured 0 usec
 	Flags: DECIBEL_VOLUME LATENCY 
 	Properties:
 		device.description = "Monitor of Built-in Audio Analog Stereo"
@@ -338,7 +338,7 @@
 		pcm
 
 Source #2
-	State: IDLE
+	State: SUSPENDED
 	Name: alsa_input.pci-0000_00_1b.0.analog-stereo
 	Description: Built-in Audio Analog Stereo
 	Driver: module-alsa-card.c
@@ -350,7 +350,7 @@
 	        balance 0.00
 	Base Volume: 5206 /   8% / -66.00 dB
 	Monitor of Sink: n/a
-	Latency: 4342 usec, configured 371519 usec
+	Latency: 0 usec, configured 0 usec
 	Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 
 	Properties:
 		alsa.resolution_bits = "16"
@@ -389,14 +389,14 @@
 	Formats:
 		pcm
 
-Client #0
+Client #1
 	Driver: protocol-native.c
 	Owner Module: 8
 	Properties:
 		application.name = "pactl"
 		native-protocol.peer = "UNIX socket client"
 		native-protocol.version = "30"
-		application.process.id = "1037"
+		application.process.id = "1045"
 		application.process.user = "root"
 		application.process.host = "intel-corei7-64"
 		application.process.binary = "pactl"
Comment 10 Jussi Kukkonen 2015-10-13 08:12:28 UTC
# amixer -c0 controls|grep Jack
numid=1,iface=CARD,name='HDMI/DP,pcm=3 Jack'
numid=7,iface=CARD,name='HDMI/DP,pcm=7 Jack'
Comment 11 Tanu Kaskinen 2015-10-13 08:58:43 UTC
It looks like the kernel doesn't report the jack state properly - PulseAudio thinks that the monitor is plugged in all the time. You can verify that with this command:

amixer -c0 cget iface=CARD,name="HDMI/DP,pcm=7 Jack"

That will print something like this:

  ; type=BOOLEAN,access=r-------,values=1
  : values=on

When the monitor is unplugged, the second line should have "values=off", but I would guess that it remains "values=on".

PulseAudio is to blame too, though, and I don't think the kernel needs to be fixed to make this work. PulseAudio sees that there are two HDMI devices, and one of them is available and one is unavailable. PulseAudio chooses the unavailable one for some reason. The port availability doesn't seem to be properly translated into profile availability. The pactl list output shows that the first HDMI port is unavailable, but the first HDMI profile is available, and it's the profile selection that goes wrong. In this case the translation should be unambiguous, since each HDMI profile uses only one port. I'll see if I can find a fix.
Comment 12 Jussi Kukkonen 2015-10-13 13:03:33 UTC
(In reply to comment #11)
> It looks like the kernel doesn't report the jack state properly - PulseAudio
> thinks that the monitor is plugged in all the time. You can verify that with
> this command:
> 
> amixer -c0 cget iface=CARD,name="HDMI/DP,pcm=7 Jack"
> 
> That will print something like this:
> 
>   ; type=BOOLEAN,access=r-------,values=1
>   : values=on
> 
> When the monitor is unplugged, the second line should have "values=off", but
> I would guess that it remains "values=on".

Yep, "values=on" also when unplugged.
Comment 13 Jussi Kukkonen 2015-10-14 08:35:06 UTC
This _might_ be the kernel issue 

http://lists.freedesktop.org/archives/intel-gfx/2015-August/074040.html

That patchset is in linus' tree since last month.
Comment 14 Tanu Kaskinen 2015-10-16 13:15:29 UTC
Sorry, I haven't been working on this actively, but I mentioned this today in a PulseAudio developers' meeting, and David Henningsson pointed out that this patch set might fix this issue: http://thread.gmane.org/gmane.comp.audio.pulseaudio.general/23504

I'm not sure it's easy to get the patches from Gmane properly formatted. If that's an issue, let me know and I'll attach the patches here.

Jussi or Cristina, could you check if those patches fix this bug? I don't have suitable hardware to test this myself. What I'll do next is review the patches so that we can get them applied upstream.
Comment 15 Jussi Kukkonen 2015-10-16 14:01:17 UTC
(In reply to comment #14)
> Sorry, I haven't been working on this actively, but I mentioned this today
> in a PulseAudio developers' meeting, and David Henningsson pointed out that
> this patch set might fix this issue:
> http://thread.gmane.org/gmane.comp.audio.pulseaudio.general/23504
> 
> I'm not sure it's easy to get the patches from Gmane properly formatted. If
> that's an issue, let me know and I'll attach the patches here.
> 
> Jussi or Cristina, could you check if those patches fix this bug? I don't
> have suitable hardware to test this myself. What I'll do next is review the
> patches so that we can get them applied upstream.

I'll test them (maybe on monday though).
Comment 16 Jussi Kukkonen 2015-10-20 12:51:57 UTC
Created attachment 2808 [details]
pa log

> PulseAudio is to blame too, though, and I don't think the kernel needs to be
> fixed to make this work. PulseAudio sees that there are two HDMI devices,
> and one of them is available and one is unavailable. PulseAudio chooses the
> unavailable one for some reason. The port availability doesn't seem to be
> properly translated into profile availability. The pactl list output shows
> that the first HDMI port is unavailable, but the first HDMI profile is
> available, and it's the profile selection that goes wrong. In this case the
> translation should be unambiguous, since each HDMI profile uses only one
> port. I'll see if I can find a fix.

I believe this is indeed happening, see full log. I added some debug output to module-switch-on-port-available (search for "(jku)" in log): 
 - port_available_hook_callback() is called for both ports
   (but card is apparently not initialized at that point). 
 - When new_sink_source is called the hashmap only includes
   hdmi-output-0 so obviously it gets selected.
Comment 17 Jussi Kukkonen 2015-10-20 12:52:51 UTC
oh and that log is with Davids patchset: it didn't help.
Comment 18 Tanu Kaskinen 2015-10-20 14:08:21 UTC
Ok, thanks for testing. I'll write a different fix tomorrow.
Comment 19 Tanu Kaskinen 2015-10-23 11:24:42 UTC
The seemingly simple problem was more complicated to fix than expected... Anyway, I sent patches to upstream: http://thread.gmane.org/gmane.comp.audio.pulseaudio.general/24301

I'll attach the patches also here, rebased on 6.0. There will be only four patches, because three of the original patches are not strictly necessary.
Comment 20 Tanu Kaskinen 2015-10-23 11:26:25 UTC
Created attachment 2814 [details]
[PATCH 1/4] card: add pa_card_profile.ports
Comment 21 Tanu Kaskinen 2015-10-23 11:27:06 UTC
Created attachment 2815 [details]
[PATCH 2/4] alsa, bluetooth: fail if user-requested profile doesn't exist
Comment 22 Tanu Kaskinen 2015-10-23 11:27:59 UTC
Created attachment 2816 [details]
[PATCH 3/4] card: move profile selection after pa_card_new()
Comment 23 Tanu Kaskinen 2015-10-23 11:28:24 UTC
Created attachment 2817 [details]
[PATCH 4/4] alsa: set availability for (some) unavailable profiles
Comment 24 Tanu Kaskinen 2015-10-23 11:29:01 UTC
I hope you can test these patches again, as I can't do that.
Comment 25 Jussi Kukkonen 2015-10-23 11:43:12 UTC
(In reply to comment #24)
> I hope you can test these patches again, as I can't do that.

Of course, I'll let you know. Thanks for your effort.
Comment 26 Jussi Kukkonen 2015-10-26 14:48:32 UTC
Created attachment 2824 [details]
active_profile init fix (as last patch)

Tests with the patches showed some crashes that seemed to be because active profile was not initialized. I'm attaching a patch that I believe fixes that: with all five patches HDMI sound works out-of-the-box. Thanks for the implementation.

If plug in headphones, HDMI is still used by default though... that's probably not correct but I don't know if it's related to this issue in any way?

Anyway, I think we're past the chance of getting this into Yocto 2.0 (and this is not a blocker) so no immediate rush here.
Comment 27 Tanu Kaskinen 2015-10-26 15:34:16 UTC
(In reply to comment #26)
> Created attachment 2824 [details]
> active_profile init fix (as last patch)
> 
> Tests with the patches showed some crashes that seemed to be because active
> profile was not initialized. I'm attaching a patch that I believe fixes
> that: with all five patches HDMI sound works out-of-the-box. Thanks for the
> implementation.

Ok, good that you found the issue, and that the patches work otherwise! I didn't notice the bug myself, because I tested on upstream master branch, and there the pa_card struct is immediately initialized to all-zero on allocation.

> If plug in headphones, HDMI is still used by default though... that's
> probably not correct but I don't know if it's related to this issue in any
> way?

That's a separate bug. On this hardware analog and HDMI outputs are on different cards, and the routing policy in PulseAudio doesn't currently automatically change the default device from one card to another, and also doesn't prefer one device type over another. I agree that headphones should be preferred by default.

If you file another bug, I'll try to do something about it. It might turn out to be tricky, though.
Comment 28 Jussi Kukkonen 2015-11-23 12:30:15 UTC
Created attachment 2862 [details]
clear port/profile hashmaps on free

Tanu, can you take a look if this patch seems correct for fixing the dangling pointer issue that David pointed out on pulseaudio-discuss? (I'm not sure how to test that it works)

If I do end up sending these as a patch to mailing list, I'll squash my two patches into patch 1, but I kept it separate for now so it's easier to see what changed.
Comment 29 Jussi Kukkonen 2015-11-23 12:36:26 UTC
(In reply to comment #28)
> Created attachment 2862 [details]
> clear port/profile hashmaps on free
> 
> Tanu, can you take a look if this patch seems correct for fixing the
> dangling pointer issue that David pointed out on pulseaudio-discuss? (I'm
> not sure how to test that it works)

Basically, what I wasn't sure about was whether I should only remove the specific port/profile or all of them as in the patches.
Comment 30 Jussi Kukkonen 2015-11-23 12:41:30 UTC
(In reply to comment #28)
> Created attachment 2862 [details]
> clear port/profile hashmaps on free
> 
> Tanu, can you take a look if this patch seems correct for fixing the
> dangling pointer issue that David pointed out on pulseaudio-discuss? (I'm
> not sure how to test that it works)

Ok I just figured out how to test: I just need to let pa quit. So I also just saw the patches are not correct: please don't waste time yet.
Comment 31 Tanu Kaskinen 2015-11-23 13:51:42 UTC
The patch is almost right. Just replace

    PA_HASHMAP_FOREACH(port, c->ports, state)
        pa_hashmap_free (port->profiles);

with

    PA_HASHMAP_FOREACH(port, c->ports, state)
        pa_hashmap_remove(port->profiles, c->name);

in pa_card_profile_free(), and do the same thing in device_port_free().
Comment 32 Jussi Kukkonen 2015-11-23 14:22:40 UTC
(In reply to comment #31)
> The patch is almost right. Just replace
> 
>     PA_HASHMAP_FOREACH(port, c->ports, state)
>         pa_hashmap_free (port->profiles);
> 
> with
> 
>     PA_HASHMAP_FOREACH(port, c->ports, state)
>         pa_hashmap_remove(port->profiles, c->name);
> 
> in pa_card_profile_free(), and do the same thing in device_port_free().

Cheers Tanu, that's the patch I'm testing right now. Looks good so far.
Comment 33 Cristina Agurida 2015-12-14 16:37:35 UTC
Tested with genericx86-64, core-image-sato-sdk image on NUC and the sound via HDMI works.

poky commit: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e
Comment 34 Jussi Kukkonen 2015-12-15 07:51:47 UTC
Oh yeah, this was patched in -- thanks for noticing. Setting to resolved fixed.
Comment 35 Cristina Agurida 2015-12-15 13:26:33 UTC
Verified on master: 5e3e2e0cbb0a49986f4653e64c4c8d2b5461645e