I have also added this as a feature request multiple times.
I ahve even called google and asked them to look into this.
Bose please fix this issue.
I was looking at doing a similar option using HDMI 2 in 1 out as i am having trouble sourcing a Google chromecast audio. Did you try using a chromecast via hdmi?