SYSTEMS AND METHODS FOR ADJUSTING DIRECTIONAL AUDIO IN A 360 VIDEO

• US 20170339507A1
• Filed: 05/10/2017
• Published: 11/23/2017
• Est. Priority Date: 05/18/2016
• Status: Abandoned Application

• Current Assignee: Cyberlink Corporation
• Original Assignee: Cyberlink Corporation
• Inventors: HSU, Chao-Hsien
• Application Number:
  US15/591,339
• Publication Number:
  US 20170339507A1
• Time in Patent Office: Days
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04S 2400/01 Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 2400/15 Aspects of sound capture and related signal processing for recording or reproduction
• CPC Class Codes: H04S 3/002 Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution control circuits for electronic adaptation of the sound field H04S7/30
• CPC Class Codes: H04S 3/004 For headphones
• CPC Class Codes: H04S 7/40 Visual indication of stereophonic sound image

Abstract

In a computing device for adjusting audio output during playback of 360 video, a 360 video bitstream is received, and the 360 video bitstream separated into video content and audio content. The audio content corresponding to a plurality of audio sources is decoded, wherein a number of audio sources is represented by N. The video content is displayed and the audio content is output through a plurality of output devices, wherein a number of output devices is represented by M. In response to detecting a change in a viewing angle for the video content, a determination is made, for each of the plurality of output devices, of a distribution ratio for each of the plurality of audio sources based on the viewing angle such that N×M distribution ratios are determined; and the audio content is output through each of the plurality of output devices based on the determined N×M distribution ratios.

Claims

1. A method implemented in a computing device for adjusting audio output during playback of 360 video, comprising:

receiving a 360 video bitstream;

separating the 360 video bitstream into video content and audio content;

decoding the audio content corresponding to a plurality of audio sources, wherein a number of audio sources is represented by N;

displaying the video content and outputting the audio content through a plurality of output devices, wherein a number of output devices is represented by M;

in response to detecting a change in a viewing angle for the video content;

determining, for each of the plurality of output devices, a distribution ratio for each of the plurality of audio sources based on the viewing angle such that N×

M distribution ratios are determined; and

outputting the audio content through each of the plurality of output devices based on the determined N×

M distribution ratios.

2. The method of claim 1, wherein detecting the change in the viewing angle for the video content comprises detecting input from at least one of:

a mouse, a touchscreen, a virtual-reality headset, and an accelerometer.

3. The method of claim 1, wherein outputting the audio content through each of the plurality of output devices based on the determined N×

M distribution ratios comprises;

generating, for each of the plurality of output devices, a magnitude for outputting audio content corresponding to each of the plurality of audio sources based on the N×

M distribution ratios such that N×

M magnitudes are adjusted;

outputting the audio content corresponding to each of the plurality of audio sources through each of the plurality of output devices based on the N×

M magnitudes.

4. The method of claim 1, wherein M is equal to 2, and wherein the output devices comprises a left channel output device and a right channel output device.

5. The method of claim 4, wherein the distribution ratios for the N audio sources for the left channel output device are determined according to:

6. The method of claim 4, wherein the N×

M magnitudes are generated according to;

7. The method of claim 1, wherein M is greater than 2, and wherein the output devices comprise multiple channels.

8. A system, comprising:

a memory storing instructions; and

a processor coupled to the memory and configured by the instructions to at least;

receive a 360 video bitstream;

separate the 360 video bitstream into video content and audio content;

decode the audio content corresponding to a plurality of audio sources, wherein a number of audio sources is represented by N;

display the video content and output the audio content through a plurality of output devices, wherein a number of output devices is represented by M;

in response to detecting a change in a viewing angle for the video content;

determine, for each of the plurality of output devices, a distribution ratio for each of the plurality of audio sources based on the viewing angle such that N×

M distribution ratios are determined; and

output the audio content through each of the plurality of output devices based on the determined N×

M distribution ratios.

9. The system of claim 8, wherein detecting the change in the viewing angle for the video content comprises detecting input from at least one of:

a mouse, a touchscreen, a virtual-reality headset, and an accelerometer.

10. The system of claim 8, wherein outputting the audio content through each of the plurality of output devices based on the determined N×

M distribution ratios comprises;

generating, for each of the plurality of output devices, a magnitude for outputting audio content corresponding to each of the plurality of audio sources based on the N×

M distribution ratios such that N×

M magnitudes are adjusted;

outputting the audio content corresponding to each of the plurality of audio sources through each of the plurality of output devices based on the N×

M magnitudes.

11. The system of claim 8, wherein M is equal to 2, and wherein the output devices comprises a left channel output device and a right channel output device.

12. The system of claim 11, wherein the distribution ratios for the N audio sources for the left channel output device are determined according to:

13. The system of claim 11, wherein the N×

M magnitudes are generated according to;

14. The system of claim 8, wherein M is greater than 2 and wherein the output devices comprise multiple channels.

15. A non-transitory computer-readable storage medium storing instructions to be implemented by a computing device having a processor, wherein the instructions, when executed by the processor, cause the computing device to at least:

receive a 360 video bitstream;

separate the 360 video bitstream into video content and audio content;

decode the audio content corresponding to a plurality of audio sources, wherein a number of audio sources is represented by N;

display the video content and output the audio content through a plurality of output devices, wherein a number of output devices is represented by M;

in response to detecting a change in a viewing angle for the video content;

determine, for each of the plurality of output devices, a distribution ratio for each of the plurality of audio sources based on the viewing angle such that N×

M distribution ratios are determined; and

output the audio content through each of the plurality of output devices based on the determined N×

M distribution ratios.

16. The non-transitory computer-readable storage medium of claim 15, wherein detecting the change in the viewing angle for the video content comprises detecting input from at least one of:

a mouse, a touchscreen, a virtual-reality headset, and an accelerometer.

17. The non-transitory computer-readable storage medium of claim 15, wherein outputting the audio content through each of the plurality of output devices based on the determined N×

M distribution ratios comprises;

generating, for each of the plurality of output devices, a magnitude for outputting audio content corresponding to each of the plurality of audio sources based on the N×

M distribution ratios such that N×

M magnitudes are adjusted;

outputting the audio content corresponding to each of the plurality of audio sources through each of the plurality of output devices based on the N×

M magnitudes.

18. The non-transitory computer-readable storage medium of claim 15, wherein M is equal to 2, and wherein the output devices comprises a left channel output device and a right channel output device.

19. The non-transitory computer-readable storage medium of claim 18, wherein the distribution ratios for the N audio sources for the left channel output device are determined according to:

20. The non-transitory computer-readable storage medium of claim 18, wherein the N×

M magnitudes are generated according to;