Audio processing device, audio processing method, and program

• US 9,351,092 B2
• Filed: 06/16/2011
• Issued: 05/24/2016
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Muramatsu, Tomoaki, Nakamura, Nobuhiko, Oguchi, Yoshio, Sayama, Masanobu
• Primary Examiner(s): Mei, Xu
• Application Number:
  US13/161,604
• Publication Number:
  US 20120002828A1
• Time in Patent Office: 1,804 Days
• Field of Search: 381/303, 381/285, 381/306, 381/333, 381/388, 382/285, 382/194
• US Class Current: 1/1
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 5/642 Disposition of sound reproducers
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
• CPC Class Codes: H04S 2400/11 Positioning of individual sound objects, e.g. moving airplane, within a sound field H04S2420/13 takes precedence
• CPC Class Codes: H04S 3/008 in which the audio signals are in digital form, i.e. employing more than two discrete digital channels data reduction aspects thereof based on psychoacoustics G10L19/02

Abstract

There is provided an audio processing device including an estimation unit configured to estimate a user'"'"'s representative perceived position of a stereoscopic image from a difference between a left-eye image and a right-eye image of the stereoscopic image displayed on a display device, and an audio controller configured to control audio output of an audio output device in accordance with the representative perceived position estimated by the estimation unit.

Claims

1. An audio processing device comprising:

an estimation unit configured to;

divide a left-eye image and a right-eye image of a stereoscopic image to be displayed on a display device into a plurality of regions;

estimate a representative perceived position of the stereoscopic image with respect to a user based on one or more perceived positions of each of the plurality of regions and a pixel size of the display device, wherein the one or more perceived positions are calculated based on a difference between the left-eye image and the right-eye image of the stereoscopic image; and

an audio controller configured to control audio output of an audio output device in accordance with the representative perceived position.

2. The audio processing device according to claim 1, wherein the audio output device comprises a front audio output device and a rear audio output device, the rear audio output device being arranged at a position that is farther from the display device than the front audio output device, and the audio controller controls the audio output of the front audio output device and the rear audio output device in accordance with a distance between the representative perceived position and a reference plane.

3. The audio processing device according to claim 2, wherein, when the representative perceived position coincides with the reference plane, the audio controller supplies a first audio signal to the front audio output device, and supplies a second audio signal to the rear audio output device.

4. The audio processing device according to claim 3, wherein, when the representative perceived position is closer to the user than the reference plane, the audio controller supplies the first audio signal to the front audio output device and the rear audio output device at a ratio in accordance with a distance between the representative perceived position and the reference plane.

5. The audio processing device according to claim 4, wherein the audio controller, supplies the first audio signal to the rear audio output device at a rate corresponding to the distance between the representative perceived position and the reference plane.

6. The audio processing device according to claim 5, wherein the audio controller, reduces the second audio signal supplied to the rear audio output device at a rate corresponding to the distance between the representative perceived position and the reference plane.

7. The audio processing device according to claim 3, wherein, when the reference plane is closer to the user than the representative perceived position, the audio controller supplies the second audio signal to the front audio output device and the rear audio output device at a ratio in accordance with the distance between the representative perceived position and the reference plane.

8. The audio processing device according to claim 7, wherein the audio controller supplies the second audio signal to the front audio output device at a rate corresponding to the distance between the representative perceived position and the reference plane.

9. The audio processing device according to claim 8, wherein the audio controller reduces the first audio signal supplied to the front audio output device at a rate corresponding to the distance between the representative perceived position and the reference plane.

10. The audio processing device according to claim 1, wherein the estimation unit estimates a perceived position of each of a plurality of face images, and estimates the representative perceived position based on the perceived position of each of the plurality of face images.

11. The audio processing device according to claim 10, wherein the estimation unit estimates, as the representative perceived position, one of the perceived positions of the plurality of face images.

12. The audio processing device according to claim 10, wherein the estimation unit estimates, as the representative perceived position, a mean value of the perceived positions of the plurality of face images.

13. The audio processing device according to claim 1, wherein the estimation unit estimates the representative perceived position based on a distribution of the one or more perceived positions of the plurality of regions.

14. The audio processing device according to claim 1, wherein the estimation unit is configured to detect a plurality of face images in the left-eye image and the right-eye image.

15. The audio processing device according to claim 14, wherein the estimation unit estimates a perceived position of each of the plurality of face images, and estimates the representative perceived position based on the perceived position of each of the plurality of face images.

16. The audio processing device according to claim 1, wherein the one or more perceived positions are calculated from the difference in horizontal display positions between an image object included in the left-eye image and an image object included in the right-eye image.

17. The audio processing device according to claim 1, wherein the estimation unit estimates distances between the one or more perceived positions of the plurality of regions and the display device.

18. The audio processing device according to claim 1, wherein the estimation unit calculates a distance based on a distribution of distances estimated for the plurality of regions, and wherein the audio controller controls the audio output of the audio output device based on the calculated distance.

19. An audio processing method comprising:

dividing a left-eye image and a right-eye image of a stereoscopic image to be displayed on a display device into a plurality of regions;

estimating a representative perceived position of the stereoscopic image with respect to a user based on one or more perceived positions of each of the plurality of regions and a pixel size of the display device, wherein the one or more perceived positions are calculated based on a difference between the left-eye image and the right-eye image of the stereoscopic image; and

controlling audio output of an audio output device in accordance with the representative perceived position.

20. A non-transitory computer-readable storage medium having stored thereon, a computer program having at least one code section for communication, the at least one code section being executable by a computer for causing the computer to perform steps comprising:

dividing a left-eye image and a right-eye image of a stereoscopic image to be displayed on a display device into a plurality of regions;

estimating a representative perceived position of the stereoscopic image with respect to a user based on one or more perceived positions of each of the plurality of regions and a pixel size of the display device, wherein the one or more perceived positions are calculated based on a difference between the left-eye image and the right-eye image of the stereoscopic image; and

controlling audio output of an audio output device in accordance with the representative perceived position.