Image display device and method for determining an audio output position based on a displayed image

• US 8,483,414 B2
• Filed: 10/16/2006
• Issued: 07/09/2013
• Est. Priority Date: 10/17/2005
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Watanabe, Yoshinori, Yamada, Naohide
• Primary Examiner(s): Nguyen, Duc
• Assistant Examiner(s): BLAIR, KILE O
• Application Number:
  US11/580,917
• Publication Number:
  US 20070104341A1
• Time in Patent Office: 2,458 Days
• Field of Search: 381/306, 381/333, 381/388, 348/500, 348/515, 348/738, 348/462, 348/483, 348/485, 348/481
• US Class Current: 381/306
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: H04R 2420/03 Connection circuits to selectively connect loudspeakers or headphones to amplifiers
• CPC Class Codes: H04R 2499/15 Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
• CPC Class Codes: H04R 3/12 for distributing signals to two or more loudspeakers specially adapted for hearing aids H04R25/407
• CPC Class Codes: H04R 5/02 Spatial or constructional arrangements of loudspeakers

Abstract

An image display device for displaying images includes an image display unit for displaying images, an audio output unit for outputting audio from multiple positions corresponding to the images displayed by the image display unit, and an audio output position control unit for analyzing the images, controlling the audio output unit according to content of the images, and selecting the position from which to output audio.

Claims

1. An image display device for displaying images, comprising:

image display means for displaying images;

audio output means for outputting audio comprising a plurality of channels from one or more of multiple positions corresponding to the images displayed with the image display means;

audio output position control means for analyzing the images, controlling the audio output means according to content of the images, and selecting a position from which to output audio;

screen region dividing means for dividing a display screen of the image display means into a predetermined plurality of regions;

difference average value calculating means for calculating a difference average value which is an average value of difference values of a luminance value between frames for each pixel having predetermined features wherein a pixel value thereof is determined in advance, for each region divided by the screen region dividing means; and

audio output control means for controlling the audio output means and selecting the position for outputting the audio, based on the difference average values calculated with the difference average value calculating means and based on a volume difference between a volume of a first channel in the plurality of channels and a volume of a second channel in the plurality of channels.

2. The image display device according to claim 1, wherein the audio output means further comprises a plurality of speakers;

and wherein the audio output position control means controls the audio output means to select a speaker from the plurality of speakers from which to output the audio.

3. The image display device according to claim 1, wherein the audio output means are capable of controlling an audio output position virtually by controlling output timing of the audio;

and wherein the audio output position control means control the audio output means to select the output timing, thus controlling a virtual position from which to output the audio.

4. The image display device according to claim 1, the audio output position control means further comprising:

volume confirming means for confirming a volume of the plurality of channels to specify the audio output positions;

difference average value correcting means for correcting the difference average value calculated by the difference average value calculating means, based on the audio output positions specified by the volume confirming means; and

region specifying means for specifying a region for outputting the audio, based on the difference average value corrected by the difference average value correcting means,

wherein the audio output control means controls the audio output means so as to output the audio from a position corresponding to the region specified by the region specifying means.

5. The image display device according to claim 1, the audio output position control means further comprising:

region specifying means for specifying a region for the audio to be output, based on the difference average value calculated by the difference average value calculating means;

volume adjusting means for determining an adjustment amount of a volume of the audio output from the audio output means, based on the region specified by the region specifying means;

volume confirming means for confirming a volume of the plurality of channels to specify the audio output positions;

comparing means for comparing the region specified by the region specifying means and the audio output positions specified by the volume confirming means; and

adjustment amount correcting means for correcting the amount of adjustment for audio volume determined by the volume adjusting means, based on the comparison results by the comparing means;

wherein, the audio output control means controls the audio output means so as to adjust the audio volume based on the volume adjustment amount corrected by the adjustment amount correcting means.

6. The image display device according to claim 1, further comprising user position corresponding processing means for detecting a position of a user viewing/listening to the images and audio, and controlling a delay amount of audio output timing by the audio output means, according to the position of the user.

7. The image display device according to claim 1, wherein the audio output means has a plurality of detachable speakers;

and

wherein the image display device further comprises region setting means for setting the regions to be divided by the screen region dividing means, according to positions of the speakers connected thereto.

8. The image display device according to claim 1, wherein the audio output means has a plurality of speakers, and a location of each of the plurality of speakers corresponds to one of the plurality of regions.

9. The image display device according to claim 8, wherein the plurality of speakers includes at least three speakers located along a single side of the display screen perimeter.

10. The image display device according to claim 8, wherein the plurality of speakers includes a plurality of speakers located on each side of the display screen perimeter.

11. The image display device according to claim 8, wherein the plurality of regions is divided into a grid, and each section of the grid corresponds to a pair of speakers which include one speaker on a first side of the display screen perimeter and another speaker on a second side of the display screen perimeter.

12. The image display device according to claim 1, wherein the audio output control means determines if the calculated difference average values are at or below a threshold in all of the plurality of regions, and when the difference average values are not at or below the threshold in all the regions, the audio output control means selects the position corresponding to the region having the greatest difference average value to output audio, and when the difference average values are at or below the threshold in all the regions, the audio output control means outputs audio from all of the positions.

13. An image display method for displaying an image on an image display device, comprising the steps of:

displaying an image having image content associated with a plurality of audio channels;

analyzing the image to be displayed, and selecting a position of audio output corresponding to the image, according to the image content and the plurality of audio channels;

dividing a display screen on which the image is displayed into a predetermined plurality of regions;

calculating a difference average value which is an average value of the difference values of the luminance value between frames for each pixel having predetermined features wherein a pixel value thereof is determined in advance, for each of the plurality of regions divided on the display screen; and

selecting the position of audio output corresponding to the image, according to the calculated difference average value in the calculating and a volume difference between a volume of a first channel in the plurality of channels and a volume of a second channel in the plurality of channels.

14. The method according to claim 13, further comprising:

determining if the calculated difference average values are at or below a threshold in all of the plurality of regions; and

when the difference average values are not at or below the threshold in all the regions, selecting the position corresponding to the region having the greatest difference average value to output audio, and when the difference average values are at or below the threshold in all the regions, outputting audio from all of the positions.

15. A computer readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method comprising:

displaying an image having image content associated with a plurality of audio channels;

analyzing the image to be displayed, and selecting a position of audio output corresponding to the image, according to the image content and the plurality of audio channels;

dividing a display screen on which the image is displayed into a predetermined plurality of regions;

calculating a difference average value which is an average value of the difference values of the luminance value between frames for each pixel having predetermined features wherein a pixel value thereof is determined in advance, for each of the plurality of regions divided on the display screen; and

selecting the position of audio output corresponding to the image, according to the calculated difference average value in the calculating and a volume difference between a volume of a first channel in the plurality of channels and a volume of a second channel in the plurality of channels.

16. An image display device for displaying images, comprising:

an image display unit configured to display images;

an audio output unit configured to output audio comprising a plurality of channels from one or more multiple positions corresponding to the images displayed with the image display unit;

an audio output position control unit configured to analyze the images, controlling the audio output unit according to content of the images, and selecting a position from which to output audio;

a screen region dividing unit configured to divide a display screen of the image display unit into a predetermined plurality of regions;

a difference average value calculating unit configured to calculate a difference average value which is an average value of the difference values of a luminance value between frames for each pixel having predetermined features wherein a pixel value thereof is determined in advance, for each region divided by the screen region dividing unit;

an audio output control unit configured to control the audio output unit and select the position for outputting the audio, based on the difference average values calculated with the difference average value calculating unit and based on a volume difference between a volume of a first channel in the plurality of channels and a volume of a second channel in the plurality of channels.

17. The image display device according to claim 16, wherein the audio control unit is further configured to determine if the calculated difference average values are at or below a threshold in all of the plurality of regions, and when the difference average values are not at or below the threshold in all the regions, the audio output control unit selects the position corresponding to the region having the greatest difference average value to output audio, and when the difference average values are at or below the threshold in all the regions, the audio output control unit outputs audio from all of the positions.