Terminal apparatus for combining images from two different cameras based on detected sound

• US 9,148,586 B2
• Filed: 01/30/2013
• Issued: 09/29/2015
• Est. Priority Date: 02/10/2012
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Mobile Communications Incorporated (Sony Group Corp.)
• Inventors: Takatsuka, Susumu, Matsumoto, Yuji, Takanashi, Shin, Ide, Yuji
• Primary Examiner(s): PRABHAKHER, PRITHAM DAVID
• Application Number:
  US13/754,504
• Publication Number:
  US 20130208136A1
• Time in Patent Office: 972 Days
• Field of Search: 348/333.05, 348/211.99, 348/239
• US Class Current: 1/1
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 5/2621 Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability

Abstract

A terminal apparatus including a display unit disposed on a case and is configured to display an image; a first camera disposed on a surface on which the display unit performs image display; a second camera disposed on a surface opposite the surface on which the first camera is disposed; a plurality of microphones disposed on the case; a sound processing unit that detects sound transmitted from a sound source on a side of the first camera with sound signals obtained by at least one of the plurality of microphones; and an image processing unit that combines a first image which is captured by the first camera or is stored in advance with a second image captured by the second camera on the basis of a result of the detection of sound transmitted from the sound source on the side of the first camera.

Claims

1. A terminal apparatus comprising:

a display that is disposed on a case and is configured to display an image;

a first camera disposed on a surface on which the display performs image display;

a second camera disposed on a surface opposite the surface on which the first camera is disposed;

a plurality of microphones disposed on the case; and

processing circuitry configured to

control detection of sound transmitted from a sound source on a side of the first camera with sound signals obtained by at least one of the plurality of microphones, and

automatically combine a first image which is captured by the first camera with a second image captured by the second camera on the basis of a result of the detection of sound transmitted from the sound source on the side of the first camera having a level equal to or higher than a threshold value,

wherein the first image and the second image are each moving images, and the processing circuitry automatically combines the first image and the second image while the moving images for the first image and the second image are being captured by the first camera and the second camera respectively,

the processing circuitry being further configured to determine whether a mouth of a user facing a side of the first camera has moved and to determine if there is a sound from the user obtained by at least one of the plurality of microphones, and automatically combine the first image with the second image when the processing circuitry determines that there is a sound from the user obtained by at least one of the plurality of microphones and that the mouth of the user has moved, and

when there is no detection of sound transmitted from the sound source on the side of the first camera by the processing circuitry, the second image is displayed on the display without the first image.

2. The terminal apparatus according to claim 1, the processing circuitry being further configured to perform face recognition processing on the basis of a captured image of a face of a user facing a side of the first camera.

3. The terminal apparatus according to claim 1, wherein the first image is combined with the second image such that the first image has a smaller size than the second image and the first image overlaps a portion of the second image.

4. The terminal apparatus according to claim 1, wherein the first image is combined with the second image such that the first image is displayed adjacent to the second image.

5. The terminal apparatus according to claim 1, wherein the processing circuitry is configured to perform sound effect processing on the sound source on the side of the first camera.

6. The terminal apparatus according to claim 5, wherein the sound effect processing is based on at least one of pitch and tempo settings set in advance.

7. A method implemented by a terminal apparatus that includes a display that is disposed on a case and is configured to display an image, a first camera disposed on a surface on which the display performs image display, a second camera disposed on a surface opposite the surface on which the first camera is disposed, and a plurality of microphones disposed on the case, the method comprising:

detecting, by processing circuitry, sound transmitted from a sound source on a side of the first camera with sound signals obtained by at least one of the plurality of microphones; and

automatically combining, by the processing circuitry, a first image which is captured by the first camera with a second image captured by the second camera on the basis of a result of the detection of sound transmitted from the sound source on the side of the first camera,

wherein the first image and the second image are each moving images, and the processing circuitry automatically combines the first image and the second image while the moving images for the first image and the second image are being captured by the first camera and the second camera respectively,

the method further including determining whether a mouth of a user facing a side of the first camera has moved and determining if there is a sound from the user obtained by at least one of the plurality of microphones, and automatically combining the first image with the second image when it is determined that there is a sound from the user obtained by at least one of the plurality of microphones and that the mouth of the user has moved, and

when there is no detection of sound transmitted from the sound source on the side of the first camera, the second image is displayed on the display without the first image.

8. A non-transitory computer readable storage medium storing a program that when executed by a terminal apparatus that includes a display that is disposed on a case and is configured to display an image, a first camera disposed on a surface on which the display performs image display, a second camera disposed on a surface opposite the surface on which the first camera is disposed, a plurality of microphones disposed on the case, and processing circuitry, causes the terminal apparatus to perform a method comprising:

detecting sound transmitted from a sound source on a side of the first camera with sound signals obtained by at least one of the plurality of microphones; and

automatically combining a first image which is captured by the first camera with a second image captured by the second camera on the basis of a result of the detection of sound transmitted from the sound source on the side of the first camera having a level equal to or higher than a threshold value,

wherein the first image and the second image are each moving images, and the processing circuitry automatically combines the first image and the second image while the moving images for the first image and the second image are being captured by the first camera and the second camera respectively,

the method further including determining whether a mouth of a user facing a side of the first camera has moved and determining if there is a sound from the user obtained by at least one of the plurality of microphones, and automatically combining the first image with the second image when it is determined that there is a sound from the user obtained by at least one of the plurality of microphones and that the mouth of the user has moved, and

when there is no detection of sound transmitted from the sound source on the side of the first camera, the second image is displayed on the display without the first image.