Face image correction using multiple camera angles

• US 8,208,010 B2
• Filed: 06/02/2005
• Issued: 06/26/2012
• Est. Priority Date: 06/30/2004
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications USA Incorporated (Sony Group Corp.)
• Inventors: Nelson, Joakim, Gulliksson, Johan
• Primary Examiner(s): KHAN, USMAN A
• Application Number:
  US11/629,671
• Publication Number:
  US 20070252674A1
• Time in Patent Office: 2,581 Days
• Field of Search: 348/14.16, 348/113, 348/116, 348/118, 348/140, 348/142, 348/143, 348/148, 348/50, 382/104, 382115-118
• US Class Current: 348/50
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Methods for adjusting a picture of an object captured by a camera in a mobile radio communication terminal include defining a first camera angle relative to the object and capturing an image of the object by means of the camera from a second camera angle relative to the object. The second camera angle is offset from the first camera angle. The methods further include storing image data relating to the captured image, and generating an angularly adjusted image of the object in response to the image data and an angular relation between the first camera angle and the second camera angle. Corresponding mobile radio communication terminals are also disclosed. The terminals include an image processing system configured to process an image of an object captured by a camera from a second camera angle by generating an angularly adjusted image of the object in response to image data for the image, and in response to an angular relation between the second camera angle and a first camera angle.

Claims

1. A method of adjusting real-time moving images of an object captured by a camera in a mobile radio communication terminal, comprising:

defining a first camera angle based on a first camera position of the camera relative to the object;

capturing a reference still image of said object from said first camera angle using the camera;

storing reference still image data relating to the reference still image of the object captured from the first camera angle in said mobile radio communication terminal;

moving the camera from the first camera position relative to the object to a second camera position relative to the object;

capturing real-time moving images of the object by means of the camera from a second camera angle based on the second camera position relative to the object, wherein the second camera angle is offset from the first camera angle;

storing moving image data relating to the captured real-time moving images in said mobile radio communication terminal; and

generating angularly adjusted moving images of the object in response to the moving image data and an angular relation between the first camera angle of the camera and the second camera angle of the camera, wherein the step of generating angularly adjusted moving images of the object is also dependent on the reference still image data, and wherein the step of generating angularly adjusted moving images comprises generating the angularly adjusted moving images in real time, and wherein the object captured by said camera is the face of a user using said mobile radio communication terminal.

2. The method as recited in claim 1, wherein generating the angularly adjusted moving images comprises:

digitally identifying corresponding structures in the captured real-time moving images and the reference still image;

establishing a mathematical transform from a mathematical relation between the moving image data of the real-time moving images and of the reference still image data, for the corresponding structures; and

transforming the captured real-time moving images to appear as captured from the first camera angle by applying the mathematical transform to the moving image data.

3. The method as recited in claim 1, wherein generating the angularly adjusted moving images comprises:

digitally identifying a key feature present both in the captured real-time moving images and in the reference still image;

detecting movement of the key feature in the captured real-time moving images;

filtering out the movement from the captured real-time moving images; and

applying the movement to the key feature in the reference still image for obtaining adjusted real-time moving images.

4. The method as recited in claim 1, wherein generating the angularly adjusted real-time moving images comprises:

slanting the captured real-time moving images to compensate for a difference between the first camera angle and the second camera angle; and

zooming and cropping the slanted real-time moving images.

5. The method as recited in claim 1, wherein the first camera angle comprises a predetermined camera angle to the object.

6. The method as recited in claim 1, wherein the first camera angle is selected from a plurality of selectable preset angles, by means of an input command to the terminal.

7. The method as recited in claim 1, wherein the second camera angle is sensed by an angle sensor in the terminal.

8. The method as recited in claim 1, wherein the second camera angle is selected from a plurality of selectable preset angles, by means of an input command to the terminal.

9. The method as recited in claim 1, comprising:

encoding the adjusted real-time moving images into a radio signal; and

transmitting the radio signal from the terminal.

10. The method as recited in claim 1, comprising:

encoding the image data for the captured real-time moving images, together with data indicative of the angular relation between the first camera angle and the second camera angle, into a radio signal;

transmitting the radio signal from the terminal;

receiving the radio signal in a radio signal receiving device; and

generating the angularly adjusted real-time moving images of the object in the radio signal receiving device.

11. A mobile radio communication terminal, comprising:

a camera configured to capture images including moving images;

means for generating and transmitting a radio signal comprising captured images to a recipient; and

an image adjusting mechanism, the image adjusting mechanism comprising;

a memory configured to store a first angle value for a first camera angle based on a first camera position of the camera relative to an object, a second angle value for a second camera angle based on a second camera position of the camera relative to the object, wherein the second camera angle is offset from the first camera angle and the second camera position is different from the first camera position, and reference still image data relating to a reference still image of the object captured from the first camera angle; and

an image processing system configured to capture the reference still image of the object from the first camera angle using the camera, to capture real-time moving images of the object from the second camera angle using the camera, to process the real-time moving images of the object captured from the second camera angle by generating angularly adjusted real-time moving images of the object in response to moving image data for the real-time moving images, in response to an angular relation between the first camera angle and the second camera angle, and in response to the reference still image data, wherein the image processing system is configured to generate the angularly adjusted moving images in real time, and wherein the object captured by said camera is the face of a user using said mobile radio communication terminal.

12. The mobile radio communication terminal as recited in claim 11, wherein the image processing system comprises:

means for digitally identifying corresponding structures in the captured real-time moving images and the reference still image;

means for establishing a mathematical transform from a mathematical relation between the moving image data and of the reference still image data, for the corresponding structures; and

means for transforming the captured real-time moving images to appear as captured from the first camera angle by applying the mathematical transform to the moving image data.

13. The mobile radio communication terminal as recited in claim 11, wherein the image processing system comprises:

means for digitally identifying a key feature in the captured real-time moving images and in the reference still image;

means for detecting movement of the key feature in the captured real-time moving images;

means for filtering out the movement from the captured real-time moving images; and

means for applying the movement to the key feature in the reference still image for obtaining the adjusted real-time moving images.

14. The mobile radio communication terminal as recited in claim 11, wherein the image processing system comprises:

means for slanting the captured real-time moving images to compensate for a difference between the first camera angle and the second camera angle; and

means for zooming and cropping the slanted real-time moving images.

15. The mobile radio communication terminal as recited in claim 11, wherein the first camera angle comprises a predetermined camera angle to the object.

16. The mobile radio communication terminal as recited in claim 11, wherein the first camera angle is selected from a plurality of selectable preset angles, by means of an input command to the terminal.

17. The mobile radio communication terminal as recited in claim 11, wherein the second camera angle is sensed by an angle sensor in the terminal.

18. The mobile radio communication terminal as recited in claim 11, wherein the second camera angle is selected from a plurality of selectable preset angles, by means of an input command to the terminal.

19. The mobile radio communication terminal as recited in claim 11, wherein the image processing system comprises:

means for encoding the adjusted real-time moving images into a radio signal; and

means for transmitting the radio signal from the terminal.