Image capturing apparatus with image compensation and method thereof

• US 20070065039A1
• Filed: 09/22/2006
• Published: 03/22/2007
• Est. Priority Date: 09/22/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Park, Tae-suh, Kim, Yeun-bae, Park, Min-kyu
• Granted Patent: US 8,842,935 B2
• Time in Patent Office: Days
• US Class Current: 382/275
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display

Abstract

An image capturing apparatus with image compensation and a method thereof, which can perform compensation of a user'"'"'s facial image that is distorted in accordance with the position of an image capturing device such as a camera. The image capturing apparatus with image compensation includes an image capturing unit capturing an image of an object, a tilt determining unit determining a tilt of the image capturing unit relative to the object, and a compensation unit compensating the captured image in accordance with the determined tilt.

Claims

1. An image capturing apparatus comprising:

an image capturing unit capturing an image of an object;

a tilt determining unit determining a tilt of the image capturing unit relative to the object; and

a compensation unit compensating the captured image in accordance with the determined tilt.

2. The image capturing apparatus of claim 1, wherein the object is a person.

3. The image capturing apparatus of claim 1, wherein the image capturing unit captures the image of the object from a position below the object.

4. The image capturing apparatus of claim 1, further comprising a tilt sensing unit sensing a tilt of a device in which the image capturing unit is installed.

5. The image capturing apparatus of claim 4, wherein the tilt sensing unit senses the tilt between a person and the image capturing unit from a relative position between facial feature points of the person, using the image captured by the image capturing unit.

6. The image capturing apparatus of claim 4, further comprising:

a first storage unit storing the image captured by the image capturing unit; and

a second storage unit storing information about the sensed tilt and the determined tilt.

7. The image capturing apparatus of claim 4, wherein the tilt sensing unit senses the tilt of the device, in which the image capturing device is installed, relative to a plane, which is perpendicular to a direction of gravity, using an inertial sensor.

8. The image capturing apparatus of claim 1, further comprising a tilt storage unit storing predetermined tilts of the image capturing unit relative to the object.

9. The image capturing apparatus of claim 8, the tilt determining unit selects one of the stored tilts as the tilt of the image capturing unit.

10. The image capturing apparatus of claim 1, wherein the compensation unit determines whether to perform the compensation of the captured image and a level of compensation in accordance with the tilt determined by the tilt determining unit.

11. The image capturing apparatus of claim 10, wherein the compensation unit compensates the captured image through a geometric transform technique.

12. The image capturing apparatus of claim 11, wherein the compensation unit adjusts transform matrix coefficients used for the image compensation in accordance with the determined tilt.

13. The image capturing apparatus of claim 11, wherein the compensation unit uses a perspective transform technique among the geometric transform technique of the captured image.

14. The image capturing apparatus of claim 11, wherein, when the captured image is transformed through the geometric transform technique, the compensation unit performs compensation so that two specified regions, which have a same shape and area as those of the image before being compensated and have different positions in upward and downward directions, respectively, are determined, and an area of a region at a relatively low position is smaller than an area of a region at a relatively high position.

15. The image capturing apparatus of claim 1, further comprising a user compensation unit adjusting a degree of the image compensation performed in the compensation unit by receiving a user'"'"'s change of the determined tilt or a compensation level determined by the compensation unit.

16. The image capturing apparatus of claim 11, further comprising a display unit displaying the image compensated by the compensation unit,

wherein, when the region of the compensated image does not coincide with a display region of the display unit, and an empty region having no image is displayed, the compensation unit performs a process of the empty region.

17. The image capturing apparatus of claim 16, wherein the compensation unit fills the empty region with image pixel values constituting the corresponding region in the image before being compensated.

18. The image capturing apparatus of claim 16, wherein the compensation unit fills the empty region with pixel values of the compensated image existing at the closest position in a same row, on a basis of a specified pixel in the empty region.

19. The image capturing apparatus of claim 16, wherein the compensation unit overlays a specified frame over a circumference of the display region so as to hide the empty region.

20. The image capturing apparatus of claim 16, wherein, when the image capturing unit captures an image that is larger than the display region, the compensation unit compensates the image and then crops the compensated image to match the display region.

21. The image capturing apparatus of claim 16, wherein the compensation unit cuts out the region, except for the empty region from the compensated image, in the form of a square, and then enlarges the cut region to a size enough to fill the display region.

22. An image compensation method, the method comprising:

capturing an image of an object;

determining a tilt of an image capturing unit relative to the object; and

compensating the captured image in accordance with the determined tilt.

23. The method of claim 22, wherein the object is a person.

24. The method of claim 22, wherein the capturing comprises capturing the image of the object from a position below the object.

25. The method of claim 22, further comprising sensing the tilt of a device in which an image capturing module, which performs the capturing, is installed.

26. The method of claim 25, wherein, in the sensing, a tilt between a person and the image capturing unit from a relative position between facial feature points of the person is sensed, using the captured image.

27. The method of claim 25, further comprising:

storing the captured image; and

storing information about the sensed tilt and the determined tilt.

28. The method of claim 25, wherein, in the sensing, the tilt of the device, in which the image capturing module is installed, relative to a plane, which is perpendicular to a direction of gravity is sensed, using an inertial sensor.

29. The method of claim 22, wherein the determining comprises determining the tilt of the image capturing module with reference to one of plural pre-stored tilts.

30. The method of claim 22, wherein the compensating comprises determining whether to perform the compensation of the captured image and a level of compensation in accordance with the determined tilt.

31. The method of claim 30, wherein, in the compensating, the captured image is compensated through a geometric transform technique.

32. The method of claim 31, wherein the compensating comprises adjusting transform matrix coefficients used for the image compensation in accordance with the determined tilt.

33. The method of claim 31, wherein, in the compensating, the captured image is compensated through a perspective transform technique among the geometric transform technique of the captured image.

34. The method of claim 31, wherein, in the compensating, when the captured image is compensated through the geometric transform technique, the compensating is performed so that two specified regions, which have a same shape and area as those of the image before being compensated and have different positions in upward and downward directions, respectively, are determined, and an area of a region at a relatively low position is smaller than an area of a region at a relatively high position.

35. The method of claim 22, further comprising adjusting a degree of the image compensation by enabling a user to change the determined tilt or a determined compensation level.

36. The method of claim 31, wherein, when the region of the compensated image does not coincide with a display region on which the compensated image is displayed, and thus an empty region having no image is displayed, the compensating comprises performing a process on the empty region.

37. The method of claim 36, wherein, in the compensating, the empty region is filled with image pixel values constituting the corresponding region in the image before being compensated.

38. The method of claim 36, wherein, in the compensating, the empty region is filled with pixel values of the compensated image existing at the closet position in a same row, on a basis of a specified pixel in the empty region.

39. The method of claim 36, wherein the compensating comprises overlaying a specified frame over a circumference of the display region so as to hide the empty region.

40. The method of claim 36, wherein, when an image that is larger than the display region is captured, the compensating comprises compensating the image and then cropping the compensated image to match the display region.

41. The method of claim 36, wherein the compensating comprises cutting out the region, except for the empty region from the compensated image, in the form of a square, and then enlarging the cut region to a size enough to fill the display region.

45. A computer-readable storage medium encoded with processing instructions for causing a processor to execute the method of claim 22.

42. A video telephony apparatus comprising:

an image capturing unit in a device, having an optical axis, and movable relative to a main body of the device;

a tilt sensing unit in the main body sensing a tilt of the device relative to horizontal;

a tilt determining unit determining a tilt of the image capturing unit based on an angle between a plane in the main body and the optical axis; and

a compensation unit compensating for distortion of an image captured by the image capturing unit, based on the calculated tilt.

43. The apparatus of claim 42, wherein the compensation unit compensates for distortion of the captured image by judging whether the captured image exhibits perspective distortion and compensates for the distortion by using perspective compensation.

44. An image compensation method, comprising:

sensing a first tilt angle which is an angle of a device in which an image capturing unit is installed relative to a horizontal plane;

calculating a second tilt angle which is an angle of the image capturing unit relative to the device;

capturing an image of an object and determining, based on the first and the second tilt angles, whether the image capturing unit is above the object, below the object or level with the object; and

compensating for image distortion of the captured image when the image capturing unit is determined to be above or below the object.

46. A computer-readable storage medium encoded with processing instructions for causing a processor to execute the method of claim 44.