PHOTOGRAPHING DEVICE AND PHOTOGRAPHING METHOD FOR TAKING PICTURE BY USING A PLURALITY OF MICROLENSES

• US 20140204183A1
• Filed: 01/17/2014
• Published: 07/24/2014
• Est. Priority Date: 01/22/2013
• Status: Abandoned Application

• Current Assignee: NTS Works, Inc., Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: HAN, Hee-chul, LEE, Tae-hee, TULIAKOV, Stepan
• Application Number:
  US14/158,148
• Publication Number:
  US 20140204183A1
• Time in Patent Office: Days
• US Class Current: 348/49
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: H01L 27/14627 Microlenses
• CPC Class Codes: H04N 13/232 using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 23/67 Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/957 Light-field or plenoptic cameras or camera modules
• CPC Class Codes: H04N 25/133 including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements
• CPC Class Codes: H04N 25/136 using complementary colours

Abstract

A photographing apparatus and method are provided. The photographing device includes: a main lens configured to transmit light beams reflected from a subject; a microlens array which includes a plurality of microlenses configured to filter and transmit the reflected light beams as different colors; an image sensor configured to sense the light beams that are transmitted by the plurality of microlenses; a data processor configured to collect pixels of positions corresponding to one another from a plurality of original images sensed by the image sensor to generate a plurality of sub images; a storage device configured to store the plurality of sub images; and a controller configured to detect pixels matching one another in the plurality of sub images stored in the storage device and to acquire color information and depth information of an image of the subject. Therefore, color information and depth information are restored without reducing resolution.

Claims

1. A photographing device comprising:

a main lens configured to transmit light beams reflected from a subject;

a microlens array which comprises a plurality of microlenses configured to filter and transmit the reflected light beams as different colors;

an image sensor configured to sense the light beams that are transmitted by the plurality of microlenses to sense a plurality of original images;

a data processor configured to collect pixels of positions corresponding to one another from the plurality of original images sensed by the image sensor to generate a plurality of sub images;

a storage device configured to store the plurality of sub images; and

a controller configured to detect pixels matching one another in the plurality of sub images stored in the storage device, and acquire color information and depth information of an image of the subject based on a result of the detection.

2. The photographing device of claim 1, wherein the controller performs at least one of a three-dimensional (3D) object detecting job and a re-focusing job by using the plurality of sub images.

3. The photographing device of claim 1, wherein the microlens array is divided into a plurality of microlens groups that are repeatedly arrayed,

wherein a plurality of microlenses are arrayed in each of the plurality of microlens groups according to preset color patterns, and

wherein colors separately selected from at least red (R), blue (B), green (G), cyan (C), yellow (Y), white (W), and emerald (E) are respectively allocated to the plurality of microlenses.

4. The photographing device of claim 1, wherein the image sensor is divided into a plurality of pixel groups which respectively correspond to the plurality of microlenses, and

wherein each of the plurality of pixel groups comprises a plurality of pixels, and the total number of pixels of the image sensor exceeds the number of the microlenses.

5. The photographing device of claim 4, wherein color coating layers are formed on surfaces of the plurality of microlenses, and

wherein colors of the color coating layers are repeated as preset patterns.

6. The photographing device of claim 4, wherein the microlens array comprises:

a first substrate on which the plurality of microlenses are arrayed in a matrix pattern; and

a second substrate on which a plurality of color filters respectively corresponding to the plurality of microlenses are arrayed,

wherein colors of the plurality of color filters are repeated as preset patterns.

7. A photographing method comprising:

filtering and transmitting light beams incident through a main lens by using a microlens array comprising a plurality of microlenses;

sensing the light beams that are transmitted by the plurality of microlenses using an image sensor to acquire a plurality of original images;

collecting pixels of positions corresponding to one another from the plurality of original images to generate a plurality of sub images;

storing the plurality of sub images; and

detecting pixels matching one another in the plurality of sub images to restore color information and depth information of a subject image.

8. The photographing method of claim 7, further comprising:

performing at least one of a three-dimensional (3D) object detecting job and a re-focusing job by using the color information and the depth information.

9. The photographing method of claim 7, wherein the microlens array is divided into a plurality of microlens groups that are repeatedly arrayed,

wherein a plurality of microlenses are arrayed in each of the plurality of microlens groups according to preset color patterns, and

wherein colors separately selected from at least red (R), blue (B), green (G), cyan (C), yellow (Y), white (W), and emerald (E) are respectively allocated to the plurality of microlenses.

10. The photographing method of claim 7, wherein color coating layers are formed on surfaces of the plurality of microlenses,

wherein colors of the color coating layers are repeated as preset patterns.

11. The photographing method of claim 7, wherein the microlens array comprises:

a first substrate on which the plurality of microlenses are arrayed in a matrix pattern; and

a second substrate on which a plurality of color filters respectively corresponding to the plurality of microlenses are arrayed,

wherein colors of the plurality of color filters are repeated as preset patterns.

12. A photographing device comprising:

a microlens array comprising a plurality of microlenses configured to filter light incident onto the microlens array according to a preset color pattern, wherein the microlens array is divided into a plurality of microlens groups that are repeatedly arrayed, and wherein a plurality of microlenses are arrayed in each of the plurality of microlens groups according to preset color patterns;

an image sensor array comprising a plurality of image sensor groups corresponding to the respective plurality of microlenses, wherein each of the plurality of image sensor groups comprise a plurality of pixels configured to sense the light filtered by a corresponding microlens to sense an original image; and

a data processor configured to collect pixels of positions corresponding to one another from a plurality of original images sensed by the image sensor array to generate a plurality of sub images.

13. The photographing device of claim 12, further comprising a main lens configured to transmit light reflected from a subject to the microlens array.

14. The photographing device of claim 13, further comprising a controller configured to detect pixels matching one another in the plurality of sub images, and acquire color information and depth information of an image of the subject based on a result of the detection.

15. The photographing device of claim 13, further comprising a field lens configured to transmit light that is transmitted by the main lens, to the microlens array.

16. The photographing device of claim 12, wherein color coating layers are formed on surfaces of the plurality of microlenses,

wherein colors of the color coating layers are repeated as preset patterns, and

wherein at least two colors separately selected from at least red (R), blue (B), green (G), cyan (C), yellow (Y), white (W), and emerald (E) are respectively allocated to the plurality of microlenses.

17. The photographing device of claim 12, wherein the microlens array comprises:

a first substrate on which the plurality of microlenses are arrayed in a matrix pattern; and

a second substrate on which a plurality of color filters respectively corresponding to the plurality of microlenses are arrayed,

wherein colors of the plurality of color filters are repeated as preset patterns, and

wherein at least two colors separately selected from at least R, B, G, C, Y, W, and E are respectively allocated to the plurality of microlenses.

18. A photographing method comprising:

filtering light incident onto a microlens array comprising a plurality of microlenses and a plurality of corresponding color filters configured to filter the light according to preset color patterns;

sensing the light that is transmitted by the plurality of microlenses using an image sensor to acquire a plurality of original images;

collecting pixels of positions corresponding to one another from the plurality of original images to generate a plurality of sub images; and

detecting pixels matching one another in the plurality of sub images to restore color information and depth information of a subject image.

19. The photographing method of claim 18, further comprising filtering and transmitting light incident through a main lens to the microlens array.

20. The photographing method of claim 18, wherein the microlens array is divided into a plurality of microlens groups that are repeatedly arrayed,

wherein a plurality of microlenses are arrayed in each of the plurality of microlens groups according to preset color patterns, and

wherein colors separately selected from at least red (R), blue (B), green (G), cyan (C), yellow (Y), white (W), and emerald (E) are respectively allocated to the plurality of microlenses.

21. The photographing method of claim 18, further comprising:

performing at least one of a three-dimensional (3D) object detection job and a re-focusing job by using the restored color information and the restored depth information.

22. The photographing method of claim 21, wherein the re-focusing job comprises:

selecting one of a plurality of sub images;

selecting a reference point on the sub image to be re-focused;

detecting depth information of the reference point; and

generating an image that is re-focused at the selected reference point.

23. The photographing method of claim 21, wherein the 3D object detection job comprises:

extracting disparity information between pixels corresponding to on another between the plurality of sub images;

generating a right eye image according to the disparity information; and

generating a left eye image according to the disparity information.