METHOD OF GATHERING VISUAL META DATA USING A REFERENCE IMAGE

• US 20080317357A1
• Filed: 06/16/2008
• Published: 12/25/2008
• Est. Priority Date: 08/05/2003
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Ireland Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Zamfir, Adrian, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei
• Granted Patent: US 8,330,831 B2
• Time in Patent Office: Days
• US Class Current: 382/209
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/20201 Motion blur correction
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 5/76 Television signal recording
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 9/8233 the additional signal being a character code signal

Abstract

A digital image processing technique gathers visual meta data using a reference image. A main image and one or more reference images are captured on a hand-held or otherwise portable or spatial or temporal performance-based image capture device. The reference images are analyzed based on predefined criteria in comparison to the main image. Based on said analyzing, supplemental meta data are created and added to the main image at a digital data storage location.

Claims

1. A digital image processing method for gathering visual meta data using a reference image, comprising:

(a) capturing, on a hand-held or otherwise portable or spatial or temporal performance-based image capture device, an original main image and one or more reference images having a temporal or spatial overlap or proximity with the original image, or combinations thereof;

(b) analyzing said one or more reference images based on predefined criteria in comparison to said main image;

(c) based on said analyzing, creating supplemental meta data; and

(d) adding said supplemental meta data to said main image at a digital data storage location.

2. The method of claim 1, further comprising:

(e) modifying said main image using said supplemental meta data; and

(f) rendering the modified image at a digital rendering device, display or printer, or combinations thereof, as output from the image capture device.

3. The method of claim 1, wherein the original main image and at least one reference image comprise one or more hi-speed versions to freeze motion of a substantially same scene.

4. The method of claim 1, wherein at least one reference image comprises an infra-red version of substantially the same scene to enhance a heat emitting object in the frame.

5. The method of claim 4, wherein said heat emitting object comprises a human body, or portion thereof.

6. The method of claim 5, wherein said heat emitting object comprises an exposed skin region of said human body.

7. The method of claim 6, wherein said heat emitting object comprises a face.

8. The method of claim 4, wherein said heat emitting object includes an animal.

9. The method of claim 1, wherein the original main image and at least one reference image comprise different focal distances.

10. The method of claim 9, wherein said reference image comprises a fixed focal length.

11. The method of claim 9, wherein said reference image includes two or more focal lengths organized in a grid pattern.

12. The method of claim 1, wherein the reference image comprises a plurality of images organized as a video clip.

13. The method of claim 1, wherein said reference image comprises a different focal length than said main image, thereby capturing a different angle of view relative to said main image.

14. The method of claim 13, wherein the reference image comprises an enlarged resolution of at least a portion of said main image.

15. The method of claim 13, further comprising dynamically determining said reference image.

16. The method of claim 15, wherein said dynamically determining comprises a function of content analysis of the image scene.

17. The method of claim 16, wherein said content analysis includes face detection, exposure analysis, color segmentation, local motion, eye detection, or edge detection, or combinations thereof.

18. The method of claim 13, wherein said different angle of view is achieved by selecting a portion of the image as captured by the sensor that is used to capture said main image.

19. The method of claim 1, wherein the reference image is a non-translucent light integrator for determining exposure and color balance.

20. The method of claim 1, wherein said predefined criteria includes facial instantiations such as frontal view, eyes wide open, smile, or a different angle of view than the main image, or combinations thereof.

21. A hand-held or otherwise portable or spatial or temporal performance-based image capture device, comprising:

(a) one or more lenses, an aperture and a photodetector for capturing an original main image and one or more reference images having a temporal or spatial overlap or proximity with the original image, or combinations thereof;

(b) a processor;

(c) one or more processor-readable media having embedded therein programming code for the processor to perform a digital image processing method that comprises;

(i) capturing, on a hand-held or otherwise portable or spatial or temporal performance-based image capture device, an original main image and one or more reference images having a temporal or spatial overlap or proximity with the original image, or combinations thereof;

(ii) analyzing said one or more reference images based on predefined criteria in comparison to said main image;

iii) based on said analyzing, creating supplemental meta data; and

(iv) adding said supplemental meta data to said main image at a digital data storage location.

22. The image capture device of claim 21, wherein the method further comprises:

(e) modifying said main image using said supplemental meta data; and

(f) rendering the modified image at a digital rendering device, display or printer, or combinations thereof, as output from the image capture device.

23. The image capture device of claim 21, wherein the original main image and at least one reference image comprise one or more hi-speed versions to freeze motion of a substantially same scene.

24. The image capture device of claim 21, wherein at least one reference image comprises an infra-red version of substantially the same scene to enhance a heat emitting object in the frame.

25. The image capture device of claim 24, wherein said heat emitting object comprises a human body, or portion thereof.

26. The image capture device of claim 25, wherein said heat emitting object comprises an exposed skin region of said human body.

27. The image capture device of claim 26, wherein said heat emitting object comprises a face.

28. The image capture device of claim 24, wherein said heat emitting object includes an animal.

29. The image capture device of claim 21, wherein the original main image and at least one reference image comprise different focal distances.

30. The image capture device of claim 29, wherein said reference image comprises a fixed focal length.

31. The image capture device of claim 29, wherein said reference image includes two or more focal lengths organized in a grid pattern.

32. The image capture device of claim 21, wherein the reference image comprises a plurality of images organized as a video clip.

33. The image capture device of claim 21, wherein said reference image comprises a different focal length than said main image, thereby capturing a different angle of view relative to said main image.

34. The image capture device of claim 33, wherein the reference image comprises an enlarged resolution of at least a portion of said main image.

35. The image capture device of claim 33, wherein the method further comprises dynamically determining said reference image.

36. The image capture device of claim 35, wherein said dynamically determining comprises a function of content analysis of the image scene.

37. The image capture device of claim 36, wherein said content analysis includes face detection, exposure analysis, color segmentation, local motion, eye detection, or edge detection, or combinations thereof.

38. The image capture device of claim 33, wherein said different angle of view is achieved by selecting a portion of the image as captured by the sensor that is used to capture said main image.

39. The image capture device of claim 21, wherein the reference image is a non-translucent light integrator for determining exposure and color balance.

40. The image capture device of claim 21, wherein said predefined criteria includes facial instantiations such as frontal view, eyes wide open, smile, or a different angle of view than the main image, or combinations thereof.