Methods and apparatus relating to detection and/or indicating a dirty lens condition

• US 10,191,356 B2
• Filed: 07/03/2015
• Issued: 01/29/2019
• Est. Priority Date: 07/04/2014
• Status: Active Grant

• Current Assignee: Blue River Technology Inc. (Deere & Company)
• Original Assignee: Light Labs Inc.
• Inventors: Laroia, Rajiv, Shroff, Sapna A.
• Primary Examiner(s): Aggarwal, Yogesh
• Application Number:
  US14/791,369
• Publication Number:
  US 20160004144A1
• Time in Patent Office: 1,306 Days
• Field of Search: 3482201-248
• CPC Class Codes: G02B 27/0006 with means to keep optical surfaces clean, e.g. by preventing or removing dirt, stains, contamination, condensation G02B1/18 takes precedence; cleaning in general B08B
• CPC Class Codes: G03B 17/02 Bodies
• CPC Class Codes: G03B 17/18 Signals indicating condition of a camera member or suitability of light indicating depth of field G03B13/30
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/634 Warning indications
• CPC Class Codes: H04N 23/811 by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor
• CPC Class Codes: H04N 25/585 with pixels having different sensitivities within the sensor, e.g. fast or slow pixels or pixels having different sizes
• CPC Class Codes: H04N 25/589 with different integration times, e.g. short and long exposures

Abstract

The present application relates to image capture and generation methods and apparatus and, more particularly, to methods and apparatus which detect and/or indicate a dirty lens condition. One embodiment of the present invention includes a method of operating a camera including the steps of capturing a first image using a first lens of the camera; determining, based on at least the first image, if a dirty camera lens condition exists; and in response to determining that a dirty lens condition exists, generating a dirty lens condition notification or initiating an automatic camera lens cleaning operation. In some embodiments multiple captured images with overlapping image regions are compared to determine if a dirty lens condition exists.

Claims

1. A method of operating a device including a plurality of at least three optical chains, the method comprising:

capturing a first image using a first lens of a first optical chain;

capturing a second image using a second lens of a second optical chain;

capturing a third image using a third lens of a third optical chain;

performing a first comparison of said first image or a characteristic of said first image to the second image or a characteristic of said second image;

performing a second comparison, said second comparison being a comparison of said third image or a characteristic of said third image to another image or a characteristic of said another image, said another image being either the first image or the second image;

determining, based on at least one of said first and second comparisons, that a dirty camera lens condition exists, at least one of said first, second and third lenses having been determined to be dirty when said dirty camera lens condition exists, said at least one lens being a dirty lens;

associating dirty lens information with an image captured using said dirty lens; and

combining captured images to generate a composite image, the image captured using the dirty lens being treated differently during the image combining process used to generate the composite image than an image captured using a clean lens.

2. The method of claim 1, further comprising:

in response to determining that a dirty lens condition exists, generating a dirty lens condition notification or initiating an automatic camera lens cleaning operation;

wherein generating the dirty lens condition notification is performed in response to determining that the dirty lens condition exists; and

wherein the generated dirty lens condition notification identifies a particular one of said first, second and third lenses being dirty.

3. The method of claim 1, wherein said first and second images were captured at the same time using outer lenses corresponding to the first and second optical chains, respectively.

4. The method of claim 2, wherein said generated dirty lens condition notification is an audio notification which informs a user of which particular lens or lenses have been determined to be dirty.

5. The method of claim 1,

wherein said first comparison is a comparison of a first image metric corresponding to said first image and a second image metric corresponding to said second image, the first and second image metrics corresponding to an overlapping image region;

and

wherein determining that said dirty camera lens condition exists includes determining if a different in the first and second image metrics is above a predetermined threshold.

6. The method of claim 1, further comprising:

moving a mirror in an optical chain determined to include a dirty lens to change which portion of a captured image is affected by dirt on the lens.

7. The method of claim 1, wherein capturing said first and second images includes operating said first and second optical chains at the same time to capture said first and second images, respectively.

8. A method, the method comprising:

capturing a first image using a first lens of a first optical chain of a device including a plurality of different optical chains, said first optical chain including a first sensor;

capturing a second image using a second lens of a second optical chain, said second optical chain including a second sensor;

capturing a third image using a third lens of a third optical chain, said third optical chain including a third sensor;

performing a first comparison of said first image or a characteristic of said first image to the second image or a characteristic of said second image;

performing a second comparison, said second comparison being a comparison of said third image or a characteristic of said third image to another image or a characteristic of said another image, said another image being either the first image or the second image;

making a decision as to whether said dirty camera lens condition exists based on said first comparison and said second comparison;

storing information indicating which of the plurality of optical chains includes a dirty lens; and

using dirty lens information as part of a composite image generation process, an image indicated to correspond to a dirty lens being treated differently during an image combining process performed during said composite image generation process than an image corresponding to a clean lens.

9. The method of claim 8, wherein said decision is a decision that said dirty camera lens condition exists when either said first comparison or said second comparison indicates a difference above a threshold.

10. The method of claim 9,

wherein said first comparison is a comparison of an average image contrast of said first image to an average image contrast of said second image;

and

wherein said second comparison is a comparison of an average image contrast of said third image to the average image contrast of said first image or the average image contrast of said second image.

11. The method of claim 8, further comprising:

in response to determining that a dirty lens condition exists, generating a dirty lens condition notification or initiating an automatic camera lens cleaning operation.

12. The method of claim 11, wherein said device is a handheld camera including said first, second and third optical chains.

13. A system comprising:

a first optical chain including a first lens for capturing a first image;

a second optical chain including a second lens;

a third optical chain including a third lens; and

a dirty lens determination module configured to;

perform a first comparison of said first image or a characteristic of said first image to the second image or a characteristic of said second image;

perform a second comparison, said second comparison being a comparison of said third image or a characteristic of said third image to another image or a characteristic of said another image, said another image being either the first image or the second image;

determine, based on at least one of said first and second comparisons, that a dirty camera lens condition exists when one of the first, second or third lenses is a dirty lens; and

combine captured images to generate a composite image, an image captured using the dirty lens being treated differently during the image combining process used to generate the composite image than an image captured using a clean lens.

14. The system of claim 13, wherein said dirty lens determination module includes a comparison module configured to, as part of said determining if a dirty camera lens condition exists, perform a first comparison of said first image or a characteristic of said first image to a second image or a characteristic of said second image;

and

a decision module configured to, as part of said determining if dirty camera lens condition exists, make a decision as to whether said dirty camera lens condition exists based on the results of said first comparison.

15. The system of claim 14,

wherein said first comparison is a comparison of a first image metric corresponding to said first image and a second image metric corresponding to said second image, the first and second image metrics corresponding to an overlapping image region;

and

wherein determining, based on at least one of said first and second comparisons, if the dirty camera lens condition exists includes determining that a dirty camera lens exists when the first comparison indicates a difference in the first and second image metrics above a predetermined threshold.

16. The system of claim 14,

wherein said first lens is an outer most lens of the first optical chain, the outermost lens being the lens furthest from a sensor in the first optical chain.

17. The system of claim 15, further including a memory storing information identifying which particular lens or lenses have been determined to be dirty.

18. The device of claim 14, wherein said decision is a decision that said dirty camera lens condition exists when either said first comparison or said second comparison indicates a difference above a threshold.

19. The system of claim 14,

wherein said first comparison is a comparison of an average image contrast of said first image to an average image contrast of said second image;

and

wherein said second comparison is a comparison of an average image contrast of said third image to the average image contrast of said first image or the average image contrast of said second image.

20. A non-transitory computer readable medium comprising computer executable instructions, said non-transitory computer readable medium comprising:

instructions, which when executed by a processor, cause the processor to;

control a first optical chain having an outermost lens to capture a first image;

control a second optical chain including a second outermost lens to capture a second image;

control a third optical chain including a third outermost lens to capture a third image;

perform a first comparison of said first image or a characteristic of said first image to the second image or a characteristic of said second image;

perform a second comparison, said second comparison being a comparison of said third image or a characteristic of said third image to another image or a characteristic of said another image, said another image being either the first image or the second image;

determine, based on at least one of said first and second comparisons, that a dirty camera lens condition exists, at least one of said first, second and third lenses having been determined to be dirty when said dirty camera lens condition exists, said at least one lens being a dirty lens;

associate dirty lens information with an image captured using said dirty lens; and

combine captured images to generate a composite image, the image captured using the dirty lens being treated differently during the image combining process used to generate the composite image than an image captured using a clean lens.