Eye feature identification

• US 10,043,075 B2
• Filed: 11/19/2015
• Issued: 08/07/2018
• Est. Priority Date: 11/19/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bennett, Michael Martin, Hitchcock, Gregory, Larson, Kevin, Linnerud, Paul, Matskewich, Tanya, McKaughan, Robert Matthew
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US14/946,591
• Publication Number:
  US 20170147858A1
• Time in Patent Office: 992 Days
• Field of Search: None
• CPC Class Codes: G02B 5/0257 creating an anisotropic diffusion characteristic, i.e. distributing output differently in two perpendicular axes
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20004 Adaptive image processing
• CPC Class Codes: G06T 2207/20208 High dynamic range [HDR] image processing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G06V 40/197 Matching; Classification
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

An image of an eye is obtained via a camera. A multi-step filter is applied to the image for multiple iterations. Applying the multi-step filter includes, for each iteration, performing one or more pixel merge operations on the image. The pixel merge operations are controlled based on one or more input parameters to control whether or not the iteration classifies pixels of the image as corresponding to a feature of the eye. The one or more input parameters vary from at least one iteration to another. The iterations each output a provisional output, in which some pixels of the image are deemed as corresponding to the feature of the eye. The provisional outputs provide diverse definitions of the eye feature, and may be combined in various ways to yield a refined output, in which some pixels of the image are deemed as corresponding to the feature of the eye.

Claims

1. A computing system comprising:

a camera configured to image an eye;

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

obtain, via the camera, an image of the eye;

apply a multi-step filter to the image of the eye for a plurality of iterations, wherein applying the multi-step filter includes, for each iteration,

(i) selecting one or more input parameters to one or more merge operations that adjust a pixel value of one or more pixels of the image of the eye, for use during the iteration, such selection being performed so that the one or more input parameters varies relative to at least one other iteration;

(ii) performing the one or more pixel merge operations on the image of the eye, using the one or more input parameters selected for the iteration, where at least one of the pixel merge operations is controlled based on the one or more input parameters so as to control whether or not the iteration classifies pixels as corresponding to a feature of the eye; and

(iii) outputting, based on operation of the one or more pixel merge operations, a provisional output in which some pixels of the image of the eye are deemed as corresponding to the feature of the eye; and

output, via combining the provisional outputs, a refined output in which some pixels of the image of the eye are deemed as corresponding to the feature of the eye.

2. The computing system of claim 1 where the one or more pixel merge operations make use of an anisotropic diffusion filter configured to apply a degree of blur to the image of the eye, where the degree of blur varies for at least some of the iterations in response to the variation in the one or more input parameters.

3. The computing system of claim 2, where the degree of blur is varied from at least one iteration to at least one other iteration based on a number of iterations that are to be used to reach the refined output.

4. The computing system of claim 2, where the degree of blur for at least one of the iterations is selected based on a property of the image.

5. The computing system of claim 2, where the one or more pixel merge operations include a contrast stretching operation.

6. The computing system of claim 1, where the one or more pixel merge operations include a contrast stretching operation configured to consolidate pixels from the image having values satisfying one or both of (i) being above an upper threshold;

and (ii) being below a lower threshold.

7. The computing system of claim 6, where for at least one of the iterations, selecting one or more input parameters for the iteration includes selecting one or both of the upper cutoff and the lower cutoff.

8. The computing system of claim 1 where the image of the eye is an RGB image, and wherein the multi-step filter is applied to only a red channel of the RGB image of the eye.

9. The computing system of claim 1, where the instructions are further executable to perform a glint removal operation on the image of the eye.

10. The computing system of claim 1 where the instructions are further executable to perform a glare removal operation on the image of the eye.

11. The computing system of claim 10, where the image of the eye is an RGB image, and where performing the glare removal operation includes (i) generating a glare difference map that is an average difference of an image parameter value of pixels in different color channels of the RGB image;

and (ii) subtracting the glare difference map from the image parameter value of pixels in each color channel of the RGB image.

12. The computing system of claim 1, where the instructions are further executable to identify a limbus boundary in the image of the eye and constrain application of the multi-step filter to pixels within the limbus boundary.

13. On a computing device, a method for identifying an eye feature, the method comprising:

obtaining, via a camera, an image of an eye;

applying a multi-step filter to the image of the eye for a plurality of iterations, wherein applying the multi-step filter includes, for each iteration,

(i) selecting one or more input parameters to one or more merge operations that adjust a pixel value of one or more pixels of the image of the eye, for use during the iteration, such selection being performed so that the one or more input parameters varies relative to at least one other iteration;

(ii) performing the one or more pixel merge operations on the image of the eye, using the one or more input parameters selected for the iteration, where at least one of the pixel merge operations is controlled based on the one or more input parameters so as to control whether or not the iteration classifies pixels as corresponding to a feature of the eye; and

(iii) outputting, based on operation of the one or more pixel merge operations, a provisional output in which some pixels of the image of the eye are deemed as corresponding to the feature of the eye; and

outputting a refined output via combining the provisional outputs, where in the refined output, some pixels of the image of the eye are deemed as corresponding to the feature of the eye.

14. The method of claim 13, where performing one or more pixel merge operations includes using an anisotropic diffusion filter configured to apply a degree of blur to the image of the eye, where the degree of blur varies for at least some of the iterations in response to the variation in the one or more input parameters.

15. The method of claim 14, where the degree of blur is varied from at least one iteration to at least another iteration based on a number of iterations that are to be used to reach the refined output.

16. The method of claim 14, where the degree of blur for at least one of the iterations is selected based on a property of the image.

17. The method of claim 14, where performing one or more pixel merge operations includes performing a contrast stretching operation.

18. The method of claim 13, where performing one or more pixel merge operations includes performing a contrast stretching operation.

19. A computing system comprising:

a camera configured to image an eye;

a logic machine; and

a storage machine holding instructions executable by the logic machine to;

obtain, via the camera, an image of the eye;

apply a multi-step filter to the image of the eye for a plurality of iterations, wherein applying the multi-step filter includes, for each iteration,

(i) applying an anisotropic diffusion filter to the image of the eye, the anisotropic diffusion filter configured to apply a degree of blur to the image of the eye, where the degree of blur used for at least one of the iterations varies from the degree of blur used for at least one other iteration;

(ii) performing a contrast stretching operation on the image of the eye;

(iii) outputting, based on operation of the anisotropic diffusion filter and the contrast stretching operation, a provisional output in which some pixels of the image of the eye are deemed as corresponding to the feature of the eye; and

output a refined output via combining the provisional outputs, where in the refined output, some pixels of the image of the eye are deemed as corresponding to the feature of the eye.

20. The computing system of claim 19, where the degree of blur applied by the anisotropic diffusion filter is varied based on one or both of (1) a number of iterations that are to be used to reach the refined output;

and (2) a property of the image.