Eyelid shape estimation

• US 10,282,611 B2
• Filed: 09/28/2018
• Issued: 05/07/2019
• Est. Priority Date: 08/21/2015
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Amayeh, Gholamreza, Kaehler, Adrian
• Primary Examiner(s): Motsinger, Sean T
• Application Number:
  US16/146,228
• Publication Number:
  US 20190034692A1
• Time in Patent Office: 221 Days
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

Systems and methods for eyelid shape estimation are disclosed. In one aspect, after receiving an eye image of an eye (e.g., from an image capture device), an eye-box is generated over an iris of the eye in the eye image. A plurality of radial lines can be generated from approximately the center of the eye-box to an upper edge or a lower edge of the eye box. Candidate points can be determined to have local maximum derivatives along the plurality of radial lines. From the candidate points, an eyelid shape curve (e.g., for an upper eyelid or a lower eyelid) can be determined by fitting a curve (e.g., a parabola or a polynomial) to the candidate points or a subset of the candidate points.

Claims

1. A method for eye tracking, comprising:

under control of a hardware processor;

generating a plurality of lines extending from a lower edge of a bounding shape, over an eye in an eye image, to an upper edge of the bounding shape;

determining candidate points on the plurality of lines;

determining parameters of a fit curve from a subset of the candidate points,

wherein the fit curve is an estimation of an eyelid shape of the eye in the eye image,

wherein a score of the fit curve is above a threshold score, and

wherein the fit curve is a preferred estimation of the eyelid shape of the eye in the eye image;

determining an eye pose of the eye in the eye image using parameters of the eyelid shape; and

tracking the eye in the eye image using the eye pose of the eye in the eye image.

2. The method of claim 1, wherein the bounding shape is rectangular, square, round, oval, or a combination thereof, in shape.

3. The method of claim 1, wherein at least one of the upper edge and the lower edge overlays a portion of an eye lid of the eye.

4. The method of claim 1, wherein the fit curve has a polynomial function form.

5. The method of claim 1, wherein the score of the fit curve is determined based on a number of the candidate points that intersect the fit curve.

6. The method of claim 1, wherein the score of the fit curve is determined based on a number of the candidate points that are within a threshold distance of the fit curve.

7. The method of claim 1, wherein determining the candidate points on the plurality of lines comprises determining the candidate points on a subset of the plurality of lines using an edge detector.

8. A wearable display system comprising:

an image capture device for tracking an eye pose of an eye of a wearer of the wearable display system;

non-transitory memory configured to store executable instructions;

a display for providing virtual image information to the wearer of the wearable display system; and

a hardware processor in communication with the non-transitory memory, the hardware processor programmed by the executable instructions to;

receive an eye image captured by the image capture device;

generate a bounding shape around an iris of the eye in the eye image, wherein a first edge of the bounding shape is approximately tangent to a limbic boundary of the eye in the eye image;

generate a plurality of lines extending to a second edge of the bounding shape;

determine an eye pose of the eye using parameters of an eyelid shape of the eye determined using a first subset of candidate points on the plurality of candidate lines; and

cause the display to provide virtual image information, determined using the eye pose of the eye, to the wearer of the HMD system.

9. The wearable display system of claim 8, wherein the plurality of lines comprises a plurality of radial lines, and wherein the plurality of radial lines extends from approximately the center of the bounding shape to the second edge of the bounding shape.

10. The wearable display system of claim 8, wherein the plurality of lines comprises a plurality of vertical lines, and wherein the plurality of vertical lines extends from a bisector line of the eye-box to the second edge of the bounding shape.

11. The wearable display system of claim 8, wherein the candidate points comprise points with local maximum derivatives along the plurality of the lines.

12. The wearable display system of claim 8, wherein the candidate points are not along a pupillary boundary or a limbic boundary.

13. The wearable display system of claim 8, wherein the hardware processor is further programmed to:

apply a morphological filter to the eye image; and

determining the plurality candidate points.

14. The wearable display system of claim 8, wherein the eyelid shape has a functional form determined using the first subset of the candidate points.

15. The wearable display system of claim 14, wherein the functional form is a parabola or a polynomial.

16. The wearable display system of claim 8, wherein the hardware processor is further programmed to:

re-determine the eyelid shape using a second subset of the candidate points

if the determined eyelid shape has a negative curvature and the determined eyelid shape is an estimation of the upper eyelid, or

if the determined eyelid shape has a positive curvature and the determined eyelid shape is an estimation of the lower eyelid.

17. A method for eye tracking, the method comprising:

under control of a hardware processor;

identifying a portion of an eye image comprising at least a portion of a pupil and an iris of an eye;

generating a plurality of lines extending to an upper edge or a lower edge of the portion of the eye image;

determining candidate points on the plurality of lines in the eye image; and

determining a first functional form of the eyelid shape using a first subset of the candidate points;

determining a preferred estimation of the eyelid shape is the first functional form of the eyelid shape if a score of the first functional form is greater than the score of a second functional form of the eyelid shape determined using a second subset of the candidate points; and

determining an eye pose of the eye in the eye image using parameters of the preferred estimation of the eyelid shape, wherein the eye pose comprises a direction toward which the eye is looking; and

tracking the eye in the eye image using the eye pose of the eye in the eye image.

18. The method of claim 17, further comprising:

randomly sampling the candidate points to generate the subset of the candidate points.

19. The method of claim 17, wherein the candidate points are determined using a filter, and wherein determining the candidate points on the plurality of lines comprises:

convolving the filter with the eye image to generate an approximation image result; and

identifying the candidate points using the approximation image result.

20. The method of claim 17, wherein identifying the portion of the eye image comprises generating a bounding box over the eye in the eye image.