Eyelid shape estimation using eye pose measurement

• US 10,146,997 B2
• Filed: 08/16/2016
• Issued: 12/04/2018
• Est. Priority Date: 08/21/2015
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Kaehler, Adrian
• Primary Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US15/238,516
• Publication Number:
  US 20170053165A1
• Time in Patent Office: 840 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

Systems and methods for eyelid shape estimation are disclosed. In one aspect, after receiving an eye image of an eye (e.g., from an image capture device), an eye pose of the eye in the eye image is determined. From the eye pose, an eyelid shape (of an upper eyelid or a lower eyelid) can be estimated using an eyelid shape mapping model. The eyelid shape mapping model relates the eye pose and the eyelid shape. In another aspect, the eyelid shape mapping model is learned (e.g., using a neural network).

Claims

1. A method for eyelid shape estimation, comprising:

under control of a hardware processor;

detecting a pupillary boundary of an eye using an edge detector;

determining an eye pose of the eye using the pupillary boundary,

wherein an eye pose coordinate system of the eye pose comprises an azimuthal angle and a zenithal angle of the eye relative to a resting orientation of the eye,

wherein a functional relationship between the eye pose coordinate system and an eyelid shape coordinate system comprises a mapping matrix, and

wherein the eyelid shape coordinate system comprises a horizontal shift, a vertical shift, and a curvature of the eyelid;

estimating an eyelid shape of the eye based at least in part on the eye pose and the functional relationship, wherein the functional relationship relates the eye pose to the eyelid shape; and

fitting a parabolic curve of an eyelid shape of the eye based on the eyelid shape.

2. The method of claim 1, wherein fitting the parabolic curve comprises:

fitting a new parabolic curve of the eyelid shape of the eye based on the eyelid shape if the curvature of the parabolic curve is positive and the eyelid is an upper eyelid of the eye or if the curvature of the parabolic curve is negative and the eyelid is a lower eyelid.

3. The method of claim 1, further comprising:

applying a morphological filter to the eye image prior to detecting the pupillary boundary of the eye using the edge detector.

4. The method of claim 1, wherein the edge detector comprises a Canny edge detector.

5. The method of claim 1, wherein the mapping matrix is determined by a regression analysis of eyelid shapes and eye poses.

6. A head mounted display system comprising:

an image capture device configured to capture an eye image;

non-transitory memory configured to store an eyelid shape mapping model; and

a hardware processor in communication with the non-transitory memory, the hardware processor programmed to;

determine a pupillary boundary of an eye;

determine an eye pose of an eye in the eye image using the determined pupillary boundary,

wherein an eye pose coordinate system of the eye pose comprises an azimuthal angle and a zenithal angle of the eye relative to a resting orientation of the eye,

wherein a functional relationship between the eye pose coordinate system and an eyelid shape coordinate system comprises the eyelid shape mapping model, and

wherein the eyelid shape coordinate system comprises a horizontal shift, a vertical shift, and a curvature of the eyelid; and

estimate an eyelid shape of the eye based at least in part on the eye pose and the eyelid shape mapping model, wherein the eyelid shape mapping model relates the eyelid shape to the eye pose.

7. The head mounted display system of claim 6, wherein the eyelid shape mapping model comprises a parametric relationship between the eye pose and the eyelid shape.

8. The head mounted display system of claim 7, wherein the parametric relationship comprises a parabola.

9. The head mounted display system of claim 7, wherein the parametric relationship comprises a polynomial.

10. The head mounted display system of claim 6, wherein the eyelid shape mapping model comprises a neural network.

11. The head mounted display system of claim 6, wherein the hardware processor is further programmed to:

determine biometric data of the eye using the eyelid shape.

12. The head mounted display system of claim 11, wherein the biometric data comprises an iris code of the eye.

13. The head mounted display system of claim 6, wherein the hardware processor is further programmed to:

re-determine the eye pose based on the eyelid shape; and

verify consistency of the determined eye pose and the re-determined eye pose.