METHOD OF DETERMINING THE SPATIAL RELATIONSHIP OF AN EYE OF A PERSON WITH RESPECT TO A CAMERA DEVICE

• US 20070171369A1
• Filed: 01/10/2007
• Published: 07/26/2007
• Est. Priority Date: 01/16/2006
• Status: Active Grant

• Current Assignee: Alcon Pharmaceuticals Limited (Alcon Incorporated)
• Original Assignee: SensoMotoric Instruments GmbH (Apple Inc.)
• Inventors: Grundig, Martin
• Granted Patent: US 7,600,873 B2
• Time in Patent Office: Days
• US Class Current: 351/210
• CPC Class Codes: A61B 3/0025 characterised by electronic signal processing, e.g. eye models
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06V 40/19 Sensors therefor

Abstract

A method of determining the spatial relationship of an eye (10) of a person with respect to a camera device (20) which provides images of the eye (10) comprises:

• • a model acquisition phase in which a customized model of the eye (10) is constructed and a reference spatial relationship of the eye model with respect to the camera device (20) is determined using a reference image of the eye (10); and
  • a tracking phase in which position and/or rotation coordinates of the eye (10) are determined by aligning the eye model to a current image of the eye (10).

Claims

1. A method of determining the spatial relationship of an eye (10) of a person with respect to a camera device (20) which provides images of the eye (10), characterized in that the method comprises:

a model acquisition phase in which a customized model of the eye (10) is constructed and a reference spatial relationship of the eye model with respect to the camera device (20) is determined using a reference image of the eye (10); and

a tracking phase in which position and/or rotation coordinates of the eye (10) are determined by aligning the eye model to a current image of the eye (10).

2. The method according to claim 1, characterized in that the eye model comprises:

a three-dimensional eye shape model representing the surface of the eye (10) and being aligned with the reference image of the eye (10) during the model acquisition phase; and

texture information associated with a plurality of points located on the surface of the eye (10).

3. The method according to claim 2, characterized in that the three-dimensional eye shape model comprises a superposition of two ellipsoids, one of which represents the globe (12) and the other of which represents the cornea (16).

4. The method according to claim 3, characterized in that at least one of the two ellipsoids is a sphere.

5. The method according to claim 2, characterized in that the three-dimensional eye shape model comprises a wire mesh structure of connected points.

6. The method according to claim 2, characterized in that the three-dimensional eye shape model comprises a plurality of points defined by linear combinations of shape eigenvectors.

7. The method according to claim 2, characterized in that the reference spatial relationship of the eye shape model with respect to the camera device (20) is obtained by applying a purkinje analysis to the reference image of the eye (10) based on corneal reflexes of illumination sources (32) located at predefined positions relative to the camera device (20).

8. The method according to claim 2, characterized in that the reference spatial relationship of the eye shape model with respect to the camera device (20) is obtained by assuming that the person is fixating at a fixed position in space previously defined relative to the camera device (20).

9. The method according to claim 2, characterized in that the texture information is stored as a feature template map comprising:

three-dimensional model coordinates of a plurality of reference image points at which eye features are located; and

templates containing parts of the reference image extracted around each associated point and showing the corresponding eye feature.

10. The method according to claim 9, characterized in that the features are selected from the group consisting of blood vessels, iris features, limbus, limbus centre, pupil centre, pupil edge and artificial markers.

11. The method according to claim 9, characterized in that the three-dimensional model coordinates of a point are obtained by detecting the corresponding eye feature in the reference image of the eye (10) and intersecting a projecting ray (40) of the eye feature with the aligned eye shape model.

12. The method according to claim 9, characterized in that the tracking phase comprises the following steps:

d) a template matching step (S10) in which, for a plurality of feature templates stored in the feature template map, a region in the current eye image bearing the largest resemblance to the respective feature template is searched;

e) a coordinate determination step (S20) in which the coordinates of the regions found in step a) are determined as respective current feature positions; and

f) an alignment step (S30) in which an image distance between the current feature positions determined in step b) and the positions of the corresponding three-dimensional model features projected into the current eye image is minimized by fitting orientation and/or rotation parameters of the eye model.

13. The method according to claim 12, characterized in that for each feature template stored in the feature template map, the search conducted during the template matching step (S10) is limited to a predefined zone of the current eye image around the feature position determined in a previous coordinate determination step (S20).

14. The method according to claim 2, characterized in that the texture information is stored as a gray value map comprising:

three-dimensional model coordinates of a plurality of previously defined reference image points; and

gray values extracted from the reference image at each of the points.

15. The method according to claim 14, characterized in that the three-dimensional model coordinates of a point are obtained by intersecting a projecting ray (40) of the reference image point with the aligned eye shape model.

16. The method according to claim 14, characterized in that the tracking phase comprises an alignment step (S130) in which the total gray value difference between the current eye image and the gray value map projected into the current eye image is minimized by fitting orientation and/or rotation parameters of the eye model.

17. The method according to claim 1, characterized in that the tracking phase is continuously repeated in regular time intervals.

18. The method according to claim 1, characterized in that it furthermore comprises a step of determining a spatial relationship of the person'"'"'s eye with respect to a surgical device (26) based on a previously determined spatial relationship of the camera device (20) with respect to the surgical device (26).

19. The method according to claim 1, characterized in that the spatial tracking phase furthermore comprises determining internal degrees of freedom of the eye (10) model selected from the group consisting of relative movements of eye model features, scaling of the whole eye model, scaling of eye model features, deformations of the whole eye model, deformations of eye model features and appearance changes due to illumination influences.

20. A computer program, comprising:

computer program code which, when executed on a computer (22) connected to a camera device (20), enables the computer (22) to carry out a method according to claim 1.