Computer vision system with improved object orientation technique

• US 4,891,630 A
• Filed: 04/22/1988
• Issued: 01/02/1990
• Est. Priority Date: 04/22/1988
• Status: Expired due to Fees

• Current Assignee: Sentient Systems Technology, Inc. (Tobii Dynavox AB)
• Original Assignee: Friedman Mark B
• Inventors: Kiliany, Gary J., Friedman, Mark B.
• Primary Examiner(s): Brigance, Gerald
• Assistant Examiner(s): Brier, Jeffery A.
• Application Number:
  US07/185,062
• Time in Patent Office: 620 Days
• Field of Search: 340/706, 340/711, 340/712, 340/815.12, 340/815.13, 340/815.14, 340/825.19, 358/93, 358/103, 358/104, 358/105, 358/107, 358/125, 358/126, 250/221, 250/222.1, 250/222.2, 250/223 R, 250/223 B, 250/224, 901/46, 901/47, 364/525
• US Class Current: 345/156
• CPC Class Codes: A61B 3/113 for determining or recording eye movement
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A system for computer vision is based upon an image sensor that maps an image to memory cells in association with a computer. An object orientation and position patch is attached to an object to be observed comprising a planar substantially coplanar and non-collinear reflective locations positioned upon the patch and a reflector having the reflective properties of the surface of a regular curved surface intersecting the planar surface. The computer has a task stored in main memory for detecting and quantifying a change in orientation and position of the object from the location of the image of the orientation and position patch.

Claims

1. A system for computer vision based upon an image sensor that maps an image to memory cells to association with a computer comprising:

an object orientation and position patch for being attached to an object to be observed comprising a planar substantially nonreflective surface, at least three substantially coplanar and non-collinear reflective locations (a, b, and c) positioned upon the patch and a reflector (d) having the reflective properties of the surface of a regular curved surface intersecting the planar surface,

an image sensor comprising means for generating a two-dimensional digital representation of the image stored in an array of memory cells arranged in a plurality of rows and column, each cell having an associated row and column address and being accessible for reading,

a lens for focusing an image of the object including the orientation and position patch upon the image sensor, said lens defining an optical axis extending through the center of the lens and intersecting the image sensor near the center thereof,

a light source fixed relative to the sensor and lens and being directed outwardly therefrom;

a computer having a processor and main memory, and means for reading the row and column addresses of the memory cells corresponding to the images (a'"'"', b'"'"', and c'"'"') of reflective locations a, b, and c and an image (dls) of the light source reflected from the reflector d, and

said computer having a task stored in main memory for detecting and quantifying a change in orientation and position of the object, said task detecting changes in the position along the optical axis and in directions perpendicular thereto by computing changes in the row and column addresses of the memory cells corresponding to the images a'"'"', b'"'"', and c'"'"', and said task detecting change in rotation about axes perpendicular to the optical axes by computing differences in the row and column addresses of the memory cells corresponding to the image dls and the images a'"'"', b'"'"', and/or c'"'"'.

2. The system for computer vision according to claim 1 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the position along the optical axis by computing the average of the changes in distance between the addresses of images a'"'"', b'"'"', and c'"'"' and interpreting a decrease in the average as a movement away from the lens.

3. The system for computer vision according to claim 1 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the positions perpendicular to the optical axis by computing the average of the changes in the row or column addresses of the images a'"'"', b'"'"', and c'"'"'.

4. The system for computer vision according to claim 1 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the rotation of the object about axes perpendicular to the optical axis by computing differences in the row or column addresses of at least one of the images a'"'"', b'"'"', c'"'"' and the row or column address of image dls.

5. The system for computer vision according to claim 1 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the rotation about an axis parallel to the optical axis by observing the change in row or column addresses of at least one of the images a'"'"', b'"'"' and c'"'"' and interpreting the direction of movements of said images as tangent to concentric circles about the axis of rotation.

6. The system for computer vision according to claims 1 to 5 wherein the image sensor comprises a solid-state image sensor having a two-dimensional array of light sensitive memory cells.

7. The system according to claim 1 having multiple light sources fixed relative to the sensor and lens and means for switching only one source on at one time.

8. The system according to claim 1 having multiple image sensors fixed relative to the light source and means for switching only one sensor on at one time.

14. The body pointer according to claims 7 to wherein the image sensor comprises a solid-state image sensor having a two-dimensional array of light sensitive memory cells.

9. A body pointer based upon an image sensor used in association with a computer comprising

an orientation and position patch for being attached to the portion of the body used for pointing comprising a planar substantially nonreflective surface, at least three substantially coplanar and nonaligned reflective locations (a, b, and c) positioned upon the patch and a reflector (d) having the reflective properties of the surface of a regular curved surface mounted upon the planar surface,

an image sensor comprising means for generating a two-dimensional representation of the image stored in an array of memory cells arranged in a plurality of rows and columns, each cell having an associated row and column address and being accessible for reading,

a lens for focusing an image of the orientation and position patch upon the image sensor, said lens defining an optical axes extending through the center of the lens and intersecting the image sensor near the center thereof,

a light source fixed relative to the sensor and lens and being directed outwardly therefrom,

a computer having a processor and main memory,

means for transferring to the computer the row and column addresses of the memory cells corresponding to the images (a'"'"', b'"'"', and c'"'"') of reflective locations a, b, and c and an image (dls) of the light source reflected from the surface reflector d to the computer,

said computer having a first task stored in main memory for calibrating the body pointer for an initial orientation and position of the pointer by recording the row and column addresses of the images a'"'"', b'"'"', c'"'"' and dls,

said computer having a second task stored in main memory for detecting and quantifying a change in orientation and position of the patch, said task detecting changes in the position along the optical axis and in direction perpendicular thereto by computing changes in the row and column addresses of the image a'"'"', b'"'"', and c'"'"', and the task detecting the change in rotation about axes perpendicular to the optical axes by computing differences in the addresses of the image dls and images a'"'"', b'"'"', and/or c'"'"'.

10. The body pointer according to claim 9 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the position along the optical axis by computing the average of the changes in distance between the addresses of images a'"'"', b'"'"', and c'"'"' and interpreting a decrease in the average as a movement away from the lens.

11. The body pointer according to claim 9 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the position perpendicular to the optical axis by computing the average of the changes in the row or column addresses of the images a'"'"', b'"'"', and c'"'"'.

12. The body pointer according to claim 9 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the rotation of the patch about axes perpendicular to the optical axis by computing differences in the row or column addresses of at least one of images a'"'"', b'"'"', c'"'"' and image dls.

13. The body pointer according to claim 12 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the rotation of the patch about an axis parallel to the optical axis by observing the change in row or column addresses of at least one of images a'"'"', b'"'"', and c'"'"' and interpreting the directions of movements of said images as tangent to concentric circles about the axis of rotation.

15. The body pointer according to claim 9 having multiple light sources fixed relative to the sensor and lens and means for switching only one source on at one time.

16. The body pointer according to claim 9 having multiple image sensors fixed relative to the light source and means for switching only one sensor on at one time.

17. An eyetracker based upon an image sensor used in association with a computer comprising

a display bearing indicia which may be selected with the user'"'"'s eyes,

a head orientation and position patch for being attached to the face near an eye comprising a planar substantially nonreflective surface, at least three substantially coplanar and nonaligned reflective locations (a, b, and c) positioned upon the patch and a reflector (d) having the reflective properties of a regular curved surface mounted upon the planar surface,

an image sensor comprising means for generating a two-dimensional representation of the image stored in an array of memory cells arranged in a plurality of rows and columns, each cell having an associated row and column address and being accessible for reading,

a lens for focusing an image of the eye and the orientation and position patch upon the image sensor, said lens defining an optical axis extending through the center of the lens and intersecting the image sensor near the center thereof,

a light source fixed relative to the sensor and lens and being directed outwardly therefrom,

a computer having a processor and main memory,

means for transferring the row and column addresses of the memory cells corresponding to the images (a'"'"', b'"'"', and c'"'"') of reflective locations a, b, and c and an image (dls) of the light source reflected from the spherical surface reflector d and the row and column addresses of the image (cls) of the light source reflected from the cornea of the eye to the computer,

said computer having a first task stored in main memory or calibrating the eyetracker for an initial head orientation and position by recording the row and column addresses of the images a'"'"', b'"'"', c'"'"', dls and cls when the user gazes sequentially at least two of the indicia upon the display while the head is substantially immobile,

said computer having a second task stored in main memory for detecting and quantifying a change in orientation and position of the head, said task selecting changes in the position along the optical axis and in directions perpendicular thereto by computing changes in the row and column addresses of the images a'"'"', b'"'"', and c'"'"', and the task detecting the change in rotation about the axes perpendicular to the optical axes by computing differences in the addresses of the image dls and the images a'"'"', b'"'"', and c'"'"', and

said computer having a third task stored in main memory for correlating the row and column address of the corneal reflection image cls with indicia upon the display taking into consideration the instantaneous head position and orientation as determined by reference to the initial head orientation and position determined by the first task and the change in head position and orientation determined by the second task.

18. The eyetracker according to claim 17 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the position along the optical axis by computing the average of the changes in addresses between the images a'"'"', b'"'"', and c'"'"' upon the image sensor and interpreting a decrease in the average as a movement away from the lens.

19. The eyetracker according to claim 17 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the positions perpendicular to the optical axis by computing the average of the changes in the row or column addresses of the images a'"'"', b'"'"', and c'"'"'.

20. The eyetracker according to claim 17 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the rotation of the head about axes perpendicular to the optical axis by computing differences in the row or column addresses of at least one of images a'"'"', b'"'"', c'"'"' and image dls.

21. The eyetracker according to claim 17 wherein the task stored in main memory for detecting and quantifying a change in orientation and position detects changes in the rotation of the head about an axis parallel to the optical axis by observing the change in row, or column addresses of at least one of images a'"'"', b'"'"', and c'"'"' and interpreting the directions of movements of said images as tangent to concentric circles about the axis of rotation.

22. The eyetracker according to claims 17 to 21 wherein the image sensor comprises a solid-state image sensor having a two-dimensional array of light sensitive memory cells.

23. The eyetracker according to claim 17 having multiple light sources fixed relative to the sensor and lens and means for switching only one source on at one time.

24. The eyetracker according to claim 17 having multiple image sensors fixed relative to the light source and means for switching only one image sensor on at one time.