Computer vision system based upon solid state image sensor

• US 4,748,502 A
• Filed: 08/18/1986
• Issued: 05/31/1988
• Est. Priority Date: 08/18/1986
• Status: Expired due to Fees

• Current Assignee: Sentient Systems Technology, Inc. (Tobii Dynavox AB)
• Original Assignee: Sentient Systems Technology, Inc. (Tobii Dynavox AB)
• Inventors: Kiliany, Gary J., Friedman, Mark B.
• Primary Examiner(s): Ng, Jin F.
• Assistant Examiner(s): Brinich, Stephen
• Application Number:
  US06/897,497
• Time in Patent Office: 652 Days
• Field of Search: 358/106, 358/139, 358/163, 358/10, 358/209, 358/212, 358/213.11, 358/213.29, 358/213.31, 358/93, 382/8, 351/209-210
• US Class Current: 348/78
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A system for computer vision in association with a computer comprises a solid state image sensor or optic ram, a lens, a clock circuit for sequentially addressing the image sensor, and an edge detect circuit for detecting edges in the output bit stream of the image sensor. The edge detect circuit generates an interrupt signal for application to the clock circuit to stop the clock from advancing the addresses and for application to the computer to signal it to read the column and row address currently generated by the clock circuit.

Claims

1. A system for computer vision based upon a solid state image sensor used in association with a host computer comprising:

(a) a solid state image sensor analogous to a memory device, said sensor comprising a two-dimensional array of light sensitive memory cells arranged in a plurality of rows and columns, each cell being accessible for reading, refreshing, and writing to set each cell to an initial signal value, the signals on said cells decaying at a rate related to light intensity thereupon, the output upon reading any cell being a signal indicative of darker if the signal on the cell has not decayed to a preset threshold or a signal indicative of lighter if the signal at said cell has decayed to said threshold,

(b) a lens for focussing a light image on said image sensor,

(c) clock means for continuously generating column address signals and row address signal for each cell during sequential accessing of each cell, said clock means having row address and column address buffers that can be read to a computer data bus, and

(d) edge detecting means for processing an output bit stream from said image sensor when each cell is being sequentially accessed for detecting a change in the value of the bits indicative of an edge between lighter and darker areas of the light image focussed on said array, said edge detecting means including means for generating an interrupt signal for application to the computer whereby the computer can read the column and row address of the edge by reading the column and row buffers.

2. A system according to claim 1 further comprising a control means for causing each row of said image sensor to be at one time in at least one of three modes comprising a "refresh mode" wherein the signal on each cell is continuously refreshed, a "read" mode wherein each cell is read by accessing in a substantially sequential manner with the output bit stream being passed to the edge detecting means and a "soak" mode wherein the signals on the cells are allowed to decay the "read" mode being selected for detecting edges and transferring row and column address signals to the computer, the "refresh" mode being selected for maintaining the signals on said cells, and the "soak" mode being selected for transferring the light image to the image sensor.

3. A system according to claim 2 wherein the control means includes means for in the refresh mode accessing one cell from each row to cause the refresh of the entire row and wherein the read mode each cell is restored to its initial signal value indicative of darker.

4. A system according to claim 1 wherein the clock means includes means for addressing the rows in interleaved sets such that each set can be treated as a different image sensor.

5. A system according to claim 2 wherein the clock means includes means for addressing the rows in interleaved sets such that the control means can place each set in a different mode than at least one other set.

6. A system as in any of claims 1 to 5 wherein the edge detecting means includes means for outputting a signal indicative of whether the last bit in the bit stream was indicative of a darker or lighter condition of the bit stream just after each edge detection.

7. A system according to claim 2 wherein the clock means includes means to restart in response to an acknowledge signal generated by the control means.

8. A system according to claim 2 wherein the clock means includes means for providing an end of frame signal when a refresh or read has been completed for selected rows in said array.

9. A system according to claim 1 or 2 further comprising means for presetting the row count in the clock means by writing a row address to the clock means for the computer data bus.

10. The system according to claim 1 wherein the clock means comprises means for interrupting the generation of address signals in response to an interrupt signal and wherein the interrupt signal generated by the edge detector is applied to the clock means.

22. An image sensor according to claim 1 wherein the image sensor comprises more than one array, each array being spaced from the next thus establishing at least one blind area between arrays and further comprising means for mechanically shifting the solid state image sensor to alter the location of the blind area.

11. A system for computer vision based upon a solid state image sensor used in association with a host computer comprising:

(a) a solid state image sensor analogous to a memory device, said sensor comprising a two-dimensional array of light sensitive memory cells arranged in a plurality of rows and columns, each cell being accessible for reading, refreshing, and writing to set each cell to an initial signal value, the signals on said cells decaying during soak times at a rate related to light intensity thereupon, the output upon reading any cell being a signal indicative of darker if the signal on the cell has not decayed to a present threshold or a signal indicative of lighter if the signal at said cell has decayed to said threshold,

(b) a lens for focussing the light image on said image sensor,

(c) clock means for continuously generating column address signals and row address signals for each cell during sequential accessing of each cell in each row, said clock means having row address and column address buffers that can be read to a computer data bus,

(d) edge detecting means for processing an output bit stream from said image sensor when each cell is being sequentially accessed for detecting a change in the value of the bits indicative of an edge between lighter and darker areas of the light image focussed on said array, said edge detecting means including means for generating an interrupt signal for application to the clock means to stop the clock and for application to the computer whereby the computer can read the column and row address of the edge by reading the column and row buffers, and

(e) a control means for causing each row of said image sensor to be at one time in at least one of three modes comprising a "refresh mode" wherein the signal on each cell is continually refreshed, a "read" mode wherein each cell is read by accessing in a substantially sequential manner with the output bit stream being passed to the edge detecting means and a "soak" mode wherein the signals on the cells are allowed to decay, the "read" mode being selected for detecting edges and transferring row and column address signals to the computer, the "refresh" mode being selected for maintaining the signals on said cells, and the "soak" mode being selected for transferring the light image to the image sensor, and

said host computer having a data and address bus, said computer being in communication with said clock means and control means at least via its data and address buses for programming the soak times and controlling the accessing of date in said row address and column address buffers and said host computer further having an associated main memory with a stored task for interactively controlling the clock and control means to enable the computer to read the data in said buffers and to build edge address tables in said memory and to interpret the data in said tables.

12. A system according to claim 11 wherein the main memory has a stored task for continuously changing the soak times and analyzing data in said tables until the desired soak times are achieved.

13. A system according to claim 12 wherein the main memory has a stored task for increasing the soak times between building tables in memory until the brightest spot on the array is detected.

14. A system according to claim 12 wherein the main memory has a stored task for decreasing the soak times between building tables in memory until the darkest area on the array is detected.

15. A system according to claim 11 wherein the clock means comprises means for addressing rows in interleaved sets and the control means can place each set in a different mode than at least one other set.

16. A system according to claim 15 wherein the main memory has a stored task for using the data from one interleaved set of rows only to detect the brightest spot on the array.

17. An eyetracker for detecting from the image of an eye focussed thereon the center of the pupil which is the darkest portion of the image and the center of the corneal reflection which is the brightest spot on the image comprising:

(a) a display bearing indicia which may be selected with the user'"'"'s eyes,

(b) a light source,

(c) a solid state image sensor analogous to a memory device, said sensor comprising a two-dimensional array of light sensitive memory cells,

(d) a lens for focussing a light image on said image sensor,

(e) clock means for continuously generating column address signals and row address signals for each cell during sequential accessing of said memory cells,

(f) a host computer having address and data buses,

(g) edge detecting means for processing an output bit stream from said image sensor, said edge detecting means including means for generating an interrupt signal for application to the computer whereby the computer can read the column and row address of a detected edge,

(h) a control means for causing each row of said image sensor to be in at lest a "read" mode or a "soak" mode, the length of time in the soak mode determining the signal level detected, the "read" mode being selected for detecting edges and transferring row and column address signals to the host computer and the "soak" mode being selected for transferring the light image to the image sensor,

said host computer being in communication with said clock means and control means at least via its data and address buses for controlling the accessing of data from the image sensor, said host computer further having an associated main memory with a stored task for interactively controlling the clock and control means to enable the computer to read the row and column address signals to build edge address tables in said memory including a subtask for finding the corneal reflection of an eye upon which the image sensor which is the brightest spot on the array and having a subtask for finding the pupil by detecting the border between the pupil and the iris and having a subtask for finding the center of the corneal reflection and the center of the pupil and establishing the the distance and direction from center to center and having a task for correlating the distance and direction with the indicia upon the display.

18. An eyetracker according to claim 17 wherein the clock means includes means for addressing rows in interleaved sets such that each set can be in a different mode than at least one other set.

19. An eyetracker according to claim 18 wherein the control means and host computer include means for using one of said sets of rows to detect the corneal reflection and another of said interleaved sets to detect the pupil.

20. An eyetracker according to claim 17 wherein the control means and host computer include means for establishing the soak periods of different durations for each set such that one set is read while another is still soaking.

21. An eyetracker according to claim 17 wherein the clock means includes means for presetting the row count by writing a row address to the clock means from the computer whereby once the corneal reflection has been located only that portion of the array likely to contain the pupil image is read.

23. An eyetracker according to claim 17 further comprising means for controlling the intensity of the light source such that during the search for the pupil the light intensity is increased.

24. A system for computer vision based upon a solid state image sensor used in association with a host computer comprising:

(a) a solid state image sensor analogous to a memory device, said sensor comprising a two-dimensional array of light sensitive memory cells arranged in a plurality of rows and columns, each cell being accessible for reading and writing to set each cell to an initial signal value, the signals on said cells decaying at a rate related to light intensity thereupon, the output upon reading any cell being a signal indicative of darker if the signal on the cell has not decayed to a preset threshold or a signal indicative of lighter if the signals at said cell has decayed to said threshold,

(b) a lens for focussing a light image on said image sensor,

(c) clock means for continuously generating column address signals and row address signals for each cell during sequential accessing of each, said clock means having row address and column address buffers that can be read to a computer data bus,

(d) edge detecting means for processing an output bit stream from said image sensor when each cell is being sequentially accessed for detecting a change in the value of the bits indicative of an edge between lighter and darker areas of the light image focussed on said array, said edge detecting means including means for generating an interrupt signal for application to the computer whereby the computer can read the column and row address of the edge by reading the column and row buffers, and

(e) a control means for causing said image sensor to be in a "read" mode wherein each cell is being accessed in a substantially sequential manner with the output bit stream being passed to the edge detecting means or a "soak" mode wherein the signals on the cells are allowed to decay, the "read" mode being selected for transferring data to the computer and the "soak" mode being selected for transferring the light image to the image sensor.