Power efficient image sensing apparatus, method of operating the same and eye/gaze tracking system

• US 9,509,910 B2
• Filed: 04/10/2014
• Issued: 11/29/2016
• Est. Priority Date: 04/29/2013
• Status: Active Grant

• Current Assignee: Tobii Technology AB
• Original Assignee: Tobii Technology AB
• Inventors: Skogo, Marten, Jonsson, Henrik, Karlsson, Mattias O., Kuldkepp, Mattias, Elvesjo, John, Karlsson, Ingemar Mattias
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US14/250,142
• Publication Number:
  US 20140320688A1
• Time in Patent Office: 964 Days
• Field of Search: 348/333.03, 348/294, 348/302, 382/116, 382/117, 382/118
• US Class Current: 1/1
• CPC Class Codes: G03B 2213/025 Sightline detection
• CPC Class Codes: G06F 1/325 Power saving in peripheral device
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/651 for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/745 Circuitry for generating timing or clock signals
• CPC Class Codes: H04N 25/75 Circuitry for providing, modifying or processing image signals from the pixel array

Abstract

An image sensing apparatus has a light sensitive area and a control unit. The light sensitive area registers image data in response to an incoming amount of light, and is operable in an active mode wherein image data can be read out there from as well as in a standby mode wherein image data cannot be read out. The control unit produces a control signal setting the light sensitive area to operate in the active mode and the standby mode respectively in a cyclic manner during an operation period, which preferably encompasses multiple data frame read-outs of image data from the light sensitive area.

Claims

1. An image sensing apparatus comprising:

a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out; and

a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, said control unit configured to produce the control signal such that the light sensitive area operates in the active mode and the standby mode in a cyclic manner during an operation period, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, separated by a respective delay period during which the light sensitive area operates in the standby mode.

2. The image sensing apparatus according to claim 1, comprising an output interface configured to deliver output image data from the light sensitive area, wherein

one data frame represents readout of the image data having been registered by the light sensitive area during a capture time, and

the light sensitive area is configured to feed a sequence of said data frames through the output interface, and

the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

3. The image sensing apparatus according to claim 1, wherein

the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows,

the light sensitive area is controllable to read out image data to an output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and

the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode.

4. The image sensing apparatus according to claim 1, wherein prior to the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area.

5. The image sensing apparatus according to claim 1, wherein

the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and

the light sensitive area is controllable to read out image data to an output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.

6. An eye/gaze tracking system comprising an image sensing apparatus, said image sensing apparatus comprising:

a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and wherein the light sensitive area is controllable to read out image data to an output interface, which image data have been registered exclusively by at least one sub-set of the first number of columns and/or the second number of rows; and

a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, said control unit configured to produce the control signal such that the light sensitive area operates in the active mode and the standby mode in a cyclic manner during an operation period, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, separated by a respective delay period during which the light sensitive area operates in the standby mode, and wherein the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode,

wherein the image sensing apparatus being arranged to capture image data of a subject whose eye and/or and or gaze is tracked, and

wherein the control unit is configured to produce the control signal so that said at least one subset of image data represents the image of at least one eye of the subject.

7. A method of controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, the method comprising:

producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively; and

producing the control signal such that the light sensitive area operates in the active mode and the standby mode in a cyclic manner during an operation period, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, separated by a respective delay period during which the light sensitive area operates in the standby mode.

8. The method according to claim 7, wherein the image sensing apparatus comprises an output interface configured to deliver output image data from the light sensitive area, and the method further comprising:

reading out one data frame representing image data having been registered by the light sensitive area during a capture time; and

feeding out a sequence of said data frames from the light sensitive area through the output interface, and

producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

9. The method according to claim 8, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises:

reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows; and

producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode.

10. The method according to claim 8, comprising, prior to the operation period, producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area.

11. The method according to claim 7, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises:

reading out image data from the light sensitive area to an output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.

12. A computer program product stored on a non-transitory computer readable medium, comprising software instructions for operation by a computer controller for controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, said instructions comprising:

first instructions for producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively; and

second instructions for producing the control signal such that the light sensitive area operates in the active mode and the standby mode in a cyclic manner during an operation period, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, separated by a respective delay period during which the light sensitive area operates in the standby mode.

13. The computer program product of claim 12, wherein the image sensing apparatus comprises an output interface configured to deliver output image data from the light sensitive area, and the computer program code further comprising:

third instructions for reading out one data frame representing image data having been registered by the light sensitive area during a capture time; and

fourth instructions for feeding out a sequence of said data frames from the light sensitive area through the output interface, and

wherein said second instructions includes instructions for producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

14. The computer program product of claim 13, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, wherein

said third instructions comprises instructions for reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows;

and

said second instructions comprises instructions for producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode.

15. The computer program product of claim 13, wherein said second instructions comprising, prior to the operation period, instructions for producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area.

16. The computer program product of claim 12, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises:

third second instructions for reading out image data from the light sensitive area to an output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.

17. An image sensing apparatus comprising:

a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out;

a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, in a cyclic manner during an operation period, wherein the operation period extends over at least one capture time where the light sensitive area operates in the active mode, and

wherein prior to the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area.

18. The image sensing apparatus according to claim 17, comprising an output interface configured to deliver output image data from the light sensitive area, wherein

one data frame represents readout of the image data having been registered by the light sensitive area during a capture time,

the light sensitive area is configured to feed a sequence of said data frames through the output interface, and

the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

19. The image sensing apparatus according to claim 17, wherein one data frame represents readout of the image data having been registered by the light sensitive area during a capture time, and wherein

the light sensitive area is configured to feed a sequence of said data frames through the output interface;

and

the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

20. The image sensing apparatus according to claim 19, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode.

21. The image sensing apparatus according to claim 17, wherein

the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows,

the light sensitive area is controllable to read out image data to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and

the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode.

22. The image sensing apparatus according to claim 17, wherein

the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and

the light sensitive area is controllable to read out image data to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.

23. An eye/gaze tracking system comprising an image sensing apparatus, said image sensing apparatus comprising:

a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and wherein the light sensitive area is controllable to read out image data to an output interface, which image data have been registered exclusively by at least one sub-set of the first number of columns and/or the second number of rows; and

a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, in a cyclic manner during an operation period, wherein the operation period extends over at least one capture time where the light sensitive area operates in the active mode, wherein the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode, and wherein prior to the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area,

wherein the image sensing apparatus being arranged to capture image data of a subject whose eye and or gaze is tracked, and

wherein the control unit is configured to produce the control signal so that said at least one subset of image data represents the image of at least one eye of the subject.

24. A method of controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out;

the method comprising;

producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively;

producing the control signal such that the light sensitive area operates in the active mode and the standby mode in a cyclic manner during an operation period; and

prior to the operation period, producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area.

25. The method according to claim 24, wherein the image sensing apparatus comprises an output interface configured to deliver output image data from the light sensitive area, and the method further comprising:

reading out one data frame representing image data having been registered by the light sensitive area during a capture time,

feeding out a sequence of said data frames from the light sensitive area through the output interface, and

producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

26. The method according to claim 25, comprising producing the control signal so that the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode.

27. The method according to claim 25, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises:

reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows; and

producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode.

28. The method according to claim 24, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises:

reading out image data from the light sensitive area to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.

33. The computer program product of claim 24, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises:

third instructions for reading out image data from the light sensitive area to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.

29. A computer program product stored on a non-transitory computer readable medium, comprising software instructions for operation by a computer controller for controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, said instructions comprising:

first instructions for producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively;

second instructions for producing the control signal such that the light sensitive area operates in the active mode and the standby mode in a cyclic manner during an operation period; and

third instructions for, prior to the operation period, producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area.

30. The computer program product of claim 29, wherein the image sensing apparatus comprises an output interface configured to deliver output image data from the light sensitive area, and the computer program code further comprising:

fourth instructions for reading out one data frame representing image data having been registered by the light sensitive area during a capture time; and

fifth instructions for feeding out a sequence of said data frames from the light sensitive area through the output interface, and

wherein said second instructions includes instructions for producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time.

31. The computer program product of claim 30, wherein said second instructions comprises instructions for producing the control signal so that the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode.

32. The computer program product of claim 30, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, wherein

said fourth instructions comprises instructions for reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and

said second instructions comprises instructions for producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode.