Computer vision-based object tracking system

• US 8,803,978 B2
• Filed: 05/23/2006
• Issued: 08/12/2014
• Est. Priority Date: 05/23/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): LI, TRACY Y
• Application Number:
  US11/439,453
• Publication Number:
  US 20070273766A1
• Time in Patent Office: 3,003 Days
• Field of Search: 348143-169
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 25/443 by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 25/47 Image sensors with pixel address output; Event-driven image sensors; Selection of pixels to be read out based on image data
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 3/155 Control of the image-sensor operation, e.g. image processing within the image-sensor
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A computer-implemented method for utilizing a camera device to track an object is presented. As part of the method, a region of interest is determined within an overall image sensing area. A point light source is then tracked within the region of interest. In a particular arrangement, the camera device incorporates CMOS image sensor technology and the point light source is an IR LED. Other embodiments pertain to manipulations of the region of interest to accommodate changes to the status of the point light source.

Claims

1. A computer-implemented method of utilizing a camera device to track an object, the method comprising:

utilizing the camera device in a detection mode to detect movement of a point light source associated with the object within an overall image sensing area, the camera device having a first frame acquisition rate in the detection mode;

switching from the detection mode to a tracking mode in response to the detected movement;

determining a potential range of movement of the point light source based on a location of the detected movement and a given time period for potential movement of the point light source;

determining boundaries of a region of interest within the overall image sensing area based at least in part upon the determined potential range of movement of the point light source, wherein the boundaries of the region of interest define an area that is less than the overall image sensing area in the detection mode; and

utilizing the camera device in the tracking mode, in which a frame acquisition rate of the camera device is increased to a second frame acquisition rate that is higher than the first frame acquisition rate in the detection mode, to track the point light source within the determined boundaries of the region of interest.

2. The method of claim 1, wherein the region of interest is imaged utilizing a CMOS image sensor, and wherein a frame acquisition rate of the CMOS image sensor is dependent upon a size of the region of interest and wherein tracking the light source comprises tracking an IR LED, and wherein the frame acquisition rate of the CMOS image sensor increases as the size of the region of interest decreases.

3. The method of claim 1, wherein the point light source comprises a light emitting device operable to produce light that is received by the camera device, the light emitting device being positioned such that it moves along with the object.

4. The method of claim 1 further comprising re-determining the boundaries of the region of interest when the point light source moves to a location that is beyond a predetermined threshold, and wherein the predetermined threshold is based upon a number of frames.

5. The method of claim 1, and further comprising:

determining that the point light source is not detected within the boundaries of the region of interest and in response, adjusting the boundaries to increase the size of the region of interest.

6. The method of claim 1, and further comprising:

determining that the point light source is detected within the boundaries of the region of interest and in response, adjusting the boundaries to decrease the size of the region of interest.

7. The method of claim 1 further comprising:

detecting movement of the point light source within the region of interest;

determining a movement potential of the point light source based on the detected movement and the given time period; and

re-determining boundaries of the region of interest based on the determined movement potential such that the re-determined region of interest covers the object within an area defined by the movement potential.

8. The method of claim 1 wherein the boundaries of the region of interest are calculated based on the location and the given time period to exclude, from the region of interest, areas that extend beyond the potential movement of the point light source during the given time period, and wherein, when in the tracking mode, the camera device monitors the entire area within the boundaries of the region of interest and portions of the overall image sensing area that are outside the region of interest are not monitored.

9. The method of claim 1 wherein the boundaries of the region of interest are determined using a model relative to movement of the point light source.

10. A computer vision-based object tracking system, the system comprising:

a point light emitting device that is operable to produce light and associated with an object such that the point light emitting device moves when the object moves; and

a camera device configured to receive the light produced by the point light emitting device to track the object, by tracking the point light emitting device within a region of interest that is less than an overall image sensing area, the overall image sensing area comprising a field-of-view of the camera device and the region of interest being a portion of the camera device field-of-view, wherein boundaries of the region of interest are actively adjusted based on a potential range of movement of the point light emitting device, to change a size of the region of interest in response to at least some movement of the point light emitting device the potential range of movement being determined based at least in part on a location of the point light emitting device and a time period for potential movement of the point light emitting device.

11. The system of claim 10, wherein the camera device is configured to track the point light emitting device using a CMOS image sensor, and wherein the camera device has a variable field of vision that enables it to track the point light emitting device at multiple distances from the camera device, the variable field of vision including a focus that is adjustable through imaging components.

12. The system of claim 10, wherein the point light emitting device is an IR LED, and wherein tracking the point emitting device includes changing the field of view.

13. The system of claim 10, wherein the camera device is configured to track the point light emitting device within a different region of interest when the point light emitting device moves to a location that is beyond a predetermined threshold, and wherein the camera device calculates a position of the point light emitting device within the region of interest as a weighted average of positions of bright pixels in the region of interest.

14. The system of claim 10, wherein the boundaries of the region of interest are decreased to reduce the region of interest in response to detected movement of the point light emitting device, and the boundaries of the region of interest are increased to enlarge the region of interest when the point light emitting device is not detected within the region of interest.

15. A computer-implemented method of utilizing a camera device having a field of vision oriented toward a surface to track an object having a point light source and a switch configured to control the point light source, the method comprising:

adjusting a characteristic of a region of interest based on a characteristic of the point light source, wherein the region of interest is a region within an overall image sensing area, and a frame acquisition rate of the camera device varies as a function of a size of the region of interest;

automatically activating the point light source using the switch in response to at least a portion of the object contacting the surface, wherein at least a portion of the light from the activated point light source passes through the surface to the camera device; and

utilizing the camera device to view the activated point light source through the surface while adjusting the characteristic of the region of interest.

16. The method of claim 15, wherein adjusting a characteristic of the region of interest comprises increasing the size of the region of interest when the point light source is not detected within the region of interest, and wherein the frame acquisition rate increases as the region of interest decreases.

17. The method of claim 15, wherein adjusting a characteristic of the region of interest comprises relocating the region of interest along the surface as a function of a location of the point light source and a calculated potential range of movement of the point light source within a given time frame.

18. The method of claim 15, wherein adjusting a characteristic of a region of interest comprises relocating the region of interest along the surface when the point light source moves beyond a predetermined threshold, and wherein utilizing the camera includes reducing a pixel integration time.

19. The method of claim 15, wherein the point light source is an IR LED, and wherein utilizing the camera device includes maintaining a set focus and a set field of view on the surface.

20. The method of claim 15, wherein the surface is at least partially transparent, and the portion comprises at least one of the point light source and the switch.