Multiple Camera Control System

• US 20080056536A1
• Filed: 10/31/2007
• Published: 03/06/2008
• Est. Priority Date: 10/03/2000
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: MacDougall, Francis, Hildreth, Evan
• Granted Patent: US 7,555,142 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A multiple camera tracking system for interfacing with an application program running on a computer is provided. The tracking system includes two or more video cameras arranged to provide different viewpoints of a region of interest, and are operable to produce a series of video images. A processor is operable to receive the series of video images and detect objects appearing in the region of interest. The processor executes a process to generate a background data set from the video images, generate an image data set for each received video image, compare each image data set to the background-data set to produce a difference map for each image data set, detect a relative position of an object of interest within each difference map, and produce an absolute position of the object of interest from the relative positions of the object of interest and map the absolute position to a position indicator associated with the application program.

Claims

1. A computer-implemented method comprising:

acquiring first and second images each capturing a different view of an object and a background;

detecting, for each of the first and second images, a difference between a filtered luminance characteristic and a background model;

determining a position of the object based on the detected differences;

controlling an application based on the determined position of the object.

2. The method of claim 1, wherein the background is a controlled background.

3. The method of claim 2, wherein the controlled background comprises a strip of light.

4. The method of claim 2, wherein the controlled background comprises a surface, wall or window frame having arbitrary luminance, textures or edges.

5. The method of claim 2, wherein the controlled background exhibits a predetermined pattern or texture.

6. The method of claim 5, wherein the predetermined pattern is a stripe pattern.

7. The method of claim 5, wherein detecting, for each of the first and second images, the occlusion of the background further comprises:

detecting, for each of the first and second images, a region that does not exhibit the predetermined pattern or texture; and

outputting, for each of the first and second images, data associated with the region as the occlusion.

8. The method of claim 1, wherein the first and second images are acquired using first and second cameras, respectively, and

wherein the determined position of the object is a relative position with respect to the first and second cameras, respectively.

9. The method of claim 1, wherein the determined position of the objection is an absolute position.

10. The method of claim 1, further comprising detecting a gesture of the object, wherein the application is controlled further based on the gesture.

11. The method of claim 1, wherein detecting, for each of the first and second images, the difference between the filtered luminance characteristic and the background model further comprises filtering the first and second images to extract the luminance characteristic of the first and second images where the background and the object differ, but are otherwise invariant within the background over time.

12. The method of claim 1, further comprising processing the first and second images to exclude pixels outside bounds of the background.

13. The method of claim 1, wherein the first and second images each comprise a single row of pixels.

14. The method of claim 1, wherein detecting, for each of the first and second images, the difference between the luminance characteristic and the background model further comprises:

tagging a pixel of the first or second image to a first state if the pixel is consistent with the background model; and

tagging the pixel to a second state if the pixel is inconsistent with the background model.

15. The method of claim 1, further comprising providing, as the background, a strip of light that is capable of being detected in the first and second images.

16. A device comprising:

means for acquiring first and second images each capturing a different view of an object and a background;

means for detecting, for each of the first and second images, a difference between a filtered luminance characteristic and a background model;

means for determining a position of the object based on the detected differences;

means for controlling an application based on the determined position of the object.

17. A device comprising:

a camera configured to acquire first and second images each capturing a different view of an object and a background; and

a processor configured to;

detect, for each of the first and second images, a difference between a filtered luminance characteristic and a background model,

determine a position of the object based on the detected differences, and

control an application based on the determined position of the object.

18. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire first and second images each capturing a different view of an object and a background;

detect, for each of the first and second images, a difference between a filtered luminance characteristic and a background model;

determine a position of the object based on the detected differences; and

control an application based on the determined position of the object.

19. A computer program product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire first and second images each capturing a different view of an object;

filter a luminance characteristic of each of the first and second images;

determine, based on filtering the luminance characteristic, a position of the object; and

control an application based on the determined position of the object.

20. A computer program product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire first and second images each capturing a different view of an object;

detect a difference between the first and second images;

determine a position of the object based on the difference; and

control an application based on the determined position of the object.

21. A computer program product tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire first and second images each capturing a different view of an object and a background, the background being oriented parallel to a reference plane;

filter a luminance characteristic of each of the first and second images;

determine, based on filtering the luminance characteristic, a position of the object perpendicular to the reference plane; and

control an application based on the determined position of the object.

22. A computer program product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

define first and second planes oriented parallel with a reference plane;

acquire first and second images each capturing a different view of an object and a background;

filter a luminance characteristic of each of the first and second images;

determine, based on filtering the luminance characteristic, a state of the object based on a position of the object perpendicular to the reference plane in relation to the first and second planes; and

generate first or second user inputs to an application if the object is determined to be in first or second states, respectively.

23. A computer program product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire an image of a view of an object;

generate a background model, further comprising;

adding the image to a buffer of images of the view;

selecting a value representative of a background for each portion of the view;

outputting, as the background model, the selected values;

determine a position of the object based on comparing the acquired image with the background model; and

control an application based on the determined position of the object.

24. A computer program;

product, tangibly embodied in a machine readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to;

acquire first through fourth images, the first and second images each capturing different views of an object at a first position, and the third and fourth images each, capturing different views of the object at a second position;

filter a luminance characteristic of each of the first through fourth images;

determine, based on filtering the luminance characteristic, a motion of the object; and

control an application based on the determined motion.

25. A computer-implemented method of tracking an object of interest, the method comprising:

acquiring a first image and a second image representing different viewpoints of the object of interest;

processing the first image into a first image data set and the second image into a second image data set;

processing the first image data set and the second image data set to generate a background data set associated with a background;

generating a first difference map by determining differences between the first image data set and the background data set and a second, difference map by determining differences between the second image data set and the background data set;

detecting a first relative position of the object of interest in the first difference map and a second relative position of the object of interest in the second difference map; and

producing an absolute position of the object of interest from the first and second relative positions of the object of interest.

26. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire a first image capturing a view of an object and a background;

detect, for the first image, an occlusion of the background;

determine a position of the object based on the detected occlusion; and

control an application based on the determined position of the object and further based on a filtered luminance characteristic of the first image.