Multiple camera control system

• US 7,421,093 B2
• Filed: 12/19/2005
• Issued: 09/02/2008
• Est. Priority Date: 10/03/2000
• Status: Expired due to Term

• Current Assignee: Qualcomm, Inc.
• Original Assignee: GestureTek Incorporated (Winning Brands Corp.)
• Inventors: MacDougall, Francis, Hildreth, Evan
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US11/304,000
• Publication Number:
  US 20060098873A1
• Time in Patent Office: 988 Days
• Field of Search: 382/103, 382/106, 382/107, 382/154, 382/192, 382/193, 382/194, 382/286, 382/291, 382/292, 348/42, 348/47, 348/48, 348/143, 348/153, 348/159, 345/863
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A multiple camera tracking system for interfacing with an application program running on a computer is provided. The tracking system includes two or more video cameras arranged to provide different viewpoints of a region of interest, and are operable to produce a series of video images. A processor is operable to receive the series of video images and detect objects appearing in the region of interest. The processor executes a process to generate a background data set from the video images, generate an image data set for each received video image, compare each image data set to the background data set to produce a difference map for each image data set, detect a relative position of an object of interest within each difference map, and produce an absolute position of the object of interest from the relative positions of the object of interest and map the absolute position to a position indicator associated with the application program.

Claims

1. A method comprising:

acquiring first and second images each capturing a different view of an object and a background;

detecting, for each of the first and second images, an occlusion of the background;

determining a position of the object based on the detected occlusions;

controlling an application based on the determined position of the object; and

filtering a luminance characteristic of each of the first and second images,

wherein detecting the occlusion of the background further comprises detecting a difference between the luminance characteristic and a background model.

2. The method of claim 1, wherein the background is a controlled background.

3. The method of claim 2, wherein the controlled background comprises a strip of light.

4. The method of claim 2, wherein the controlled background comprises a surface, wall or window frame having arbitrary luminance, textures or edges.

5. The method of claim 2, wherein the controlled background exhibits a predetermined pattern or texture.

6. The method of claim 5, wherein the predetermined pattern is a stripe pattern.

7. The method of claim 5, wherein detecting, for each of the first and second images, the occlusion of the background further comprises:

detecting, for each of the first and second images, a region that does not exhibit the predetermined pattern or texture; and

outputting, for each of the first and second images, data associated with the region as the occlusion.

8. The method of claim 1, wherein the first and second images are acquired using first and second cameras, respectively, and

wherein the determined position of the object is a relative position with respect to the first and second cameras, respectively.

9. The method of claim 1, wherein the determined position of the objection is an absolute position.

10. The method of claim 1, further comprising detecting a gesture of the object, wherein the application is controlled further based on the gesture.

11. The method of claim 1, wherein detecting, for each of the first and second images, an occlusion of the background further comprises filtering the first and second images to extract a characteristic of the first and second images where the background and the object differ, but are otherwise invariant within the background over time.

12. The method of claim 11, wherein the characteristic is a luminance characteristic.

13. The method of claim 12, wherein the characteristic is an expected background luminance characteristic, an expected object luminance characteristic, or an expected ambient luminance characteristic.

14. The method of claim 12, wherein the characteristic is a luminance variability characteristic.

15. The method of claim 1, further comprising processing the first and second images to exclude pixels outside bounds of the background.

16. The method of claim 1, wherein the first and second images each comprise a single row of pixels.

17. The method of claim 1, wherein detecting, for each of the first and second images, the occlusion of the background further comprises:

tagging a pixel of the first or second image to a first state if the pixel is consistent with the background; and

tagging the pixel to a second state if the pixel is inconsistent with the background.

18. The method of claim 1, further comprising providing, as the background, a strip of light that is capable of being detected in the first and second images.

19. A device comprising:

means for acquiring first and second images each capturing a different view of an object and a background;

means for detecting, for each of the first and second images, an occlusion of the background;

means for determining a position of the object based on the detected occlusions;

means for controlling an application based on the determined position of the object; and

means for filtering a luminance characteristic of each of the first and second images,

wherein detecting the occlusion of the background further comprises detecting a difference between the luminance characteristic and a background model.

20. A device comprising:

a camera configured to acquire first and second images each capturing a different view of an object and a background; and

a processor configured to;

detect, for each of the first and second images, an occlusion of the background,

determine a position of the object based on the detected occlusions,

control an application based on the determined position of the object; and

filter a luminance characteristic of each of the first and second images,

wherein detecting the occlusion of the background further comprises detecting a difference between the luminance characteristic and a background model.

21. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause the computer to:

acquire first and second images each capturing a different view of an object and a background;

detect, for each of the first and second images, an occlusion of the background;

determine a position of the object based on the detected occlusions;

control an application based on the determined position of the object; and

filter a luminance characteristic of each of the first and second images,

wherein detecting the occlusion of the background further comprises detecting a difference between the luminance characteristic and a background model.

22. A computer-implemented method comprising:

acquiring first and second images capturing image views of a human hand and a controlled background, using first and second cameras, respectively, the controlled background comprising a window frame that exhibits a predetermined striped background;

processing the first and second images to exclude pixels outside bounds of the controlled background;

detecting, for each of the processed first and second images, a region that does not exhibit the predetermined stripe pattern;

outputting, for each of the processed first and second images, data associated with the region, as an occlusion;

determining a relative position of the human hand with respect to the first and second cameras, based on the detected occlusions;

determining a gesture of the hand; and

controlling an application based on the determined position of the human hand and the gesture.

23. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

acquire first and second images each capturing a different view of an object and a background;

detect, for each of the first and second images, an occlusion of the background;

determine a position of the object based on the detected occlusions;

control an application based on the determined position of the object,

wherein detecting, for each of the first and second images, an occlusion of the background further comprises filtering the first and second images to extract a characteristic of the first and second images where the background and the object differ, but are otherwise invariant within the background over time.

24. A computer program product, tangibly embodied in a machine-readable medium, the computer program product comprising instructions that, when read by a machine, operate to cause data processing apparatus to:

provide, as the background, a strip of light that is capable of being detected in the first and second images;

acquire first and second images each capturing a different view of an object and a background;

detect, for each of the first and second images, an occlusion of the background;

determine a position of the object based on the detected occlusions; and

control an application based on the determined position of the object.