Multiple camera control system

• US 8,131,015 B2
• Filed: 06/29/2009
• Issued: 03/06/2012
• Est. Priority Date: 10/03/2000
• Status: Expired due to Term

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan, Macdougall, Francis
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US12/493,958
• Publication Number:
  US 20090262984A1
• Time in Patent Office: 981 Days
• Field of Search: 382/103, 382/106, 382/107, 382/154, 382/286, 382/291, 382/292, 348/42, 348/47, 348/48, 348/143, 348/153, 348/159, 345/863
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A multiple camera tracking system for interfacing with an application program running on a computer is provided. The tracking system includes two or more video cameras arranged to provide different viewpoints of a region of interest, and are operable to produce a series of video images. A processor is operable to receive the series of video images and detect objects appearing in the region of interest. The processor executes a process to generate a background data set from the video images, generate an image data set for each received video image, compare each image data set to the background data set to produce a difference map for each image data set, detect a relative position of an object of interest within each difference map, and produce an absolute position of the object of interest from the relative positions of the object of interest and map the absolute position to a position indicator associated with the application program.

Claims

1. A computer-implemented method comprising:

accessing, by a processor, a first image of a scene and a second image of the scene, wherein the first image comprises a first view of the scene and the second image comprises a second view of the scene;

analyzing, by the processor, the first image and the second image to detect an object of interest in at least one of the first image and the second image;

determining, by the processor, three-dimensional coordinates representing a location of the detected object of interest;

generating, by the processor, a history comprising the determined three-dimensional coordinates of the object of interest;

detecting, by the processor, an occurrence of a gesture made by the object of interest by analyzing the history of three-dimensional coordinates; and

controlling, by the processor, an electronic application based on detecting the occurrence of the gesture.

2. The method of claim 1, wherein the object comprises a human hand.

3. The method of claim 1, wherein the electronic application presents a visual representation of an instance of the electronic application, the visual representation comprises a selectable screen object, and controlling the electronic application comprises causing selection of the screen object based on detecting the occurrence of the gesture.

4. The method of claim 3, wherein the selectable screen object comprises one or more of a button or a visual icon.

5. The method of claim 1, wherein controlling the electronic application based on detection of the occurrence of the gesture comprises providing the electronic application with an emulation of a user-input generated by a keyboard or a joystick.

6. The method of claim 1, wherein:

the object of interest is detected in the first image and in the second image, determining three-dimensional coordinates comprises determining a first three-dimensional coordinate representing a location of the detected object of interest in the first image and a second three-dimensional coordinate representing a location of the object of interest the second image, and

the history comprises the first and second three-dimensional coordinates.

7. The method of claim 6, wherein detecting an occurrence of the gesture made by the object of interest by analyzing the history of three-dimensional coordinates comprises comparing the first three-dimensional coordinate to the second three-dimensional coordinate.

8. The method of claim 1, wherein detecting the occurrence of the gesture made by the object of interest by analyzing the history comprises analyzing the history to determine a minimum and a maximum value for each dimension, determining a difference between the minimum and maximum values, and comparing the difference to a threshold.

9. The method of claim 1, wherein the electronic application presents a three-dimensional virtual environment, the virtual environment comprises a screen object, and controlling the electronic application comprises causing a cursor to touch or allow interaction with the screen object.

10. The method of claim 1, wherein the gesture comprises a poking gesture.

11. The method of claim 1, wherein the gesture represents an operation of an input device.

12. The method of claim 11, wherein the gesture represents a clicking of a mouse.

13. The method of claim 11, wherein the gesture represents typing on a keyboard.

14. The method of claim 11, wherein the gesture represents an operation made with a joystick.

15. An apparatus comprising:

one or more processing devices and a computer-readable medium coupled to the one or more processing devices, the medium storing instructions which, when executed by the one or more processing devices, cause the one or more processing devices to;

access a first image of a scene and a second image of the scene, wherein the first image comprises a first view of the scene and the second image comprises a second view of the scene;

analyze the first image and the second image to detect an object of interest in at least one of the first image and the second image;

determine three-dimensional coordinates representing a location of the detected object of interest;

generate a history comprising the determined three-dimensional coordinates of the object of interest; and

means for detecting an occurrence of a gesture made by the object of interest by analyzing the history of three-dimensional coordinate, and controlling an electronic application based on detecting the occurrence of the gesture.

16. A device comprising:

a camera configured to acquire a first image of a scene; and

a processor configured to;

access the first image of the scene and a second image of the scene;

analyze the first image and the second image to detect an object of interest in at least one of the first image and the second image;

determine three-dimensional coordinates representing a location of the detected object of interest;

generate a history comprising the determined three-dimensional coordinates of the object of interest;

detect an occurrence of a gesture made by the object of interest by analyzing the history of three-dimensional coordinates; and

control an electronic application based on detecting the occurrence of the gesture.

17. A computer-readable medium encoded with a computer program comprising instructions that, when executed, operate to cause a processor to:

access a first image of a scene and a second image of the scene, wherein the first image comprises a first view of the scene and the second image comprises a second view of the scene;

analyze the first image and the second image to detect an object of interest in the first image and the second image;

determine three-dimensional coordinates representing a location of the detected object of interest;

generate a history comprising the determined three-dimensional coordinates of the object of interest;

detect an occurrence of a gesture made by the object of interest by analyzing the history of three-dimensional coordinates; and

control an electronic application based on detecting the occurrence of the gesture.

18. A computer-implemented method comprising:

capturing a series of images of a scene using one or more cameras;

analyzing, by a computer system, the images to detect at least one object of interest in at least one of the images;

determining, by the computer system, three-dimensional coordinates representing a location of the at least one-detected object of interest in the at least one image;

generating, by the computer system, a history comprising the determined three-dimensional coordinates of the detected object of interest;

detecting, by the computer system, an occurrence of a gesture made by at least one detected object of interest by analyzing the history of three-dimensional coordinates; and

controlling, by the computer system, an electronic application based on detecting the occurrence of the gesture.