Method and system for processing captured image information in an interactive video display system

• US 7,536,032 B2
• Filed: 10/25/2004
• Issued: 05/19/2009
• Est. Priority Date: 10/24/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Reactrix Systems Incorporated
• Inventors: Bell, Matthew
• Primary Examiner(s): Tabatabai; Abolfazl
• Application Number:
  US10/974,044
• Publication Number:
  US 20050089194A1
• Time in Patent Office: 1,667 Days
• Field of Search: 382/103, 382/107, 382/149, 382/236, 382/218, 348/69, 348/142, 348/169, 348/153, 348/239, 348/460, 348/471, 348/474, 348/510, 348/512, 348/522, 348/51, 348/52, 348/177, 348/191, 725/12, 725/38, 725/41, 725/60, 725/61, 725/135, 725/136, 715/761
• US Class Current: 382/107
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

A method and system for processing captured image information in an interactive video display system. In one embodiment, a special learning condition of a captured camera image is detected. The captured camera image is compared to a normal background model image and to a second background model image, wherein the second background model is learned at a faster rate than the normal background model. A vision image is generated based on the comparisons. In another embodiment, an object in the captured image information that does not move for a predetermined time period is detected. A burn-in image comprising the object is generated, wherein the burn-in image is operable to allow a vision system of the interactive video display system to classify the object as background.

Claims

1. A method for adapting to changes in captured camera images in an interactive video display system, the method comprising:

developing a slow-changing background model and a fast-changing background model, each of the background models based in part on trends in the captured camera images, the models developed at a computing device;

detecting a background change in one or more of the captured camera images with respect to the slow-changing background model, the background changed captured by a camera;

determining that the background change exceeds a threshold associated with the slow-changing background model, the determination made by the computing device; and

generating a vision image using the fast-changing background model as a result of the background change exceeding the threshold, the vision image rendered on a video display.

2. The method of claim 1, wherein the threshold includes a percentage of the camera image being obscured for a specified period of time.

3. The method of claim 1, wherein the threshold includes a specified portion of the pixels in the camera image having a value that exceeds a particular threshold for a particular length of time.

4. The method of claim 1, further comprising:

replacing the slow-changing background model with the fast-changing background model; and

developing a new fast-changing background model to replace the old fast-changing background model.

5. The method of claim 4, wherein the slow-changing background model is replaced with the fast-changing background model once the background change no longer exceeds the threshold.

6. The method of claim 1, further comprising generating a further vision image using the slow-changing background model as a result of the background change no longer exceeding the threshold, the further video image rendered on the video display.

7. The method of claim 6, wherein the threshold includes a percentage of the camera image no longer being obscured for a specified period of time.

8. The method of claim 6, wherein the threshold includes a specified portion of the pixels in the camera image having a value that no longer exceed a particular threshold for a particular length of time.

9. The method of claim 6, wherein the threshold includes a subset of the camera image staying constant for a specified period of time.

10. An interactive display system, the system comprising:

a camera configured to capture images;

a computing device configured to;

developed a slow-changing background model and a fast-changing background model, each of the background models based in part on tends in the images captured from the camera,

detect a background change in one or more of the images captured from the camera with respect to the slow-changing background model, and

determine that the background change exceeds a threshold associated with the slow-changing background model; and

a video display configured to display a vision image generated using the fast-changing background model in response to background change exceeding the threshold.

11. The interactive display system of claim 10, wherein the threshold includes a percentage of the camera image being obscured for a specified period of time.

12. The interactive display system of claim 10, wherein the threshold includes a specified portion of the pixels in the camera image having a value that exceeds a particular threshold for a particular length of time.

13. The interactive display system of claim 10, wherein the computing device is further configured to

replace the slow-changing background model with the fast-changing background model;

and

develop a new fast-changing background model to replace the old fast-changing background model.

14. The interactive display system of claim 13, wherein the slow-changing background model is replaced with the fast-changing background model once the background change no longer exceeds the threshold.

15. The interactive display system of claim 10, wherein a further vision image is generated using the slow-changing background model as a result of the background change no longer exceeding the threshold, the further video image rendered on the video display.

16. The interactive display system of claim 15, wherein the threshold includes a percentage of the camera image no longer being obscured for a specified period of time.

17. The interactive display system of claim 15, wherein the threshold includes a specified portion of the pixels in the camera image having a value that no longer exceed a particular threshold for a particular length of time.

18. The interactive display system of claim 15, wherein the threshold includes a subset of the camera image staying constant for a specified period of time.

19. A computer-readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for adapting to changes in captured camera images, the method comprising:

developing a slow-changing background model and a fast-changing background model, each of the background models based in part on trends in the captured camera images;

detecting a background change in one or more of the captured camera images with respect to the slow-changing background mode;

determining that the background change exceeds a threshold associated with the slow-changing background model; and

generating a vision image using the fast-changing background model as a result of the background change exceeding the threshold.