METHOD AND SYSTEM FOR PROCESSING CAPTURED IMAGE INFORMATION IN AN INTERACTIVE VIDEO DISPLAY SYSTEM

• US 20090225196A1
• Filed: 05/19/2009
• Published: 09/10/2009
• Est. Priority Date: 10/24/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Intellectual Ventures Holding 67 LLC (Intellectual Ventures LLC)
• Inventors: Bell, Matthew
• Granted Patent: US 7,809,167 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

A method and system for processing captured image information in an interactive video display system. In one embodiment, a special learning condition of a captured camera image is detected. The captured camera image is compared to a normal background model image and to a second background model image, wherein the second background model is learned at a faster rate than the normal background model. A vision image is generated based on the comparisons. In another embodiment, an object in the captured image information that does not move for a predetermined time period is detected. A burn-in image comprising the object is generated, wherein the burn-in image is operable to allow a vision system of the interactive video display system to classify the object as background.

Claims

1. A tangible computer readable storage medium having computer-executable instructions stored thereon that, if executed by a computing system comprising one or more computing devices, cause the computing system to perform a method comprising:

generating a slow-changing background model based in part on trends in captured camera images;

generating a fast changing-background model based in part on trends in captured camera images;

determining a background change in one or more of the captured camera images with respect to the slow-changing background model; and

generating a vision image using the fast-changing background model in response to determining that the background change exceeds a threshold associated with the slow-changing background model.

2. The computer readable medium of claim 1, wherein the slow-changing background model is replaced with the fast-changing background model once the background change no longer exceeds the threshold.

3. An interactive display system comprising:

means for capturing images;

means for generating a slow-changing background model based in part on trends in one or more captured images;

means for generating a fast-changing background model based in part on trends in one or more captured images;

means for detecting one or more changes in a background of one or more of the captured images used in generating the slow-changing background model, and

means for determining if the detected change exceeds a threshold; and

means for displaying vision images generated using the fast-changing background model in response to determining that the detected change exceeds the threshold.

4. The interactive display system of claim 3, further comprising:

means for generating a slow-changing background model as a result of the detected change no longer exceeding the threshold, and wherein a further video image is rendered on the means for displaying vision images.

5. The interactive display system of claim 4, wherein the threshold includes a percentage of the captured images no longer being obscured for a specified period of time.

6. The interactive display system of claim 4, wherein the threshold includes at least a portion of the pixels in the captured images having a value that no longer exceeds a predefined intensity value for a particular length of time.

7. An interactive display system, the system comprising:

a camera interface configured to receive captured images from one or more cameras;

a computing device configured to;

develop a slow-changing background model and a fast-changing background model, each of the background models based in part on trends in the captured images received from the camera;

detect a background change in one or more of the captured images with respect to the slow-changing background model; and

initiate display of a vision image generated using the fast-changing background model on a video display in response to determining that the detected background change exceeds a threshold; and

a video interface configured to be coupled to a video display device.

8. The interactive display system of claim 7, wherein the computing device is further configured to:

replace the slow-changing background model with the fast-changing background model; and

develop a new fast-changing background model to replace the old fast-changing background model.

9. The interactive display system of claim 8, wherein the slow-changing background model is replaced with the fast-changing background model once the background change no longer exceeds the threshold.

10. A method for determining if one or more objects in an image are burned-in, the method comprising:

storing a burn-in image in a computer readable memory;

receiving a plurality of images from a camera device;

determining that a first object has remained substantially fixed in the foreground of the images for at least first predetermined period of time; and

adding the first object to the burn-in image.

11. The method of claim 10, further comprising:

storing a background image in a computer readable memory;

determining that a second object has remained substantially fixed in place in the background of the input images for at least a second predetermined period of time; and

adding the second object to the background image.

12. The method of claim 11, further comprising:

generating an output image combining both the burn-in image and the background image.

13. The method of claim 12, further comprising:

in response to determining that the first object no longer remains substantially fixed in place in the foreground of the input image, removing the first object from the burn-in image.

14. An imaging system for determining if one or more objects in a series of images are burned in, the system comprising:

an interface configured to receive images from one or more imaging devices;

one or more storage devices configured to store a burn-in image and a background image; and

a processor configured to

detect a first object in one or more of the images that remains substantially stationary in the foreground of the one or more images for at least first predetermined time period; and

classify one or more pixels associated with the first object as burned-in pixels, wherein the burned-in pixels collectively comprise the burn-in image.

15. The system of claim 14, wherein the processor is further configured to

detect a second object in the one or more of the images that remains substantially stationary in the background for at least second predetermined time period;

classify one or more pixels associated with the second object as background pixels, wherein the background pixels collectively comprise the background image.

16. The system of claim 15, wherein the processor is further configured to generate an output image by combining the burn-in pixels and the background pixels, wherein the pixels in the output image corresponding to the pixels in the burn-in image that are classified as burned-in are classified as background pixels in the output image.

17. A method for adapting a background in an interactive video display, the method comprising:

generating a first background model and a second background model, each of the background models based in part on trends in a plurality of images received from one or more cameras;

determining a background change in one or more of the plurality of images relative to the first background model; and

initiating display of an output image generated using the second background model on a video display in response to determining that the detected background change exceeds a threshold.