Image manipulation for videos and still images

• US 20080181507A1
• Filed: 01/28/2008
• Published: 07/31/2008
• Est. Priority Date: 01/29/2007
• Status: Abandoned Application

• Current Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Original Assignee: IntelliVision Technologies Corp. (Melrose Industries Plc)
• Inventors: Agarwal, Amit, Bovyrin, Alexander, Gope, Chandan, Afraimovich, Lev, Popov, Ilya, Nathan, Vaidhi
• Application Number:
  US12/011,705
• Publication Number:
  US 20080181507A1
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

In an embodiment, an image is received having a first portion and one or more other portions. The one or more other portions are replaced with one or more other images. The replacing of the one or more portions results in an image including the first portion and the one or more other images. In an embodiment, the background of an image is replaced with another background. In an embodiment, the foreground is extracted by identifying the background based on an image of the background without any foreground. In an embodiment, the foreground is extracted by identifying portions of the image that have characteristics that are expected to be associated with the background and characteristics that are expected to be associated with foreground. In an embodiment any of the images can be still images. In an embodiment, any of the images are video images.

Claims

1. A method comprising:

receiving an image having a first portion and one or more other portions; and

replacing the one or more other portions with one or more other image, the replacing results in an image consisting of the first portion and the one or more other image.

2. The method of claim 1 further comprising:

receiving at least one image that has the one or more other images, but that does not have the first image; and

forming a model of the one or more other images based on the at least one image that has the one or more other images, but that does not have the first image.

3. The method of claim 1, further comprising:

remove image elements that are expected to be noise as a result of a frequency of variation associated with that element;

construct an edge map indicative of edges of image objects;

apply a smoothing technique within regions bounded by the edges identified by the edge map; and

construct a background model.

4. The method of claim 3, further comprising:

constructing the background model includes collecting information about the background based on images showing the background without the foreground.

5. The method of claim 4, the collecting of the information about the background including collecting information about a luminance, a chrominance, a hue, a texture, and a gradient associated with one or more pixels of the background.

6. The method of claim 1 including computing a transformation for the image that compensates for a shaking of a camera capturing the image.

7. A system comprising a machine readable medium storing instruction that cause the system to implement the method of claim 1.

8. A method comprising:

extracting one or more image elements from a first image;

retrieving one or more image elements from another source;

combining the one or more image elements from the first image and the one or more image elements from the other source to form a new image.

9. The method of claim 8, the other source being a storage media that stores predefined image elements.

10. The method of claim 8, wherein the other source being another image, the retrieving including at least extracting the one or more other image elements from the other image.

11. The method of claim 8, further comprising transforming one or more other image elements in conjunction with the combining.

12. The method of claim 8, the first image being a set of images forming a video.

13. The method of claim 8, the other source being a set of images forming a video.

14. The method of claim 8, results of the combining being a set of images forming a video.

15. A system comprising a machine readable medium storing instruction that cause the system to implement the method of claim 8.

16. A method comprising:

determining whether a pixel is part of a foreground portion of an image, the determining of whether the pixel is part of the foreground being based on a current frame;

determining whether the pixel is part of a current background portion of the image, the determining of whether the pixel is part of the current background being based on the current frame; and

extracting an image of the foreground that does not include the current background based on the determining of whether the pixel is part of the foreground and based on the determining of whether the pixel is part of the background.

17. The method of claim 16 further comprising:

the determining of whether the pixel is the foreground portion does not determine the pixel to be part of the foreground portion, and

the determining of whether the pixel is part of the current background portion does not determine the pixel to be part of the background portion,

then, determining whether the pixel is part of the background portion or foreground portion based on temporal data.

18. A system comprising a machine readable medium storing instruction that cause the system to implement the method of claim 16.

19. The method of claim 16 further comprising:

determining a motion associated with regions of the image; and

determining which pixels are background pixels based on whether the motion is within a range of values of motion that is expected to be associated with the background.

20. The method of claim 16, the foreground being one or more images of one or more people and determining the foreground pixel includes at least determining whether the pixels having a coloring that are expected to be associated with the one or more people.

21. The method of claim 20, the coloring including a hue associated with skin.

22. The method of claim 16, determining regions to be part of the background, based on the regions having a motion that is less than a particular amount, and updating the background based on the determining of the regions.

23. The method of claim 22, the updating of the background including changing pixel values of background pixels to indicate changes in lighting associated with the background.

24. The method of claim 16,

the extracting of the image of the foreground including a first phase and a second phase,

the first phase including at least

classifying pixels having a first range of motion values as background pixels

classifying pixels having a second of range of motion values as foreground pixels, the first range does not overlap the second range,

undetermined pixels, which are pixels having a motion value that is not in the first range and not in the second range, are not classified as background or foreground as part of the first phase;

and

during the second phase, classifying the undetermined pixels as background or foreground based one or more other criteria.

25. The method of claim 16, further comprising:

determining a complexity for one or more regions of a scene, and

adjusting one more criteria for determining whether a pixel is a background or foreground pixel, based on the complexity.