Method and system for real time motion picture segmentation and superposition

• US 7,742,108 B2
• Filed: 05/24/2004
• Issued: 06/22/2010
• Est. Priority Date: 06/28/2000
• Status: Active Grant

• Current Assignee: Rateze Remote Mgmt. L.L.C. (Intellectual Ventures LLC)
• Original Assignee: Somle Development LLC (Intellectual Ventures LLC)
• Inventors: Sheraizin, Vitaly S., Sheraizin, Semion M.
• Primary Examiner(s): Lee; M.
• Application Number:
  US10/851,190
• Publication Number:
  US 20040212738A1
• Time in Patent Office: 2,220 Days
• Field of Search: 348563-569, 348/586, 348/588, 348589-599, 348/239, 348/701, 348/702
• US Class Current: 348/586
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

Methods and systems for separating a picture part of interest from an arbitrary background in real time videos are presented. The background may be a moving or still frame. The picture part of interest and background frames may be in or out of focus. One separation approach employs the difference between luminance and chrominance values of the input and background frames where changes in luminance from frame to frame are compensated for. In another approach, picture part of interest separation is based solely on spatial resolution differences between the background and the picture part of interest frames. The present invention also teaches a method and system for parameter matching of the picture part of interest and the basic picture into which the picture part of interest is embedded. The invention further teaches a method and system for embedding a separated picture part of interest into a basic picture containing text.

Claims

1. A method comprising:

receiving a background frame having an arbitrary background;

receiving an input frame having a picture part of interest within said arbitrary background; and

separating said picture part of interest from said arbitrary background using said background frame, said input frame, and time alignment signals derived from said input frame, wherein said separating is based on one or more resolution differences between said picture part of interest and said arbitrary background.

2. A method comprising:

receiving a background frame having an arbitrary background;

receiving an input frame having a picture part of interest within said arbitrary background; and

separating, using said background frame and said input frame, said picture part of interest from said arbitrary background, wherein said separating comprises time aligning the background frame and the input frame to account for different path lengths and generating one or both of;

a luminance difference between luminance signal values of said input frame and said background frame;

or

a chrominance difference between chrominance signal values of said input frame and said background frame.

3. The method of claim 2, wherein said generating is based on one or more signals converted from red, green, and blue (RGB) color signals of said input frame.

4. A system comprising:

a converting unit configured to convert red, green, and blue (RGB) color signals of an input frame to an input frame luminance signal and input frame chrominance signals, said input frame comprising a picture part of interest and an arbitrary background;

a mask generating unit configured to generate a picture part of interest mask using one or both of;

a luminance difference based on said input frame luminance signal and a background frame;

or

a chrominance difference based on said input frame chrominance signals and said background frame; and

a separator unit configured to separate the picture part of interest from said input frame using said picture part of interest mask and time alignment signals derived from the input frame to align the mask with the input frame.

5. The system of claim 4, wherein said mask generating unit comprises a luminance compensation unit configured to compute a coefficient to compensate for changes in said arbitrary background.

6. The system of claim 4, wherein said mask generating unit comprises one or more of:

a luminance summer configured to calculate said luminance difference;

a luminance comparator to compare said luminance difference to a luminance threshold;

a chrominance summer configured to calculate said chrominance difference;

or

a chrominance comparator to compare said chrominance difference to a chrominance threshold.

7. A picture part of interest separator comprising:

a window building unit configured to create a background luminance window from a frame input to the picture part of interest separator;

a converting unit configured to convert signals of the input frame to a frame luminance signal and frame chrominance signals;

a mask generating unit configured to use the background luminance window from the window building unit and converted signals from the converting unit to;

identify pixels of the input frame corresponding to a picture part of interest; and

generate a mask indicating a location of the picture part of interest within the input frame based on the identification; and

a separator unit configured to separate the picture part of interest from said input frame using the mask generated by the mask generating unit to output a separated picture part of interest (PPI), the separator unit including a time aligner configured to generate time aligned signals from the input frame.

8. The picture part of interest separator as recited in claim 7, wherein the input frame comprises a picture part of interest in an arbitrary background (PPIABG).

9. The picture part of interest separator as recited in claim 8, wherein the input frame is input as red, blue, and green (RGB) signals and synchronized luminance signals of the picture part of interest in an arbitrary background (PPIABG).

10. The picture part of interest separator as recited in claim 7, wherein the input frame comprises a still image.

11. The picture part of interest separator as recited in claim 7, wherein the input frame is a frame of a video.

12. The picture part of interest separator as recited in claim 7, wherein identification of the pixels of the input frame is based on one or more resolution differences determined by a comparison of the converted signals to a background image stored in a frame memory of the mask generating unit.

13. The picture part of interest separator as recited in claim 12, wherein generation of the mask is configured to occur when either a luminance difference or a chrominance difference computed via the comparison of the converted signals to the background image exceeds a respective threshold value.

14. The picture part of interest separator as recited in claim 7, wherein the mask comprises a pixel-by-pixel representation of the input frame that indicates whether each pixel is included in the picture part of interest.

15. The picture part of interest separator as recited in claim 7, wherein the separator unit includes a switch configured to produce the separated PPI from the time aligned signals and the mask.

16. The picture part of interest separator as recited in claim 7, wherein the separator unit is configured to output the separated PPI to a combining device that is operable to combine the separated PPI with a basic picture to embed the separated PPI within the basic picture.

17. The picture part of interest separator as recited in claim 7, wherein the converting unit converts red, blue, and green (RGB) signals of the input frame to the frame luminance signal and the frame chrominance signals including a red chrominance signal and a blue chrominance signal.

18. An image processing system comprising:

a picture part of interest separator configured to;

convert signals of a frame input to the picture part of interest separator to a frame luminance signal and frame chrominance signals;

identify pixels of the input frame corresponding to a picture part of interest based in part upon the converted signals;

generate a mask indicating a location of the picture part of interest within the input frame;

obtain time alignment signals corresponding to the input frame; and

separate the picture part of interest from said input frame using the mask and the time alignment signals to produce a separated picture part of interest (PPI); and

a combining unit configured to produce an embedded image including the picture part of interest from the separated PPI and a basic image input to the combining unit.

19. An image processing system as recited in claim 18, wherein:

the picture part of interest separator includes a frame memory to store a background frame in response to a setup command; and

identification of the pixels of the input frame comprises computing a luminance difference and a chrominance difference by comparing the converted signals to the background frame stored in the frame memory.