System and method for online processing of video images in real time

• US 9,661,239 B2
• Filed: 06/05/2013
• Issued: 05/23/2017
• Est. Priority Date: 04/17/2013
• Status: Expired due to Fees

• Current Assignee: Digital Makeup Ltd.
• Original Assignee: Digital Makeup Ltd.
• Inventors: Guissin, Rami
• Primary Examiner(s): Diep, Trung
• Application Number:
  US14/784,529
• Publication Number:
  US 20160065864A1
• Time in Patent Office: 1,448 Days
• Field of Search: 3482084, 348652
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

The invention is directed to real-time processing of video data. In some examples at least one image of the video data is processed utilizing reference data comprising reference image data of background within a region of interest and clutter image data indicative thereof, to determine a pixel deviation level of each pixel in the at least one image and generate pixel-deviation image data indicative thereof. The pixel-deviation image data is processed to enhance its tonal pixel distribution and generating enhanced image data, which is processed to determine a threshold level based on the tonal pixel distribution. A binary image map is then generated using the determined threshold level, the binary image map being indicative of the background and foreground components of the at least one image.

Claims

1. A computerized system for real-time processing of video data, the system comprising a processor utility comprising:

a reference data module configured and operable to provide reference data comprising reference image data of background within a region of interest and clutter image data indicative thereof;

a pixel deviation calculator configured and operable to receive at least one image of said video data, and process said at least one image utilizing said background reference image data and said clutter image data to determine a pixel deviation level of each pixel in said at least one image and generate pixel-deviation image data indicative thereof;

a filtering module configured and operable to enhance tonal pixel distribution in the pixel-deviation image data and generating enhanced image data;

a binary image generator configured and operable to process the enhanced image data and determine a threshold level based on the tonal pixel distribution, and generate a binary image map being indicative of the background and foreground components of said at least one image.

2. The system of claim 1, wherein the reference image data module is configured and operable to receive input data of a background image of the region of interest from which foreground components were removed, generate the background reference image data, and determine pixel clutter level of each pixel in the background reference image and generate the clutter image data indicative thereof.

3. The system of claim 1, wherein said filtering module comprises a histogram stretching module configured and operable to process the pixel-deviation image data by stretching a color histogram thereof, thereby enhancing the tonal pixel distribution in the pixel-deviation image data.

4. The system of claim 1 wherein the binary image generator comprises a morphological operation module configured and operable to fill holes and detach false detections.

5. The system of claim 1 comprising an image background editing unit configured and operable to modify the background of the at least one image based on the binary image map and generate modified background image data indicative thereof.

6. The system of claim 1 comprising an image foreground editing unit configured and operable to modify the foreground components of the at least one image based on the binary image map and generate modified foreground image data indicative thereof.

7. The system of claim 6 comprising:

an image foreground editing unit configured and operable to modify the foreground components of the at least one image bases on the binary image map and generate modified foreground image data indicative thereof; and

an image combining unit configured and operable to combine the modified background image data and the modified foreground image data and generate a modified image of the at least one image.

8. The system of claim 1 wherein the reference data module is configured and operable to determine the pixel clutter level using successive image filtering stages comprising a first filtering stage to attenuate low frequency components in said background image and generate filtered background image data indicative thereof, and a second filtering stage to perform temporal integration filtering operation on said filtered background image data.

9. The system of claim 8 wherein the reference image data module comprises a module for determining an absolute value of each pixel in the filtered background image data for use in the second filtering stage.

10. The system of claim 8 wherein the first filtering stage is configured to perform at least one of the following filtering operations:

spatial high pass filter;

temporal high pass filter;

spatio-temporal high pass filter;

spatial band pass filter;

temporal band pass filter;

spatio-temporal band pass filter.

11. The system of claim 1 wherein the reference image data module is configured and operable to generate background reference image data using a temporal integration filter to filter the background image.

12. The system of claim 5 wherein the image background editing unit comprises a background manipulation module configured and operable to perform at least one of removing, replacing and augmenting, the background of the at least one image.

13. The system of claim 5 wherein the image background editing unit comprises a background illumination correction module configured and operable to generate illumination correction data usable for correcting illumination non uniformities in the background of the at least one image.

14. The system of claim 5 wherein the image background editing unit comprises a background stabilization module configured and operable to generate offset data indicative of background offsets induced in the at least one image due to camera movements.

15. The system of claim 14 wherein the pixel deviation calculator is configured and operable to use the offset data in the pixel deviation level computation to compensate the camera movements.

16. The system of claim 5 wherein the image background editing unit comprises a background graphic overlay module configured and operable to introduce graphical items in the background of the at least one image.

17. The system of claim 6 wherein the image foreground editing unit is configured and operable to perform digital makeup to one or more regions of the foreground.

18. The system of claim 17 wherein the image foreground editing unit comprises a skin segmentation module configured and operable to identify skin regions of a user in the foreground portion of the at least one image, and selectively apply the digital makeup to said skin regions.

19. The system according to claim 16 comprising touchless sensing detector configured and operable to detect body gestures of a user in the foreground of the video images towards the graphical items in the background of the at least one image.

20. The system of claim 1 wherein the pixel deviation calculator is configured and operable to generate the pixel-deviation image data by normalization according to respective histogramic standard deviation of at least one of said luminance and Chroma components of said reference image data.

21. The system according to claim 20 wherein the standard deviation is implemented by at least one of the following operations:

histogramic stretching and factorization by predetermined gain factor.

22. A method for real-time processing of video data comprising:

receiving reference data comprising reference image data of background within a region of interest and clutter image data indicative thereof;

receiving at least one image of said video data and processing it utilizing said background reference image data and said clutter image data to determine a pixel deviation level of each pixel in said at least one image and generate pixel-deviation image data indicative thereof;

processing the pixel-deviation image data to enhance its tonal pixel distribution and generating enhanced image data;

processing the enhanced image data and determining a threshold level based on the tonal pixel distribution;

generating a binary image map being indicative of the background and foreground components of said at least one image;

manipulating at least one said background and foreground components.