Automatic image synthesis method

• US 10,460,487 B2
• Filed: 07/08/2016
• Issued: 10/29/2019
• Est. Priority Date: 06/27/2016
• Status: Active Grant

• Current Assignee: KAMI Vision Incorporated, Xiaoyi Technology Co., Ltd.
• Original Assignee: Shanghai Xiaoyi Technology Co., Ltd.
• Inventors: Ding, Dajun, Zhao, Lili
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US15/205,082
• Publication Number:
  US 20170372498A1
• Time in Patent Office: 1,208 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20032 Median filtering
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: H04N 9/78 for separating the brightness signal or the chrominance signal from the colour television signal, e.g. using comb filter

Abstract

Methods and apparatus for automatically synthesizing images are disclosed. The methods may include receiving a plurality of input frames with a common background. The methods may also include determining a number of the input frames. The methods may also include selecting, based on the number, a method to detect foregrounds of the input frames. The methods may further include using the selected method to generate an output frame comprising a combination of a plurality of the foregrounds.

Claims

1. An image synthesis method, comprising:

receiving a plurality of input frames with a common background;

determining a number of the input frames;

selecting, based on the number, a method to detect foregrounds of the input frames; and

using the selected method to generate an output frame comprising a combination of a plurality of the foregrounds,

wherein generating an output frame comprising a combination of a plurality of the foregrounds comprises;

determining image positions of the foregrounds;

determining, based on the image positions, amounts of overlap among the foregrounds; and

selecting the foregrounds for the combination, based on the amounts of overlap;

wherein determining the amounts of overlap comprises;

determining a percentage of a first one of the foregrounds that overlaps with other ones of the foregrounds; and

selecting the one or more foregrounds for the combination comprises;

when the percentage is higher than a second predetermined threshold, excluding the first foreground from the combination.

2. The method of claim 1, wherein using the method to detect the foregrounds comprises:

when the number of input frames is greater than a first predetermined threshold, determining median pixel values of corresponding pixels of the input frames;

generating a background frame comprising pixels having the median pixel values;

subtracting the background frame from the input frames to generate respective difference frames; and

detecting the foregrounds of the input frames based on the difference frames.

3. The method of claim 2, wherein the first predetermined threshold is equal to 6.

4. The method of claim 1, wherein using the method to detect the foregrounds comprises:

when the number of input frames is less than 6 but greater than 2, generating a first difference frame based on a first input frame and a second input frame, the first difference frame including a first region representing a foreground;

generating a second difference frame based on the first input frame and a third input frame, the second difference frame including a second region representing a foreground; and

determining a common portion of the first and second region as a foreground of the first input frame.

5. The method of claim 1, wherein using the method to detect the foregrounds comprises:

when the number of input frames is equal to 2, generating a difference frame based on the two input frames;

determining a foreground in the difference frame;

determining whether an edge of the foreground is present in the two input frames; and

when it is determined that a first input frame includes at least part of the edge, determining that the foreground is a foreground of the first input frame.

6. The method of claim 5, wherein determining whether the edge of the foreground is present in the two input frames comprises:

running an edge filter in regions of the two input frames that encompass the edge, the edge filter returning a first value in the first input frame and a second value in the second input frame;

comparing the first and second values; and

when the first value is greater than the second value, determining that the first input frame includes at least part of the edge.

7. The method of claim 1, wherein generating an output frame comprising a combination of one or more foregrounds comprises:

determining image sizes of the foregrounds; and

selecting the foregrounds for the combination, based on the image sizes.

8. The method of claim 1, wherein generating an output frame comprising a combination of a plurality of the foregrounds:

determining image positions of the detected foregrounds;

determining a sequence based on the image positions; and

combining the detected foregrounds according to the sequence.

9. The method of claim 1, wherein using the selected method to detect the foregrounds comprises:

reducing image sizes of the input frames to generate reduced frames; and

detecting the foregrounds based on the reduced frames.

10. A device, comprising:

a memory storing instructions; and

a processor configured to execute the instructions to;

receive a plurality of input frames with a common background;

determine a number of the input frames;

select, based on the number, a method to detect foregrounds of the input frames; and

use the selected method to generate an output frame comprising a combination of a plurality of the foregrounds,

wherein the processor is further configured to execute the instructions to;

determining a percentage of a first one of the foregrounds that overlaps with other ones of the foregrounds; and

when the percentage is higher than a second predetermined threshold, excluding the first foreground from the combination.

11. The device of claim 10, wherein the processor is further configured to execute the instructions to:

when the number of input frames is greater than a first predetermined threshold, determine median pixel values of corresponding pixels of the input frames;

generate a background frame comprising pixels having the median pixel values;

subtract the background frame from the input frames to generate respective difference frames; and

detect the foregrounds of the input frames based on the difference frames.

12. The device of claim 10, wherein the processor is further configured to execute the instructions to:

when the number of input frames is less than 6 but greater than 2, generate a first difference frame based on a first input frame and a second input frame, the first difference frame including a first region representing a foreground;

generate a second difference frame based on the first input frame and a third input frame, the second difference frame including a second region representing a foreground; and

determine a common portion of the first and second region as a foreground of the first input frame.

13. The device of claim 10, wherein the processor is further configured to execute the instructions to:

when the number of input frames is equal to 2, generate a difference frame based on the two input frames;

determine a foreground in the difference frame;

determine whether an edge of the foreground is present in the two input frames; and

when it is determined that a first input frame includes at least part of the edge, determine that the foreground is a foreground of the first input frame.

14. The device of claim 13, wherein the processor is further configured to execute the instructions to:

run an edge filter in regions of the two input frames that encompass the edge, the edge filter returning a first value in the first input frame and a second value in the second input frame;

compare the first and second values; and

when the first value is greater than the second value, determine that the first input frame includes at least part of the edge.

15. The device of claim 10, wherein the processor is further configured to execute the instructions to:

determine image positions of the foregrounds;

determine, based on the image positions, amounts of overlap among the foregrounds; and

select the foregrounds for the combination, based on the amounts of overlap.

16. The device of claim 10, wherein the processor is further configured to execute the instructions to:

determining image sizes of the foregrounds; and

selecting the foregrounds for the combination, based on the image sizes.

17. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a device, cause the processor to perform an image-synthesis method, the method comprising:

receiving a plurality of input frames with a common background;

determining a number of the input frames;

selecting, based on the number, a method to detect foregrounds of the input frames; and

using the selected method to generate an output frame comprising a combination of a plurality of the foregrounds,

wherein generating an output frame comprising a combination of a plurality of the foregrounds comprises;

determining image positions of the foregrounds;

determining, based on the image positions, amounts of overlap among the foregrounds; and

selecting the foregrounds for the combination, based on the amounts of overlap;

wherein determining the amounts of overlap comprises;

determining a percentage of a first one of the foregrounds that overlaps with other ones of the foregrounds; and

selecting the one or more foregrounds for the combination comprises;

when the percentage is higher than a second predetermined threshold, excluding the first foreground from the combination.