Systems and Methods for Automatically Applying Effects Based on Media Content Characteristics

• US 20150117777A1
• Filed: 04/08/2014
• Published: 04/30/2015
• Est. Priority Date: 10/28/2013
• Status: Active Grant

• Current Assignee: Cyberlink Corporation
• Original Assignee: Cyberlink Corporation
• Inventors: Hsun, Chen Yen
• Granted Patent: US 9,251,613 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/50 Depth or shape recovery

Abstract

Disclosed are systems and methods for automatically applying special effects based on media content characteristics. A digital image is obtained and depth information in the digital image is determined. A foreground region and a background region in the digital image are identified based on the depth information. First and second effects are selected from a grouping of effects, where the first effect is applied to at least a portion of the foreground region and the second effect is applied to at least a portion of the background region.

Claims

1. A method implemented in a media editing apparatus, comprising:

obtaining a digital image;

obtaining depth information in the digital image;

obtaining a foreground region and a background region in the digital image based on the depth information;

selecting a first effect from a grouping of effects;

selecting a second effect from the grouping of effects;

applying the first effect to at least a portion of the foreground region; and

applying the second effect to at least a portion of the background region.

2. The method of claim 1, wherein applying the first effect to at least the portion of the foreground region comprises:

identifying at least one object in the foreground region; and

applying the first effect to the at least one object in the foreground region.

3. The method of claim 2, wherein applying the second effect to at least the portion of the background region comprises:

identifying at least one object in the background region; and

applying the second effect to the at least one object in the background region.

4. The method of claim 1, wherein the grouping of effects further comprises sub-groups of effects, wherein each sub-group comprises a pre-selected effect for the foreground region and a pre-selected effect for the background region.

5. The method of claim 4, wherein selecting the first effect and selecting the second effect from the grouping of effects comprises selecting a sub-group of pre-selected effects based on at least one of:

the at least one object identified in the foreground region; and

the at least one object identified in the background region.

6. The method of claim 1, further comprising obtaining a plurality of effects specified by a user, wherein the user further specifies sub-groups of effects among the plurality of effects.

7. The method of claim 1, wherein obtaining depth information in the digital image comprises one of:

analyzing depth information and extracting depth data embedded in the digital image.

8. The method of claim 7, wherein analyzing the depth information comprises:

applying image segmentation based on color distribution to identify objects in the digital image; and

determining a relative depth of each of the objects, wherein the depth information in the digital image comprises information relating to at least one of;

sizes of similar objects, overlapping regions between objects, shadow regions of objects, salient objects, facial features, background objects, vanishing points, and depth of field in the digital image.

9. The method of claim 7, wherein determining the foreground region and the background region in the digital image based on the depth information comprises:

determining a depth map based on the relative depth of each of the identified objects; and

comparing, utilizing the depth map, each of a plurality of identified objects to a predetermined depth threshold to generate region data, the region data including a depth mask.

10. The method of claim 9, wherein the region data identifies the foreground region and the background region.

11. A media editing system, comprising:

a processor; and

at least one application executable in the processor, the at least one application comprising;

a media interface configured to obtain a digital image;

a depth analyzer configured to determine depth information in the digital image;

a region parser configured to determine a foreground region and a background region in the digital image based on the depth information;

an effect selector configured to select a first effect from a grouping of effects, the effect selector being further configured to select a second effect from the grouping of effects; and

an image editor configured to apply the first effect to at least a portion of the foreground region, the effect selector configured to apply the second effect to at least a portion of the background region.

12. The system of claim 11, wherein the image editor applies the first effect to at least the portion of the foreground region by identifying at least one object in the foreground region;

and applying the first effect to the at least one object in the foreground region.

13. The system of claim 12, wherein the image editor applies the second effect to at least the portion of the background region by identifying at least one object in the background region;

and applying the second effect to the at least one object in the background region.

14. The system of claim 11, wherein the grouping of effects further comprises sub-groups of effects, wherein each sub-group comprises a pre-selected effect for the foreground region and a pre-selected effect for the background region.

15. The system of claim 14, wherein the effect selector selects the first effect and the second effect from the grouping of effects by selecting a sub-group of pre-selected effects based on at least one of:

the at least one object identified in the foreground region; and

the at least one object identified in the background region.

16. The system of claim 11, wherein the grouping of effects comprises a plurality of effects specified by a user.

17. The system of claim 11, wherein the depth analyzer determines depth information in the digital image by determining an object distribution in the digital image;

applying image segmentation based on the object distribution to identify objects in the digital image; and

determining a relative depth of each of the identified objects.

18. The system of claim 17, wherein the region parser determines the foreground region and the background region in the digital image based on the depth information by determining a depth map based on the relative depth of each of the identified objects;

comparing, utilizing the depth map, each of the identified objects to a predetermined depth threshold to generate region data, the region data including a depth mask; and

applying the first effect and the second effect based on the depth mask.

19. A non-transitory computer-readable medium embodying a program executable in a computing device, comprising:

code that obtains a digital content comprising one of;

a digital image; and

a frame in a video sequence;

code that generates a depth mask based on depth information in the digital content;

code that determines an order of objects in the digital content based on a relative depth of each object specified in the depth mask;

code that categorizes each of the objects as one of a background object and a foreground object based on the order of objects;

code that selects a first effect and a second effect from a grouping of effects;

code that applies the first effect to at least a portion of the foreground objects; and

code that applies the second effect to at least a portion of the background objects.

20. The non-transitory computer-readable medium of claim 19, wherein the grouping of effects further comprises sub-groups of effects, wherein each sub-group comprises a pre-selected effect for the foreground region and a pre-selected effect for the background region.

21. The non-transitory computer-readable medium of claim 19, further comprising code that obtains a plurality of effects specified by a user, wherein the user further specifies sub-groups of effects among the plurality of effects.