Summary image browsing system and method

• US 10,032,483 B2
• Filed: 03/26/2014
• Issued: 07/24/2018
• Est. Priority Date: 01/14/2014
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited (Hanwha Corporation)
• Original Assignee: Hanwha Techwin Co., Ltd. (Hanwha Corporation)
• Inventors: Ryu, Yeon Geol, Park, Dong Jun, Shin, Hak Chul
• Primary Examiner(s): Hunter, Mishawn
• Application Number:
  US15/107,011
• Publication Number:
  US 20170040036A1
• Time in Patent Office: 1,581 Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 21/8549 Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

An embodiment of the present invention provides a summary image browsing system and method. A summary image browsing method of the present invention may comprise the steps of: tracking motion trajectory of an object from an input video; extracting the object from the input video and then generating a series of object segments; and synthesizing the series of object segments with a background image along the motion trajectory of the object and generating a summary image having a thickness according to an occurrence time interval for each object extracted from the input video.

Claims

1. A method for browsing summary images comprising:

tracking, by an object extractor, a motion trajectory of an object from an input video;

extracting, by the object extractor, the object from the input video and generating a series of object segments;

synthesizing, by a summary image generator, the series of object segments with a background image along the motion trajectory of the object; and

generating, by the summary image generator, a summary image having a thickness proportional to an appearance duration of the object extracted from the input video.

2. The method of claim 1, wherein the summary image comprises a display region and a text region indicating information of the object, and a specific color is assigned to an edge of the display region according to a category of the object.

3. The method of claim 1, wherein the series of object segments are arranged not to overlap the background image along the motion trajectory of the object,

wherein each of the object segments in the series of object segments is displayed in a specific color or a specific figure and has a size that varies according to a position thereof in the background image.

4. The method of claim 1, further comprising:

aligning and displaying, along a time axis in an order in which a plurality of objects appear over time, one or more summary images matching a search condition, from among summary images generated according to the plurality of objects.

5. The method of claim 4, further comprising:

receiving an input for changing view points of the aligned summary images; and

rotating and moving the aligned summary images based on transformation relationships respectively assigned to the aligned summary images and displaying the rotated and moved summary images.

6. The method of claim 4, further comprising:

detecting an operation of dragging and dropping a first summary image to a second summary image, from among the aligned summary images; and

aligning and displaying a synthesized summary image, which is obtained by synthesizing the first summary image with the second summary image, with remaining summary images.

7. The method of claim 4, further comprising aligning and displaying a synthesized summary image, which is obtained by synthesizing one or more summary images, from among the aligned summary images, whose appearance durations partially overlap, with remaining summary images.

8. The method of claim 7, wherein the displaying of the synthesized summary image comprises aligning and displaying a synthesized summary image, which is obtained by synthesizing a summary image selected by a user with one or more summary images, from among the aligned summary images, whose appearance durations partially overlap, with remaining summary images.

9. The method of claim 4, further comprising:

receiving a request to re-align the aligned summary images according to categories; and

classifying the aligned summary images according to the categories, and aligning and displaying the classified summary images in each of the categories along the time axis in an order in which objects appear over time.

10. The method of claim 4, further comprising:

receiving an input for selecting a summary image from among the aligned summary images; and

highlighting the selected summary image, and separately displaying the highlighted summary image so that the highlighted summary image overlaps remaining summary images.

11. The summary image browsing method of claim 10, further comprising:

adding a replay icon to the highlighted summary image; and

when the replay icon is selected, replaying an object of the highlighted image in a time order.

12. The method of claim 4, further comprising:

receiving a request to remove or add a background from or to one or more summary images from among the aligned summary images; and

displaying the one or more summary images after removing or adding the background from or to the one or more summary images.

13. The method of claim 4, further comprising:

clustering summary images that are searched according to search conditions.

14. A system for browsing summary images comprising:

an object extractor configured to track a motion trajectory of an object from an input video and generate a series of object segments by extracting the object from the input video; and

a summary image generator configured to synthesize the series of object segments with a background image along the motion trajectory of the object and generate a summary image having a thickness proportional to an appearance duration of the object extracted from the input video.

15. The system of claim 14, wherein the summary image comprises a display region and a text region indicating information of the object, and a specific color is assigned to an edge of the display region according to a category of the object, and

the series of object segments are arranged not to overlap the background image along the motion trajectory of the object,

wherein each of the object segments in the series of object segments is displayed in a specific color or a specific figure and has a size that varies according to a position thereof in the background image.

16. The system of claim 14, further comprising:

a browsing engine configured to align and display, along a time axis in an order in which a plurality of objects appear over time, one or more summary images matching a search condition from among summary images generated according to the plurality of objects.

17. The system of claim 16, wherein the browsing engine is configured to rotate and move, in response to a request to change view points of the aligned summary images, the aligned summary images based on transformation relationships respectively assigned to the aligned summary images and display the rotated and moved summary images.

18. The system of claim 16, wherein the browsing engine is configured to align and display, according to a user selection or partially overlapping object appearance durations, a synthesized summary image, which is obtained by synthesizing some summary images from among the aligned summary images, with remaining summary images.

19. The system of claim 16, wherein the browsing engine is configured to highlight a summary image selected from among the aligned summary images and separately display the highlighted summary image so that the highlighted summary image overlaps remaining summary images,

wherein a replay icon is added to the highlighted image, and when an input for selecting the replay icon is received, an object in the highlighted summary image is replayed in a time order.

20. The system of claim 16, wherein the browsing engine is configured to cluster summary images that are searched according to search conditions.