Method and system for providing video content based on image

• US 9,906,820 B2
• Filed: 05/09/2016
• Issued: 02/27/2018
• Est. Priority Date: 07/06/2015
• Status: Active Grant

• Current Assignee: Piamond Corp.
• Original Assignee: Korea Advanced Institute of Science and Technology
• Inventors: Choi, JunKyun, Jeon, Kyu Yeong, Yang, JinHong, Kim, YongRok
• Primary Examiner(s): KIM, WILLIAM JW
• Application Number:
  US15/149,729
• Publication Number:
  US 20170013292A1
• Time in Patent Office: 659 Days
• Field of Search: None
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02

Abstract

Disclosed is a method for providing a content, the method including extracting at least one still image from video included in the content, extracting audio, which corresponds to the still image, and generating a script corresponding to the audio, adding a caption to the still image based on the generated script, and providing the content in response to a request of consumption to the content and providing the caption-added still image for the video that is streaming in real time.

Claims

1. A method for providing content, the method comprising:

extracting at least one still image from video included in the content;

extracting audio, which corresponds to the at least one still image, and generating a script corresponding to the audio;

adding a caption to the at least one still image based on the generated script;

providing, in response to a request of consumption to of the content, the at least one caption-added still image for the video that is streaming in real time; and

loading the at least one caption-added still image on a user terminal if a difference between segment identification data of a buffer and segment identification data of the at least one caption-added still image for the content requested for consumption is lower than a value; and

wherein the at least one caption-added still image is displayed in an interface that is separate and distinct from a video viewing interface for the video.

2. The method of claim 1, wherein the extracting of at least one still image from video included in the content comprises:

extracting the at least one still image based on identification data of the content by determining inclusion of video that is included in the content or by recognizing state information of a user.

3. The method of claim 2, wherein the extracting of at least one still image from video included in the content comprises:

extracting the at least one still image, which is relevant to a section where an effect is inserted, based on audio mining from the video or extracting the at least one still image using metadata information of the video.

4. The method of claim 3, wherein the extracting of at least one still image from video included in the content comprises:

adding segment identification data by extracting the at least one still image.

5. The method of claim 1, wherein the providing of the at least one caption-added still image comprises:

updating the at least one caption-added still image in an asynchronous form at a user terminal and providing an interface for interlocking the content based on an advertisement policy.

6. The method of claim 1, wherein the providing of the at least one caption-added still image comprises:

providing at least one of functions of adding metadata to the at least one caption-added still image, sharing the at least one caption-added still image with an external service, replaying video that is included in the content, and adding the at least one caption-added still image.

7. A non-transitory computer-readable medium with an instruction controlling a computer system to provide a content, wherein the instruction controls the computer system by a method comprising:

extracting at least one still image from video included in the content;

extracting audio, which corresponds to the at least one still image, and generating a script corresponding to the audio;

adding a caption to the at least one still image based on the generated script;

providing, the content in response to a request of consumption to of the content, and providing the at least one caption-added still image for the video that is streaming in real time; and

loading the at least one caption-added still image on a user terminal if a difference between segment identification data of a buffer and segment identification data of the at least one caption-added still image for the content requested for consumption is lower than a value; and

wherein the at least one caption-added still image is displayed in an interface that is separate and distinct from a video viewing interface for the video.

8. A system for providing content, the system comprising:

an extraction part configured to extract at least one still image from video included in the content;

a generation part configured to extract audio, which corresponds to the at least one still image, and generating a script corresponding to the audio;

an

addition part configured to add a caption to the at least one still image based on the generated script;

a provision part configured to provide the at least one caption-added still image in response to a request of consumption to of the content and providing the at least one caption-added still image for the video that is streaming in real time; and

wherein the provision part loads the at least one caption-added still image on a user terminal if a difference between segment identification data of a buffer and segment identification data of the at least one caption-added still image for the content requested for consumption is lower than a value and the at least one caption-added still image is displayed in an interface that is separate and distinct from a video viewing interface for the video.

9. The system of claim 8, wherein the extraction part extracts the at least one still image based on identification data of the content by determining inclusion of video that is included in the content or by recognizing state information of a user.

10. The system of claim 9, wherein the extraction part extracts the at least one still image, which is relevant to a section where an effect is inserted, based on audio mining from the video or extracts the at least one still image using metadata information of the video.

11. The system of claim 10, wherein the extraction part adds segment identification data by extracting the at least one still image.

12. The system of claim 8, wherein the provision part updates the at least one caption-added still image in an asynchronous form at a user terminal and provides an interface for interlocking the content based on an advertisement policy.

13. The system of claim 8, wherein the provision part provides at least one of functions of adding metadata to the at least one caption-added still image, sharing the at least one caption-added still image with an external service, replaying video that is included in the content, and adding the at least one caption-added still image.