METHODS AND SYSTEMS FOR INDEXING MULTIMEDIA CONTENT

• US 20170017838A1
• Filed: 07/14/2015
• Published: 01/19/2017
• Est. Priority Date: 07/14/2015
• Status: Active Grant

• Current Assignee: VideoKen, Inc.
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Arsikere, Harish, Biswas, Arijit, Shrivastava, Kundan, Deshmukh, Om D.
• Granted Patent: US 9,785,834 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06V 20/47 Detecting features for summarising video content
• CPC Class Codes: G06V 20/80 Recognising image objects characterised by unique random patterns
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

According to embodiments illustrated herein, a method and system is provided for indexing a multimedia content. The method includes extracting, by one or more processors, a set of frames from the multimedia content, wherein the set of frames comprises at least one of a human object and an inanimate object. Thereafter, a body language information pertaining to the human object is determined from the set of frames by utilizing one or more image processing techniques. Further, an interaction information is determined from the set of frames. The interaction information is indicative of an action performed by the human object on the inanimate object. Thereafter, the multimedia content is indexed in a content database based at least on the body language information and the interaction information.

Claims

1. A method for indexing a multimedia content, the method comprising:

extracting, by one or more processors, a set of frames from said multimedia content, wherein said set of frames comprises at least one of a human object and an inanimate object;

determining, by said one or more processors, a body language information pertaining to said human object from said set of frames by utilizing one or more image processing techniques;

determining, by said one or more processors, an interaction information from said set of frames, wherein said interaction information is indicative of an action performed by said human object on said inanimate object; and

indexing, by said one or more processors, said multimedia content in a content database based at least on said body language information and said interaction information.

2. The method of claim 1 further comprising analyzing, by said one or more processors, said multimedia content to determine an emotion information by utilizing one or more of one or more image processing techniques, one or more speech/audio processing techniques, or one or more natural language processing techniques, wherein said emotion information is indicative of an emotion presented by said human object in said multimedia content.

3. The method of claim 1, wherein said multimedia content is further indexed based on said emotion information.

4. The method of claim 1 further comprising analyzing, by said one or more processors, an audio content within said multimedia content to determine an audio characteristics information by utilizing one or more speech/audio processing techniques, wherein said audio characteristics information comprises one or more of a speech rate, an accent, a speaking style, a background audio, or a background noise.

5. The method of claim 4, wherein said multimedia content is further indexed based on said audio characteristics information.

6. The method of claim 1 further comprising determining, by said one or more processors, a first textual content from said multimedia content by utilizing one or more text recognition techniques, wherein said first textual content comprises one or more of a textual content located on said inanimate object or a close-captioned text within said multimedia content.

7. The method of claim 6 further comprising determining, by said one or more processors, a second textual content from an audio content within said multimedia content, by utilizing one or more speech-to-text conversion techniques.

8. The method of claim 7 further comprising determining, by said one or more processors, a discourse rate associated with said multimedia content based on said first textual content and said second textual content.

9. The method of claim 8, wherein said multimedia content is further indexed based on said discourse rate.

10. The method of claim 7 further comprising extracting, by said one or more processors, one or more keywords from said first textual content and said second textual content by utilizing one or more natural language processing techniques, wherein said one or more keywords relate to one or more concepts explained in said multimedia content.

11. The method of claim 10 further comprising determining, by said one or more processors, a number of repetitions of said one or more concepts in said multimedia content.

12. The method of claim 11, wherein said multimedia content is further indexed based on one or more of said one or more concepts or said number of repetitions of said one or more concepts in said multimedia content.

13. The method of claim 1 further comprising determining, by said one or more processors, a personality type associated with said human object based on one or more of said body language information, said interaction information, an emotion information indicative of an emotion presented by said human object in said multimedia content, a speech rate of said human object, a speaking style of said human object, or a second textual content determined from an audio content within said multimedia content.

14. The method of claim 13, wherein said multimedia content is further indexed based on said personality type associated with said human object.

15. The method of claim 1, wherein said multimedia content includes one or more of an educational lecture, a corporate e-learning module (ELM), or a marketing/promotional video.

16. The method of claim 1, wherein said inanimate object comprises one or more of a presentation slide, a writing board, a poster, a paper, or a prop/model.

17. The method of claim 16, wherein said action performed by said human object on said inanimate object includes one or more of:

said human object writing on said inanimate object,

said human object pointing towards or touching said inanimate object,

said human object holding said inanimate object,

said human object scrolling through a textual content on said inanimate object, or

said human object modifying or highlighting said textual content on said inanimate object.

18. The method of claim 1, wherein said body language information is determined based on one or more of a hand motions of said human object in said multimedia content, a body motion of said human object, a facial expression/emotion of said human object, a proximity of said human object to a video capturing device utilized for creation of said multimedia content, or an eye contact of said human object towards said video capturing device.

23. The system of claim 1, wherein said body language information is determined based on one or more of a hand motions of said human object in said multimedia content, a body motion of said human object, a facial expression/emotion of said human object, a proximity of said human object to a video capturing device utilized for creation of said multimedia content, or an eye contact of said human object towards said video capturing device.

19. A system for indexing a multimedia content, the system comprising:

one or more processors configured to;

extract a set of frames from said multimedia content, wherein said set of frames comprises at least one of a human object and an inanimate object;

determine a body language information pertaining to said human object from said set of frames by utilizing one or more image processing techniques;

determine an interaction information from said set of frames, wherein said interaction information is indicative of an action performed by said human object on said inanimate object; and

index said multimedia content in a content database based at least on said body language information and said interaction information.

20. The system of claim 19, wherein said multimedia content includes one or more of an educational lecture, a corporate e-learning module (ELM), or a marketing/promotional video.

21. The system of claim 19, wherein said inanimate object comprises one or more of a presentation slide, a writing board, a poster, a paper, or a prop/model.

22. The system of claim 21, wherein said action performed by said human object on said inanimate object includes one or more of:

said human object writing on said inanimate object,

said human object pointing towards or touching said inanimate object,

said human object holding said inanimate object,

said human object scrolling through a textual content on said inanimate object, or

said human object modifying or highlighting said textual content on said inanimate object.

24. A computer program product for use with a computing device, said computer program product comprising a non-transitory computer readable medium, said non-transitory computer readable medium stores a computer program code for indexing a multimedia content, said computer program code is executable by one or more processors in said computing device to:

extract a set of frames from said multimedia content, wherein said set of frames comprises at least one of a human object and an inanimate object;

determine a body language information pertaining to said human object from said set of frames by utilizing one or more image processing techniques;

determine an interaction information from said set of frames, wherein said interaction information is indicative of an action performed by said human object on said inanimate object; and

index said multimedia content in a content database based at least on said body language information and said interaction information.