System and method for automatically creating personal profiles for video characters

• US 20080059522A1
• Filed: 08/29/2006
• Published: 03/06/2008
• Est. Priority Date: 08/29/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Li, Ying, Park, Youngja
• Application Number:
  US11/511,816
• Publication Number:
  US 20080059522A1
• Time in Patent Office: Days
• US Class Current: 707/104.1
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A system, method and computer program product for generating personal profiles for subjects appearing in a video media source. The method includes extracting audiovisual-related personal information related to a subject appearing in the video media source; extracting text-related personal information that are related to the subject in the video source; correlating the extracted audiovisual-related personal information and the extracted text-related personal information related to the subject; and assembling a personal profile data structure for the subject, the personal profile data structure comprising the text-related personal information and audiovisual-related personal information related to the subject. The text-related personal information forms the name identity of the subject, while the audiovisual-related personal information includes audiovisual-related features including information forming one or more of: a visual identity, a kinematic identity and, a voice identity of the subject. In an alternate embodiment, in an iterative manner, the correlated extracted audiovisual-related personal information and extracted text-related personal information may be fed back and utilized for performing an additional search from external information sources, via a search engine, to obtain additional texts relating to the subject or obtain additional video media sources having the subject. There is further enabled the updating of an assembled personal profile of a subject as a new video media source having said subject becomes available.

Claims

1. A system for generating a personal profile for a subject appearing in a video media source, said system comprising:

a means for extracting audiovisual-related personal information related to said subject appearing in said video media source;

a means for extracting text-related personal information that are related to said subject in said video source;

a means for correlating said extracted audiovisual-related personal information with said extracted text-related personal information to form a personal profile for said subject, said personal profile comprising a data structure including said text-related personal information and audiovisual-related personal information related to said subject.

2. The system for generating a personal profile as claimed in claim 1, further comprising:

a means for extracting video texts from one of;

said video media source and additional information sources, said text-related personal information extracting means receiving said extracted video texts to extract said text-related personal information.

3. The system for generating a personal profile as claimed in claim 1, wherein said audiovisual-related personal information includes audiovisual-related features that pertain to personal identity comprising one or more of:

a visual identity, a kinematic identity, and a voice identity of said subject.

4. The system for generating a personal profile as claimed in claim 3, wherein said audiovisual-related features forming said kinematic identity comprise a video clip showing motion including a gait or gesture of said subject.

5. The system for generating a personal profile as claimed in claim 3, wherein said audiovisual-related features forming said visual identity comprise the face of said subject.

6. The system for generating a personal profile as claimed in claim 3, wherein said audiovisual-related features forming said voice identity comprise an audio clip of said subject'"'"'s voice.

7. The system for generating a personal profile as claimed in claim 2, wherein said extracted text-related personal information comprises personal information associated with said subject that can be automatically extracted from a video media source.

8. The system for generating a personal profile as claimed in claim 7, wherein said extracted text-related personal information includes text-related personal identity information including one or more of:

a subject'"'"'s name, affiliation, and job position.

9. The system for generating a personal profile as claimed in claim 2, wherein said means for extracting video texts that are related to a video source includes one or more of:

a video scene text extractor for extracting texts that are displayed over video frames, and a video transcriber device.

10. The system for generating a personal profile as claimed in claim 2, wherein said means for extracting video texts from additional information sources includes obtaining text materials that are related to the said video source from said additional information sources.

11. The system for generating a personal profile as claimed in claim 9, wherein said video transcriber device includes one of:

a speech recognizer and a closed caption extractor.

12. The system for generating a personal profile as claimed in claim 7, further comprising:

a search engine means for receiving said extracted text-related personal information relating to said subject and performing a search from additional information sources to obtain additional texts relating to said subject or obtain additional video media sources having said subject.

13. The system for generating a personal profile as claimed in claim 12, wherein said search engine means further receives said extracted audiovisual-related personal information related to said subject appearing in said video media source and performs a search from said additional information sources to obtain additional texts relating to said subject or obtain additional video media sources having said subject.

14. The system for generating a personal profile as claimed in claim 12, wherein said search engine means performs one or more of:

an Internet/World Wide Web search, and a database search.

15. The system for generating a personal profile as claimed in claim 1, wherein said correlating means comprises means for performing semantic context analysis for correlating said extracted audiovisual-related personal information with said extracted text-related personal information.

16. The system for generating a personal profile as claimed in claim 15, wherein said correlated extracted audiovisual-related personal information and extracted text-related personal information output from said correlating means is input to said search engine means for performing an additional search from additional information sources to obtain additional texts relating to said subject or obtain additional video media sources having said subject.

17. The system for generating a personal profile as claimed in claim 1, further comprising means for updating an assembled personal profile of a subject as a new video media source having said subject becomes available.

18. A method for generating a personal profile for a subject appearing in a video media source, said method comprising:

extracting audiovisual-related personal information related to said subject appearing in said video media source;

extracting text-related personal information that are related to said subject in said video source;

correlating said extracted audiovisual-related personal information with said extracted text-related personal information related to said subject; and

assembling a personal profile data structure for said subject, said personal profile data structure comprising said text-related personal information and audiovisual-related personal information related to said subject.

19. The method for generating a personal profile as claimed in claim 18, wherein said extracting of text-related personal information comprises:

extracting video texts from one of;

said video media source and additional information sources, said text-related personal information related to said subject being extracted from said extracted video texts.

20. The method for generating a personal profile as claimed in claim 19, wherein said extracting video texts from said video media source includes implementing one or more of:

a video scene text extractor for extracting texts that are displayed over video frames, and a video transcriber device.

21. The method for generating a personal profile as claimed in claim 19, wherein said extracting video texts from said additional information sources includes obtaining text materials that are related to the said video source from external information sources.

22. The method for generating a personal profile as claimed in claim 20, wherein said video transcriber device includes one of:

a speech recognizer, and a closed caption extractor.

23. The method for generating a personal profile as claimed in claim 18, further comprising:

receiving, by a search engine means, one or more of;

said extracted text-related personal information relating to said subject, and said extracted audiovisual-related personal information related to said subject, and,

performing a search, via said search engine means, to obtain additional texts relating to said subject or obtain additional video media sources having said subject.

24. The method for generating a personal profile as claimed in claim 18, wherein said correlating includes performing semantic context analysis for correlating said extracted audiovisual-related personal information with said extracted text-related personal information.

25. The method for generating a personal profile as claimed in claim 24, further comprising:

receiving, at said search engine means, said correlated extracted audiovisual-related personal information and extracted text-related personal information output from said correlating means;

performing an additional search to obtain additional texts relating to said subject or obtain additional video media sources having said subject; and

,

conducting additional audiovisual-related personal information and text-related personal information extracting steps and correlating said extracted additional audiovisual-related personal information with said extracted additional text-related personal information related to said subject prior to assembling said personal profile data structure for said subject.

26. The method for generating a personal profile as claimed in claim 18, further comprising updating an assembled personal profile of a subject as a new video media source having said subject becomes available.

27. A program storage device tangibly embodying software instructions which are adapted to be executed by a machine to perform a method of generating a personal profile for a subject appearing in a video media source, said method comprising:

extracting audiovisual-related personal information related to said subject appearing in said video media source;

extracting text-related personal information that are related to said subject in said video source;

correlating said extracted audiovisual-related personal information with said extracted text-related personal information related to said subject; and

assembling a personal profile data structure for said subject, said personal profile data structure comprising said text-related personal information and audiovisual-related personal information related to said subject.

28. The program storage device tangibly embodying software instructions as claimed in claim 27, wherein said extracting of text-related personal information comprises:

extracting video texts from one of;

said video media source and additional information sources, said text-related personal information related to said subject being extracted from said extracted video texts.

29. The program storage device tangibly embodying software instructions as claimed in claim 28, wherein said extracting video texts from said video media source includes implementing one or more of:

a video scene text extractor for extracting texts that are displayed over video frames, and a video transcriber device.

30. The program storage device tangibly embodying software instructions as claimed in claim 28, wherein said extracting video texts from said additional information sources includes obtaining text materials that are related to the said video source from external information sources.

31. The program storage device tangibly embodying software instructions as claimed in claim 29, wherein said video transcriber device includes one of:

a speech recognizer, and a closed caption extractor.

32. The program storage device tangibly embodying software instructions as claimed in claim 27, further comprising:

receiving, by a search engine means, one or more of;

said extracted text-related personal information relating to said subject, and said extracted audiovisual-related personal information related to said subject, and,

performing a search, via said search engine means, to obtain additional texts relating to said subject or obtain additional video media sources having said subject.

33. The program storage device tangibly embodying software instructions as claimed in claim 27, wherein said correlating includes performing semantic context analysis for correlating said extracted audiovisual-related personal information with said extracted text-related personal information.

34. The program storage device tangibly embodying software instructions as claimed in claim 33, further comprising:

receiving, at said search engine means, said correlated extracted audiovisual-related personal information and extracted text-related personal information output from said correlating means;

performing an additional search to obtain additional texts relating to said subject or obtain additional video media sources having said subject; and

,

conducting additional audiovisual-related personal information and text-related personal information extracting steps and correlating said extracted additional audiovisual-related personal information with said-extracted additional text-related personal information related to said subject prior to assembling said personal profile data structure for said subject.

35. The program storage device tangibly embodying software instructions as claimed in claim 27, further comprising updating an assembled personal profile of a subject as a new video media source having said subject becomes available.