Annotating programs for automatic summary generation

• US 7,028,325 B1
• Filed: 09/13/2000
• Issued: 04/11/2006
• Est. Priority Date: 09/13/1999
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, Gupta, Anoop, Acero, Alejandro
• Primary Examiner(s): Srivastava, Vivek
• Application Number:
  US09/660,529
• Time in Patent Office: 2,036 Days
• Field of Search: 725/38, 725/39, 725/61, 725/37, 463/43, 707/1, 707/3, 707/7, 707/10, 715/721, 715/751
• US Class Current: 725/37
• CPC Class Codes: G06F 16/634 Query by example, e.g. query by humming
• CPC Class Codes: G06F 16/68 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

Audio/video programming content is made available to a receiver from a content provider, and meta data is made available to the receiver from a meta data provider. The meta data corresponds to the programming content, and identifies, for each of multiple portions of the programming content, an indicator of a likelihood that the portion is an exciting portion of the content. In one implementation, the meta data includes probabilities that segments of a baseball program are exciting, and is generated by analyzing the audio data of the baseball program for both excited speech and baseball hits. The meta data can then be used to generate a summary for the baseball program.

Claims

1. A method comprising:

receiving a program including both audio and video;

receiving meta data corresponding to the program, wherein the meta data comprises a plurality of probabilities, each corresponding to a portion of the program, the probabilities representing a probabilistic combination of sports-specific events and sports-generic events identified in the program; and

rendering, based on the meta data, portions of the program as a summary of the program.

2. A method as recited in claim 1, wherein the rendering comprises displaying the video of the portions and playing the audio of the portions.

3. A method as recited in claim 1, wherein a probability indicates whether a portion is an exciting portion of the program.

4. A method as recited in claim 3, wherein the rendering comprises selecting the plurality of portions that have probabilities that exceed a threshold value, and rendering the selected portions as the summary.

5. A method as recited in claim 1, wherein the receiving a program and the receiving meta data comprise receiving both the program and the meta data from the same source.

6. A method as recited in claim 1, wherein the receiving meta data comprises receiving the meta data from a remote source via a network.

7. A method as recited in claim 6, wherein the network comprises the Internet.

8. A method as recited in claim 1, wherein the receiving meta data comprises receiving meta data generated manually.

9. A method as recited in claim 1, wherein the receiving meta data comprises receiving meta data generated automatically.

10. A method as recited in claim 1, wherein the sports-specific events comprise baseball hits, and wherein the sports-generic events comprise excited speech.

11. One or more computer readable media including a computer program that is executable by a processor to perform the method recited in claim 1.

12. The method of claim 1 wherein the rendering comprises displaying the video of the portions and playing the audio of the portions.

13. A system comprising:

a content provider to make programming content available to requesting clients;

a metadata provider to make meta data, corresponding to the programming content, available to the requesting clients, wherein the meta data identifies, for each of a plurality of portions of the programming content, an indicator of a likelihood that the corresponding portion is an exciting portion of the programming content, and wherein the meta data comprises a plurality of probabilities, each corresponding to a portion of the programming content, the probabilities representing a probabilistic combination of sports-specific events and sports-generic events identified in the program content; and

a plurality of receivers coupled to receive the programming content from the content provider and the meta data from the meta data provider.

14. A system as recited in claim 13, wherein the content provider and the meta data provider are the same.

15. A system as recited in claim 13, wherein the content provider and the meta data provider are coupled to the plurality of receivers via the Internet.

16. A system as recited in claim 13, wherein the plurality of receivers are further to render, based on the meta data, portions of the programming content as a summary of the programming content.

17. A system as recited in claim 13, wherein the plurality of receivers are further to select, from the plurality of portions, portions that have probability indicators that exceed a threshold value, and render the selected portions as the summary.

18. A system as recited in claim 13, wherein the meta data is generated manually.

19. A system as recited in claim 13, wherein the meta data is generated automatically.

20. A system as recited in claim 13, wherein the sports-specific events comprise baseball hits, and wherein the sports-generic events comprise excited speech.

21. A method of automatically summarizing a program, the method comprising:

identifying a plurality of sports-generic events from the audio of the program;

identifying a plurality of sports-specific events from the audio of the program; and

identifying, by combining the sports-generic events and the sports-specific events, a set of portions of the program as a summary of the program

wherein the sports-specific events comprise baseball hits, and wherein the sports-generic events comprise excited speech.