Multimedia summary generation employing user feedback

• US 6,557,042 B1
• Filed: 03/19/1999
• Issued: 04/29/2003
• Est. Priority Date: 03/19/1999
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: He, Li-Wei, Gupta, Anoop
• Primary Examiner(s): Chin, Wellington
• Assistant Examiner(s): HO, CHUONG T
• Application Number:
  US09/273,296
• Time in Patent Office: 1,502 Days
• Field of Search: 709/231, 709/232, 709/204, 709/203, 709/217, 709/219, 709/218, 709/236, 709/223, 707/2, 707/3, 707/5, 707/6, 707/10, 707/104.1, 707/1
• US Class Current: 709/231
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A multimedia server computer or other device can provide multimedia content, as well as summaries of the multimedia content, to one or more client computers. Each of the users of a client computer is identified as being part of a particular group and summaries are generated for each group. Each summary includes those portions of the multimedia content that are most interesting to previous users of the corresponding group.

Claims

1. One or more computer-readable storage media having stored thereon a computer program for generating a summary of multimedia content, the program comprising the following steps:

associating a score with each of a plurality of multimedia segments corresponding to the multimedia content;

modifying at least one of the scores based on feedback from a plurality of users; and

determining one or more of the plurality of multimedia segments that are part of the summary based at least in part on the scores of each of the plurality of multimedia segments.

2. One or more computer-readable storage media as recited in claim 1, wherein the program further comprises the following steps:

maintaining a separate set of scores for each of a plurality of user groups;

checking which of the plurality of user groups each of the plurality of users is part of; and

wherein, for each of the plurality of users, the modifying comprises modifying only scores from the set of scores corresponding to the group the user is part of.

3. One or more computer-readable storage media as recited in claim 1, wherein the determining comprises:

adding individual multimedia segments of the plurality of multimedia segments to a summary list, based at least in part on the scores of each of the plurality of multimedia segments, until a combined presentation time of the multimedia segments in the summary list exceeds a threshold time.

4. One or more computer-readable storage media as recited in claim 1, wherein:

the modifying comprises increasing a score of a multimedia segment in response to positive user feedback regarding the multimedia segment, and decreasing the score of the multimedia segment in response to negative user feedback regarding the multimedia segment; and

the determining comprises using the multimedia segments having the highest scores as the summary of the multimedia content.

5. One or more computer-readable storage media as recited in claim 1, wherein the feedback can be explicit feedback or implicit feedback.

6. One or more computer-readable storage media as recited in claim 5, wherein the modifying comprises modifying the at least one of the scores by different amounts based on whether the feedback is explicit feedback or implicit feedback.

7. One or more computer-readable storage media as recited in claim 5, wherein the explicit feedback comprises selection of a positive feedback button.

8. One or more computer-readable storage media as recited in claim 5, wherein the implicit feedback comprises one of:

selection of a rewind button;

selection of a fast forward button;

selection of an add annotation button; and

selection of a play button to increase the multimedia playback speed.

9. One or more computer-readable storage media as recited in claim 1, wherein the program further comprises the following step:

ordering the multimedia segments that are part of the multimedia summary for presentation to the user according to the scores of the segments.

10. One or more computer-readable storage media as recited in claim 1, wherein the program further comprises the following step:

ordering the multimedia segments that are part of the multimedia summary for presentation to the user according to the chronological order of the segments in the multimedia content.

11. One or more computer-readable storage media as recited in claim 1, wherein one or more of the plurality of segments is defined dynamically based on accesses, by the plurality of users, to the plurality of multimedia segments.

12. A method of providing a summary of multimedia content, wherein the multimedia content includes a plurality of multimedia segments, the method comprising:

identifying a first group of a plurality of groups to which a user belongs;

identifying a set, corresponding to the first group, of the plurality of multimedia segments, wherein the set of multimedia segments is determined based at least in part on feedback of previous users that belong to the first group; and

providing the set of multimedia segments to the user as the summary of the multimedia content.

13. A method as recited in claim 12, wherein the identifying of the set comprises selecting the plurality of multimedia segments for the set and storing information identifying the set prior to providing any of the set of multimedia segments to the user.

14. A method as recited in claim 12, wherein the identifying of the set comprises selecting the plurality of multimedia segments for the set as the summary is being provided to the user.

15. A method as recited in claim 12, wherein the feedback comprises explicit feedback.

16. A method as recited in claim 12, wherein the feedback comprises implicit feedback.

17. A method as recited in claim 12, further comprising collecting, as the summary is provided to the user, feedback from the user and using the collected feedback in identifying sets of the plurality of multimedia segments for provision to subsequent users as the summary of the multimedia content.

18. A method as recited in claim 12, wherein the providing comprises streaming the multimedia segments to a client computer for presentation to the user.

19. A method as recited in claim 12, wherein the providing comprises ordering the multimedia segments in the set for provision to the user according to an order in which more interesting segments are provided prior to less interesting segments.

20. A method as recited in claim 19, further comprising maintaining a record of segment scores identifying which of the multimedia segments are more interesting to users than others.

21. A computer-readable memory containing a computer program that is executable by a computer to perform the method recited in claim 12.

22. A system comprising:

a client computer configured to present a summary of multimedia content to a user, wherein the user is a member of one of a plurality of different user groups;

a multimedia server, coupled to the client computer, configured to provide the summary of multimedia content to the client computer, wherein the multimedia content comprises a plurality of multimedia segments;

a storage device to store summary information for each of the plurality of different user groups, compiled from a plurality of users that are part of the group, identifying feedback from the plurality of users regarding each of the plurality of multimedia segments; and

wherein the summary comprises a set, determined at least in part based on the summary information of the group of which the user is a part, of the plurality of media segments.

23. A system as recited in claim 22, wherein the multimedia server is configured to provide the summary to the client computer by streaming the set of media segments to the client computer.

24. A system as recited in claim 22, wherein the multimedia server is further configured to receive user feedback from the client computer and update the summary information based on the received feedback.

25. An apparatus comprising:

a filtering module to maintain summary information corresponding to multimedia content based at least in part on feedback from users having previously been presented with at least a portion of the multimedia content; and

a multimedia module, coupled to the filtering module, to provide a summary, based on the summary information, to a current user.

26. An apparatus as recited in claim 25, wherein the multimedia module is to stream the summary to a client computer for presentation to the current user.

27. An apparatus as recited in claim 25, wherein the filtering module is further to:

use the feedback from the users to generate scores for each of a plurality of segments of the multimedia content;

maintain a plurality of sets of scores, each set corresponding to one of the plurality of user groups;

assign each user to one of the plurality of groups; and

modify each set of scores based only on feedback from users in the group corresponding to the set.

28. An apparatus as recited in claim 27, wherein the multimedia module is further to provide the multimedia segments to the current user as the summary based on the scores from the group to which the current user belongs.

29. An apparatus as recited in claim 27, wherein the filter module is further to modify a set of scores by different amounts based on the type of user feedback.

30. An apparatus as recited in claim 29, wherein the types of user feedback include implicit and explicit feedback.

31. An apparatus as recited in claim 25, wherein the multimedia module is further to generate the summary of multimedia content prior to a request by the current user for the summary.

32. An apparatus as recited in claim 25, wherein the multimedia module is further to generate the summary of multimedia content while the summary is being provided to the current user.