System and method for providing user control over repeating objects embedded in a stream

• US 7,523,474 B2
• Filed: 05/02/2003
• Issued: 04/21/2009
• Est. Priority Date: 07/01/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Herley, Cormac
• Primary Examiner(s): Kelley; Christopher
• Assistant Examiner(s): BROWN, RUEBEN M
• Application Number:
  US10/428,812
• Publication Number:
  US 20040001161A1
• Time in Patent Office: 2,181 Days
• Field of Search: 725/20
• US Class Current: 725/19
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/56 Arrangements characterised by components specially adapted for monitoring, identification or recognition covered by groups H04H60/29-H04H60/54

Abstract

Many media streams contain “objects” that repeat. Repeating objects in a media stream are defined as any section of non-negligible duration, i.e., a song, video, advertisement, jingle, etc., which would be considered to be a logical unit by a human listener or viewer. An “object controller” identifies such repeating objects as they occur, and provides an interactive user interface for allowing users to specify how individual repeating objects are to be handled either in real time, or upon subsequent occurrences of particular repeating objects. In general, the object controller includes a mechanism for identifying repeating objects, a mechanism for identifying temporal endpoints of those objects, a user interface for specifying actions to be taken when a particular object repeats within a media stream, and, in one embodiment, a buffer having sufficient length to allow for real-time deletion of objects from the media stream without obvious interruption in the stream.

Claims

1. A system for controlling repeating media objects in one or more media streams, comprising using a computing device to:

provide a playback of at least one media stream;

specify at least one action during playback of the at least one media stream, said action being automatically associated with any media object being represented by the playback of the at least one media stream at the time that the at least one action is specified;

identify each repeat instance of media objects within the at least one media stream;

identify temporal endpoints of each repeat instance of media objects within the at least one media stream by automatically aligning and comparing portions of the at least one media stream centered on at least two repeating instances of media objects in the at least one media stream to determine temporal endpoints by determining positions within the at least one media stream where the portions of the media stream centered on the at least two repeating instances of media objects diverge; and

perform any specified action associated with any repeat instance of media objects within any of the at least one media streams during playback of any of the at least one media streams.

2. The system of claim 1 wherein identifying each repeat instance of media objects within the at least one media stream comprises computing parametric information for each portion of the at least one media stream, and comparing the parametric information to at least one database of parametric information to locate at least one media object having matching parametric information.

3. The system of claim 1 wherein specifying at least one action comprises specifying at least one of:

fast forwarding through a media object during playback of the at least one media stream;

slowing playback of the media object during playback of the at least one media stream;

changing the media stream during playback of the at least one media stream;

adding a media object to a favorites list during playback of the at least one media stream;

rating a media object during playback of the at least one media stream;

automatically changing a playback volume for particular media objects during playback of the at least one media stream;

automatically blanking a display screen when a particular media object is detected during playback of the at least one media stream;

skipping to the end of particular media objects within the at least one media stream during playback of the at least one media stream;

deleting particular media objects from the at least one media stream during playback of the at least one media stream;

extracting and saving a copy of particular media objects from the at least one media stream during playback of the at least one media stream;

limiting a particular media object to a maximum number of occurrences during a predefined period during playback of the at least one media stream;

limiting a relative frequency of appearance of one or more particular media objects relative to other objects within the at least one media stream;

automatically replacing particular media objects in the at least one media stream with another previously stored media object during playback of the at least one media stream;

automatically jumping from a buffered playback of the at least one media stream to a real-time playback of the at least one media stream on the occurrence of one or more predefined events during playback of the at least one media stream;

automatically jumping from a real-time playback to a buffered playback on the occurrence of one or more predefined events; and

user defined actions.

4. The system of claim 1 further comprising a user interface for specifying the at least one action during playback of the at least one media stream.

5. The system of claim 1 further comprising a user interface for specifying the at least one action after playback of the at least one media stream and storing those specified actions for use during subsequent playback of one or more additional media streams.

6. The system of claim 1 further comprising a remote control device for specifying the at least one action during playback of the at least one media stream.

7. The system of claim 1 further comprising a voice recognition system for specifying the at least one action during playback of the at least one media stream.

8. The system of claim 1 wherein the playback of the at least one media stream is buffered.

9. A computer storage media having computer executable instructions for controlling repeating media objects within a media stream, comprising:

capturing a media stream;

examining the media stream to locate possible media objects within the stream;

storing parametric information for each possible object in an object database;

searching the database to identify media objects that potentially match each possible media object;

comparing one or more potentially matching media objects to each possible media object to identify repeating media objects by comparing a portion of the media stream centered on a location of each potentially matching media object to a portion of the media stream centered on a location of each possible media object;

aligning each repeating instance of each repeating media object to identify endpoints of each repeating media object;

providing a buffered playback of the media stream;

specifying at least one command to associated with at least one media object, said command to be executed upon subsequently identified repeating media objects that match any media object having at least one associated command.

10. The computer storage media of claim 9 wherein identifying endpoints of each repeating media object comprises aligning each repeating instance of each repeating media object and tracing backwards and forwards in each of the aligned media objects to determine locations within the media stream where each aligned media object is still approximately equivalent to the other aligned media objects.

11. The computer storage media of claim 10 wherein the locations within the media stream at which each aligned media object is still approximately equivalent to the other aligned media objects correspond to the endpoints of each repeating media object.

12. The computer storage media of claim 9 wherein the media stream is an audio media stream.

13. The computer storage media of claim 9 wherein the media stream is a video stream.

14. The computer storage media of claim 9 wherein the media objects are any of songs, music, advertisements, video clips, station identifiers, speech, images, and image sequences.

15. The computer storage media of claim 9 wherein capturing the media stream comprises receiving and storing a broadcast media stream.

16. The computer storage media of claim 9 wherein examining the media stream to locate possible media objects within the stream comprises computing parametric information for at least one segment of the media stream, and analyzing the parametric information to determine whether the parametric information represents a possible media object.

17. The computer storage media of claim 9 wherein searching the database to identify media objects that potentially match each possible media object comprises comparing the parametric information for each possible object to previous entries in the object database to locate similar possible objects.

18. The computer storage media of claim 9 wherein comparing one or more potentially matching media objects to each possible media object comprises comparing a low-dimensional version of portions of the media stream centered on a location of each potentially matching media object to a low-dimensional version of a portion of the media stream centered on a location each possible media object.

19. The computer storage media of claim 9 wherein comparing one or more potentially matching media objects to each possible media object comprises:

computing characteristic information from portions of the media stream centered on a location of each potentially matching media object;

computing characteristic information from a portion of the media stream centered on a location each possible media object; and

comparing the characteristic information for each potentially matching media object to the characteristic information each possible object.

20. The computer storage media of claim 9 further comprising storing at least one representative copy of each repeating media object on a computer readable medium.

21. The computer storage media of claim 9 further comprising storing the endpoint information for each repeating media object in the object database.

22. The computer storage media of claim 9 further comprising at least one of:

a computer user interface for specifying the at least one command during the buffered playback of the media stream;

a remote control device for specifying the at least one command during the buffered playback of the media stream; and

a voice recognition system for specifying the at least one command during the buffered playback of the media stream.

23. The computer storage media of claim 9 further comprising a computer user interface for manually scrolling through the object database at any time including during and after playback of a current media stream, and manually associating at least one command with at least one object via the computer user interface.

24. The computer storage media of claim 9 further comprising a computer user interface to group subsets of the object database by property at any time including during and after playback of a current media stream, and associating at least one command with at least one of the objects in that subset of the database.

25. The computer storage media of claim 9 further comprising automatically computing statistical information regarding objects within the media stream and automatically specifying at least one command based on the computed statistical information.

26. The computer storage media of claim 9 wherein specifying the at least one command comprises specifying at least one of:

fast forwarding through a media object during playback of the media stream;

slowing playback of the media object during playback of the media stream;

changing the media stream during playback of the media stream;

adding a media object to a favorites list during playback of the media stream;

rating a media object during playback of the media stream;

automatically changing a playback volume for particular media objects during playback of the media stream;

automatically blanking a display screen when a particular media object is detected during playback of the media stream;

skipping to the end of particular media objects within the media stream during playback of the media stream;

deleting particular media objects from the media stream during playback of the media stream;

extracting and saving a copy of particular media objects from the media stream during playback of the media stream;

limiting a particular media object to a maximum number of occurrences during a predefined period during playback of the media stream;

limiting a relative frequency of playback of one or more particular media objects relative to other media objects;

automatically replacing particular media objects in the media stream with another previously stored media object during playback of the media stream;

automatically jumping from a buffered playback of the media stream to a real-time playback of the media stream on the occurrence of one or more predefined events during playback of the media stream;

automatically jumping from a real-time playback to a buffered playback on the occurrence of one or more predefined events; and

user defined actions.

27. The computer storage media of claim 9 further comprising saving and exporting at least a portion of the object database, said object database including at least one command associated with at least one media object in the object database.

28. A computer-implemented process for controlling repeating media objects within a media stream, comprising:

importing at least a portion of one or more object databases comprising media object parametric information and one or more commands associated with one or more media objects;

wherein at least a portion of any imported object databases further comprise low dimension versions of at least one of the media objects represented by the media object parametric information;

providing a playback of at least one media stream;

identifying each repeat instance of media objects within the at least one media stream through a comparison of the media stream to the imported media object parametric information;

identifying temporal endpoints of each repeat instance of media objects within the at least one media stream; and

automatically performing any command associated with any identified repeat instance of media objects in real time during playback of the at least one media stream and relative to the identified temporal endpoints of the identified repeat instance of media objects.

29. The computer-implemented process of claim 28 further comprising any of an automatic specification and a manual specification, via a computer user interface, of at least one additional command to be associated with one or more media objects during playback of the at least one media stream.

30. The computer-implemented process of claim 29 wherein the computer user interface is a text-based computer user interface.

31. The computer-implemented process of claim 29 wherein the computer user interface is a graphical computer user interface.

32. The computer-implemented process of claim 28 wherein the one or more commands associated with the one or more media objects include at least one of:

fast forwarding through a media object during playback of the at least one media stream;

slowing playback of the media object during playback of the at least one media stream;

changing the at least one media stream during playback of the at least one media stream;

adding a media object to a favorites list during playback of the at least one media stream;

rating a media object during playback of the at least one media stream;

automatically changing a playback volume for particular media objects during playback of the at least one media stream;

automatically blanking a display screen when a particular media object is detected during playback of the at least one media stream;

skipping to the end of particular media objects within the at least one media stream during playback of the at least one media stream;

deleting particular media objects from the at least one media stream during playback of the at least one media stream;

extracting and saving a copy of particular media objects from the at least one media stream during playback of the at least one media stream;

limiting a particular media object to a maximum number of occurrences during a predefined period during playback of the at least one media stream;

limiting a relative frequency of playback of one or more particular media objects relative to other media objects;

automatically replacing particular media objects in the at least one media stream with another previously stored media object during playback of the at least one media stream;

automatically jumping from a buffered playback of the at least one media stream to a real-time playback of a media stream on the occurrence of one or more predefined events during playback of the at least one media stream;

automatically jumping from a real-time playback to a buffered playback on the occurrence of one or more predefined events; and

user defined actions.

33. The computer-implemented process of claim 28 wherein identifying each repeat instance of media objects within the at least one media stream comprises computing parametric information for each portion of the at least one media stream, and comparing the computed parametric information to the imported parametric information to locate at least one media object having matching parametric information.

34. The computer-implemented process of claim 28 wherein identifying temporal endpoints of each repeat instance of media objects within the at least one media stream comprises:

automatically aligning and comparing portions of the at least one media stream centered on at least two repeating instances of media objects in the at least one media stream; and

determining temporal endpoints by determining positions within the at least one media stream where the aligned portions of the media stream centered on the at least two repeating instances of media objects diverge.

35. The computer-implemented process of claim 28 further comprising a remote control device for specifying at least one additional command to be associated with one or more media objects during playback of the at least one media stream.

36. The computer-implemented process of claim 28 further comprising a voice recognition system for specifying at least one additional command to be associated with one or more media objects during playback of the at least one media stream.

37. The computer-implemented process of claim 28 wherein any imported object databases are automatically combined with any existing object databases.

38. The computer-implemented process of claim 28 further comprising a computer user interface for allowing a user to specify at least one subset of the object database for associating at least one command with at least one of the objects in that subset of the database.

39. The computer-implemented process of claim 38 further wherein the computer user interface provides a capability for automatically searching the object database and grouping media objects based on one or more common properties of those media objects to create the at least one subset of the object database.

40. The computer-implemented process of claim 39 further comprising the capability to specifically include or exclude particular objects from any subset of the object database.