Method and apparatus for creating and viewing customized multimedia segments

• US 7,873,982 B2
• Filed: 06/22/2006
• Issued: 01/18/2011
• Est. Priority Date: 06/22/2006
• Status: Active Grant

• Current Assignee: TiVo Solutions Inc. (Adeia Inc.)
• Original Assignee: TiVo Inc. (Adeia Inc.)
• Inventors: Smith, Kevin P., Stevens, Paul, Feder, Adam, Barton, James M.
• Primary Examiner(s): Beliveau; Scott
• Assistant Examiner(s): DUFFIELD, JEREMY S
• Application Number:
  US11/473,990
• Publication Number:
  US 20070300250A1
• Time in Patent Office: 1,671 Days
• Field of Search: 725 34- 36, 725135-142
• US Class Current: 725/136
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04H 20/31 using in-band signals, e.g. subsonic or cue signal
• CPC Class Codes: H04H 60/13 Arrangements for device control affected by the broadcast information
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/73 using meta-information
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/43 Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/432 Content retrieval operation from a local storage medium, e.g. hard-disk details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/433 Content storage operation, e.g. storage operation in response to a pause request, caching operations
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/47214 for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8545 : for generating interactive applications
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/782 : on tape
• CPC Class Codes: H04N 7/0885 : for the transmission of subtitles

Abstract

A system for creating and viewing customized multimedia segments allows a user to create event identification data from in-band data in a program stream. The in-band data is processed and compiled into event identification data associated with the content stream. The event identification data identifies patterns or sequences of in-band data associated with events in the content stream. Metadata are also included in the event identification data. The event identification data is provided to multimedia devices which receive multimedia content streams containing content streams with in-band data. The in-band data are parsed and compared to the event identification data. If a match occurs, the multimedia device uses the metadata to identify events or perform specified actions.

Claims

1. A method for creating and viewing customized multimedia streams, comprising:

receiving a first content stream at a first multimedia display device, wherein the first content stream contains in-band data;

displaying the first content stream to a user;

receiving input from the user during display of the first content stream indicating a location within the first content stream that the user has selected;

in response to receiving input from the user indicating the location, computing at least two hash values from the in-band data in a vicinity of the location and one or more time values, each time value indicating a period of time between generation of two sequential hash values of the at least two hash values;

identifying metadata associated with the location; and

storing the at least two hash values, the one or more time values, and the metadata as event identification data on at least one storage device at the first multimedia display device.

2. The method of claim 1, wherein the in-band data is closed-caption data.

3. The method of claim 1, further comprising:

sending the event identification data to a second multimedia display device that is able to play a content stream.

4. The method of claim 3, wherein the user specifies the second multimedia display device.

5. The method of claim 1, further comprising:

sending the event identification data to a server; and

wherein the server distributes the event identification data to other devices.

6. The method of claim 1, wherein the at least one storage device is a hard disk.

7. The method of claim 1, wherein the at least one storage device is any type of removable storage media.

8. The method of claim 1, wherein the at least one storage device is any type of non-volatile and/or volatile storage medium.

9. The method of claim 1, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

10. The method of claim 9, wherein metadata associated with at least two hash values describes actions relating to one or more locations.

11. The method of claim 1, further comprising:

receiving, at a second multimedia display device, the event identification data;

receiving a second content stream at the second multimedia display device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

processing, at the second multimedia display device, the in-band data of the second content stream by calculating hash values for the in-band data;

determining that at least two hash values of the second content stream matches a particular at least two hash values in the event identification data, wherein the event identification data associates the particular at least two hash values with a location in the second content stream; and

identifying the location within the second content stream.

12. The method of claim 11, further comprising the step of:

in response to identifying the location, displaying an indication that the location is identified within the second content stream to a user while playing the second content stream to a user.

13. The method of claim 11, wherein a user is able to skip to an identified location while playing the second content stream.

14. The method of claim 11, wherein the second multimedia display device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

15. The method of claim 11, wherein metadata associated with the identified location instructs the second multimedia display device to skip a segment of the second content stream.

16. The method of claim 11, wherein metadata associated with the identified location instructs the second multimedia display device to reorder two or more segments of the second content stream.

17. The method of claim 16, further comprising:

creating, at the second multimedia display device, a customized multimedia content stream containing the two or more re-ordered segments; and

displaying the customized content stream to a user.

18. The method of claim 11, wherein metadata associated with the identified location instructs the second multimedia display device to store one or more segments of the second content stream on at least one storage device at the second multimedia display device, and wherein the at least one storage device contains one or more extracted segments from one or more different content streams.

19. The method of claim 18, further comprising:

creating, at the second multimedia display device, a customized multimedia content stream containing any combination of segments stored on the at least one storage device; and

displaying the customized content stream to a user.

20. The method of claim 1, wherein the first content stream is a television broadcast stream.

95. The method of claim 19, wherein the customized content stream is based on instructions in the event identification data.

96. The method of claim 11, wherein event identification data instructs the second multimedia display device to store one or more different content streams.

97. The method of claim 96, further comprising:

creating, at the second multimedia playback device, a customized multimedia content stream based upon instructions in the event identification data containing any combination of segments or content streams stored on the at least one storage device; and

playing back the customized content stream to a user.

21. An apparatus for creating and viewing customized multimedia streams, comprising:

a content receiving component that receives a first content stream at a first multimedia display device, wherein the first content stream contains in-band data;

a user receiving component that receives input from the user during display of the first content stream indicating a location within the first content stream that the user has selected;

a computational and identification component that computes, in response to receiving input from the user indicating the location, at least two hash values from the in-band data in a vicinity of the location and one or more time values, each time value indicating a period of time between generation of two sequential hash values of the at least two hash values;

a metadata identification component that identifies metadata associated with the location;

a storage component that stores the at least two hash values, the one or more time values, and the associated metadata as event identification data on at least one storage device at the first multimedia display device.

22. The apparatus of claim 21, wherein the in-band data is closed-caption data.

23. The apparatus of claim 21, further comprising:

a transmission component that sends the event identification data to a second multimedia display device that is able to play a content stream.

24. The apparatus of claim 23, wherein the user specifies the second multimedia display device.

25. The apparatus of claim 21, further comprising:

a transmission component that sends the event identification data to a server; and

wherein the server distributes the event identification data to other devices.

26. The apparatus of claim 21, wherein the at least one storage device is a hard disk.

27. The apparatus of claim 21, wherein the at least one storage device is any type of removable storage media.

28. The apparatus of claim 21, wherein the at least one storage device is any type of non-volatile and/or volatile storage medium.

29. The apparatus of claim 21, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

30. The apparatus of claim 29, wherein metadata associated with at least two hash values describes actions relating to one or more locations.

31. The apparatus of claim 29, further comprising:

a receiving component that receives, at a second multimedia display device, the event identification data;

a content receiving component that receives a second content stream at the second multimedia display device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

an in-band processing component that processes, at the second multimedia display device, the in-band data of the second content stream by calculating hash values for the in-band data;

a processing component that determines that at least two hash values of the second content stream matches a particular at least two hash values in the event identification data, wherein the event identification data associates the particular at least two hash values with a location in the second content stream; and

a location identification component that identifies the location within the second content stream.

32. The apparatus of claim 31, further comprising:

in response to identifying the location, a display component that displays an indication that the location is identified within the second content stream to a user while playing the second content stream to the user.

33. The apparatus of claim 31, wherein a user is able to skip to an identified location while playing the second content stream.

34. The apparatus of claim 31, wherein the second multimedia display device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

35. The apparatus of claim 31, wherein metadata associated with the identified location instructs the second multimedia display device to skip a segment of the second content stream.

36. The apparatus of claim 31, wherein metadata associated with the identified location instructs the second multimedia display device to reorder two or more segments of the second content stream.

37. The apparatus of claim 36, further comprising:

a generating component that creates, at the second multimedia display device, a customized multimedia content stream containing the two or more re-ordered segments; and

a display component that displays the customized content stream to a user.

38. The apparatus of claim 31, wherein metadata associated with the identified location instructs the second multimedia display device to store one or more segments of the second content stream on at least one storage device at the second multimedia display device, and wherein the at least one storage device contains one or more extracted segments from one or more different content streams.

39. The apparatus of claim 38, further comprising:

a generating component that creates, at the second multimedia display device, a customized multimedia content stream containing any combination of segments stored on the at least one storage device; and

a display component that displays the customized content stream to a user.

40. The apparatus of claim 21, wherein the first content stream is a television broadcast stream.

98. The apparatus of claim 39, wherein the customized content stream is based on instructions in the event identification data.

99. The apparatus of claim 31, wherein event identification data instructs the second multimedia display device to store one or more different content streams.

100. The apparatus of claim 99, further comprising:

a customized media stream creation component at the second multimedia playback device that creates, at the second multimedia playback device, a customized multimedia content stream based upon instructions in the event identification data containing any combination of segments or content streams stored on the at least one storage device; and

a playback component at the second multimedia playback device that plays back the customized content stream to a user.

41. A non-transitory computer-readable medium carrying one or more sequences of instructions for creating and viewing customized multimedia streams, which instructions, when executed by one or more processors, cause the one or more processors to carry out the steps of:

receiving a first content stream at a first multimedia display device, wherein the first content stream contains in-band data;

displaying the first content stream to a user;

receiving input from the user during display of the first content stream indicating a location within the first content stream that the user has selected;

in response to receiving input from the user indicating the location, computing at least two hash values from the in-band data in a vicinity of the location and one or more time values, each time value indicating a period of time between generation of two sequential hash values of the at least two hash values;

identifying metadata associated with the location; and

storing the at least two hash values, the one or more time values, and the metadata as event identification data on at least one storage device at the first multimedia display device.

42. The non-transitory computer-readable medium of claim 41, wherein the in-band data is closed-caption data.

43. The non-transitory computer-readable medium of claim 41, further comprising instructions for:

sending the event identification data to a second multimedia display device that is able to play a content stream.

44. The non-transitory computer-readable medium of claim 43, wherein the user specifies the second multimedia display device.

45. The non-transitory computer-readable medium of claim 41, further comprising instructions for:

sending the event identification data to a server; and

wherein the server distributes the event identification to other devices.

46. The non-transitory computer-readable medium of claim 41, wherein the at least one storage device is a hard disk.

47. The non-transitory computer-readable medium of claim 41, wherein the at least one storage device is any type of removable storage media.

48. The non-transitory computer-readable medium of claim 41, wherein the at least one storage device is any type of non-volatile and/or volatile storage medium.

49. The non-transitory computer-readable medium of claim 41, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

50. The non-transitory computer-readable medium of claim 49, wherein metadata associated with at least two hash values describes actions relating to one or more locations.

51. The non-transitory computer-readable medium of claim 41, further comprising instructions for:

receiving, at a second multimedia display device, the event identification data;

receiving a second content stream at the second multimedia display device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

processing, at the second multimedia display device, the in-band data of the second content stream by calculating hash values for the in-band data;

determining that at least two hash values of the second content stream matches a particular at least two hash values in the event identification data, wherein the event identification data associates the particular at least two hash values with a location in the second content stream; and

identifying the location within the second content stream.

52. The non-transitory computer-readable medium of claim 51, further comprising instructions for, in response to identifying the location, displaying an indication that the location is identified within the second content stream to a user while playing the second content stream to the user.

53. The non-transitory computer-readable medium of claim 51, wherein a user is able to skip to an identified location while playing the second content stream.

54. The non-transitory computer-readable medium of claim 51, wherein the second multimedia display device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

55. The non-transitory computer-readable medium of claim 51, wherein metadata associated with the identified location instructs the second multimedia display device to skip a segment of the second content stream.

56. The non-transitory computer-readable medium of claim 51, wherein metadata associated with the identified location instructs the second multimedia display device to reorder two or more segments of the second content stream.

57. The non-transitory computer-readable medium of claim 56, further comprising instructions for:

creating, at the second multimedia display device, a customized multimedia content stream containing the two or more re-ordered segments; and

displaying the customized content stream to a user.

58. The non-transitory computer-readable medium of claim 51, wherein metadata associated with the identified location instructs the second multimedia display device to store one or more segments of the second content stream on at least one storage device at the second multimedia display device, and wherein the at least one storage device contains one or more extracted segments from one or more different content streams.

59. The non-transitory computer-readable medium of claim 58, further comprising instructions for:

creating, at the second multimedia display device, a customized multimedia content stream containing any combination of segments stored on the at least one storage device; and

displaying the customized content stream to a user.

60. The non-transitory computer-readable medium of claim 41, wherein the first content stream is a television broadcast stream.

101. The non-transitory computer-readable medium of claim 59, wherein the customized content stream is based on instructions in the event identification data.

102. The non-transitory computer-readable medium of claim 51, wherein event identification data instructs the second multimedia display device to store one or more different content streams.

103. The non-transitory computer-readable medium of claim 102, further comprising:

creating, at the second multimedia playback device, a customized multimedia content stream based upon instructions in the event identification data containing any combination of segments or content streams stored on the at least one storage device; and

playing back the customized content stream to a user.

61. A method for creating and playback of customized multimedia streams, comprising:

reading a first content stream from a removable storage medium at a first multimedia playback device, wherein the first content stream contains in-band data;

playing back the first content stream to a user;

receiving input from the user during playing back of the first content stream indicating a location within the first content stream that the user has selected;

in response to receiving input from the user indicating the location, computing at least two hash values from the in-band data in a vicinity of the location and one or more time values, each time value indicating a period of time between generation of two sequential hash values of the at least two hash values;

receiving input from the user specifying an action associated with the location, wherein the action is saved as metadata associated with the hash values;

storing the at least two hash values, the one or more time values, and the metadata as event identification data on at least one storage device at the first multimedia playback device;

sending the event identification data to a second multimedia playback device;

receiving, at a second multimedia playback device, the event identification data;

reading a second content stream from a removable storage medium at the second multimedia playback device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

processing, at the second multimedia playback device, the in-band data of the second content stream by calculating at least two hash values for the in-band data;

determining, at the second multimedia playback device, that at least two hash values of the second content stream matches a particular at least two hash values in the event identification data, the event identification data associates the particular at least two hash values with a location in the second content stream; and

identifying, at the second multimedia playback device, the location within the second content stream, metadata associated with the identified locations instructs the second multimedia playback device to reorder two or more segments of the second content stream.

62. The method of claim 61, wherein the in-band data is closed-caption data.

63. The method of claim 61, wherein the in-band data is Enhanced Television (ETV) signaling data.

64. The method of claim 61, wherein the user specifies the second multimedia playback device.

65. The method of claim 61, wherein sending the event identification data to a second multimedia playback device further comprises:

sending the event identification data to a server; and

wherein the server distributes the event identification data to the second multimedia playback device.

66. The method of claim 61, wherein the at least one storage device is a hard disk.

67. The method of claim 61, wherein the at least one storage device is any type of removable storage media.

68. The method of claim 61, wherein the at least one storage device is any type of non-volatile and/or volatile storage medium.

69. The method of claim 61, wherein metadata associated with at least two hash values describes actions relating to one or more locations.

70. The method of claim 61, further comprising the step of:

in response to identifying the location, displaying an indication that the location is identified within the second content stream to a user while playing the second content stream to the user.

71. The method of claim 61, wherein a user is able to skip to an identified location while playing the second content stream.

72. The method of claim 61, wherein the second multimedia playback device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

73. The method of claim 61, wherein metadata associated with the identified location instructs the second multimedia playback device to skip a segment of the second content stream.

74. The method of claim 61, further comprising:

creating, at the second multimedia playback device, a customized multimedia content stream containing the two or more re-ordered segments; and

playing back the customized content stream to a user.

75. The method of claim 61, wherein metadata associated with the identified location instructs the second multimedia playback device to store one or more segments of the second content stream on at least one storage device at the second multimedia playback device, and wherein the at least one storage device contains one or more extracted segments from one or more different content streams.

76. The method of claim 75, further comprising:

creating, at the second multimedia playback device, a customized multimedia content stream containing any combination of segments stored on the at least one storage device; and

playing back the customized content stream to a user.

77. The method of claim 61, wherein the first content stream is a television broadcast stream.

78. An apparatus for creating and playback of customized multimedia streams, comprising:

a reading component that reads a first content stream from a removable storage medium at a first multimedia playback device, wherein the first content stream contains in-band data;

a user receiving component that receives input from the user during playback of the first content stream indicating a location within the first content stream that the user has selected;

a computational and identification component that computes, in response to receiving input from the user indicating the location, at least two hash values from the in-band data in a vicinity of the location and one or more time values, each time value indicating a period of time between generation of two sequential hash values of the at least two hash values;

an action receiving component that receives input from the user specifying an action associated with the location, wherein the action is saved as metadata associated with the location;

a storage component that stores the at least two hash values, the one or more time values, and the associated metadata as event identification data on at least one storage device at the first multimedia playback device;

a transmission component that sends the event identification data to a second multimedia playback device;

a receiving component at the second multimedia playback device that receives the event identification data;

a content receiving component that receives a second content stream at the second multimedia playback device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

an in-band processing component that processes, at the second multimedia playback device, the in-band data of the second content stream by calculating at least two hash values for the in-band data;

a processing component at the second multimedia playback device that determines that at least two hash values of the second content stream matches a particular at least two hash values in the event identification data, the event identification data associates the particular hash values with a location in the second content stream; and

a location identification component at the second multimedia playback device that identifies the location within the second content stream, metadata associated with the identified location instructs the second multimedia playback device to reorder two or more segments of the second content stream.

79. The apparatus of claim 78, wherein the in-band data is closed-caption data.

80. The apparatus of claim 78, wherein the in-band data is Enhanced Television (ETV) signaling data.

81. The apparatus of claim 78, wherein the user specifies the second multimedia playback device.

82. The apparatus of claim 78, wherein the transmission component that sends the event identification data to a second multimedia playback device further comprises:

a transmission component that sends the event identification data to a server; and

wherein the server distributes the event identification data to the second multimedia playback device.

83. The apparatus of claim 78, wherein the at least one storage device is a hard disk.

84. The apparatus of claim 78, wherein the at least one storage device is any type of removable storage media.

85. The apparatus of claim 78, wherein the at least one storage device is any type of non-volatile and/or volatile storage medium.

86. The method of claim 78, wherein metadata associated with at least two hash values describes actions relating to one or more locations.

87. The apparatus of claim 78, further comprising:

in response to identifying the location, a display component that displays an indication that the location is identified within the second content stream to a user while playing the second content stream to the user.

88. The apparatus of claim 78, wherein a user is able to skip to an identified location while playing the second content stream.

89. The apparatus of claim 78, wherein the second multimedia playback device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

90. The apparatus of claim 78, wherein metadata associated with the identified location instructs the second multimedia playback device to skip a segment of the second content stream.

91. The apparatus of claim 78, further comprising:

a generating component that creates, at the second multimedia playback device, a customized multimedia content stream containing the two or more re-ordered segments; and

a playback component that plays back the customized content stream to a user.

92. The apparatus of claim 78, wherein metadata associated with the identified location instructs the second multimedia playback device to store one or more segments of the second content stream on at least one storage device at the second multimedia playback device, and wherein the at least one storage device contains one or more extracted segments from one or more different content streams.

93. The apparatus of claim 92, further comprising:

a generating component that creates, at the second multimedia playback device, a customized multimedia content stream containing any combination of segments stored on the at least one storage device; and

a playback component that plays back the customized content stream to a user.

94. The apparatus of claim 78, wherein the content stream is a television broadcast stream.