Method and apparatus for creating and viewing customized multimedia segments

• US 20070300250A1
• Filed: 06/22/2006
• Published: 12/27/2007
• Est. Priority Date: 06/22/2006
• Status: Active Grant

• Current Assignee: TiVo Solutions Inc. (Adeia Inc.)
• Original Assignee: TiVo Inc. (Adeia Inc.)
• Inventors: Smith, Kevin P., Stevens, Paul, Feder, Adam, Barton, James M.
• Granted Patent: US 7,873,982 B2
• Time in Patent Office: Days
• US Class Current: 725/20
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04H 20/31 using in-band signals, e.g. subsonic or cue signal
• CPC Class Codes: H04H 60/13 Arrangements for device control affected by the broadcast information
• CPC Class Codes: H04H 60/27 Arrangements for recording or accumulating broadcast information or broadcast-related information
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/73 using meta-information
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4147 PVR [Personal Video Recorder] H04N5/76 takes precedence ; arrangements for broadcast with accumulation-type receivers H04H20/40
• CPC Class Codes: H04N 21/43 Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/432 Content retrieval operation from a local storage medium, e.g. hard-disk details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/433 Content storage operation, e.g. storage operation in response to a pause request, caching operations
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/458 Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules ; time-related management operations arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10
• CPC Class Codes: H04N 21/47214 for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/4884 for displaying subtitles
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8545 : for generating interactive applications
• CPC Class Codes: H04N 21/858 : Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 5/782 : on tape
• CPC Class Codes: H04N 7/0885 : for the transmission of subtitles

Abstract

A system for creating and viewing customized multimedia segments allows a user to create event identification data from in-band data in a program stream. The in-band data is processed and compiled into event identification data associated with the content stream. The event identification data identifies patterns or sequences of in-band data associated with events in the content stream. Metadata are also included in the event identification data. The event identification data is provided to multimedia devices which receive multimedia content streams containing content streams with in-band data. The in-band data are parsed and compared to the event identification data. If a match occurs, the multimedia device uses the metadata to identify events or perform specified actions.

Claims

1. A method for creating and viewing customized multimedia streams, comprising:

receiving a first content stream at a first device, wherein the first content stream contains in-band data;

displaying the first content stream to a user;

receiving input from the user selecting one or more locations within the first content stream;

identifying a pattern of in-band data associated with a location within the first content stream;

identifying metadata associated with the location; and

storing the pattern and metadata as event identification data on a storage device.

2. The method of claim 1, wherein the in-band data is closed-caption data.

3. The method of claim 2, wherein identifying a pattern of in-band data includes computing a sequence of hash values from the closed caption data.

4. The method of claim 1, wherein the in-band data is Enhanced Television (ETV) signaling data.

5. The method of claim 1, further comprising:

sending the event identification data to a second device.

6. The method of claim 5, wherein the user specifies the second device.

7. The method of claim 1, further comprising:

sending the event identification data to a server; and

wherein the server distributes the event identification data to other devices.

8. The method of claim 1, wherein the storage device is a hard disk.

9. The method of claim 1, wherein the storage device is any type of removable storage media.

10. The method of claim 1, wherein the storage device is any type of non-volatile and/or volatile storage medium.

11. The method of claim 1, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

12. The method of claim 11, wherein metadata associated with a pattern can describe actions relating to one or more locations.

13. The method of claim 1, further comprising:

receiving, at a second device, the event identification data;

receiving a second content stream at the second device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

processing, at the second device, the in-band data of the second content stream;

determining that at least a portion of the in-band data of the second content stream matches a particular pattern among a plurality of patterns, wherein the event identification data associates the particular pattern with a location in the second content stream; and

identifying the location within the second content stream.

14. The method of claim 13, further comprising the step of:

in response to identifying the location, displaying the location within the second content stream to a user while playing the second content stream to a user.

15. The method of claim 13, wherein the in-band data is closed-caption data, wherein the determining step calculates hash values from the closed caption data, and wherein the particular pattern includes a sequence of hash values.

16. The method of claim 13, wherein a user can skip to a displayed location while playing the second content stream.

17. The method of claim 13, wherein the second device automatically skips a segment of the content stream upon reaching an identified location when playing the second content stream to a user.

18. The method of claim 17, wherein metadata associated with the identified location instructs the second device to skip the segment of the second content stream.

19. The method of claim 13, wherein metadata associated with the identified location instructs the second device to reorder two or more segments of the second content stream.

20. The method of claim 19, further comprising:

creating, at the second device, a customized multimedia content stream containing the two or more re-ordered segments; and

displaying the customized content stream to a user.

21. The method of claim 13, wherein metadata associated with the identified location instructs the second device to store one or more segments of the second content stream on a storage device, and wherein the storage device contains one or more extracted segments from one or more different content streams.

22. The method of claim 21, further comprising:

creating, at the second device, a customized multimedia content stream containing any combination of segments stored on the storage device,

displaying the customized content stream to a user.

23. The method of claim 1, wherein the first content stream is a television broadcast stream.

24. An apparatus for creating and viewing customized multimedia streams, comprising:

a module for receiving a first content stream at a first device, wherein the first content stream contains in-band data;

a module for receiving input from the user selecting one or more locations within the first content stream;

a module for identifying a pattern of in-band data associated with a location within the first content stream;

a module for identifying metadata associated with the location;

a module for storing the pattern and the associated metadata as event identification data on a storage device.

25. The apparatus of claim 24, wherein the in-band data is closed-caption data.

26. The apparatus of claim 25, wherein identifying a pattern of in-band data includes computing a sequence of hash values from the closed caption data.

27. The apparatus of claim 24, wherein the in-band data is Enhanced Television (ETV) signaling data.

28. The apparatus of claim 24, further comprising:

a module for sending the event identification data to a second device.

29. The apparatus of claim 28, wherein the user specifies the second device.

30. The apparatus of claim 24, further comprising:

a module for sending the event identification data to a server; and

wherein the server distributes the event identification data to other devices.

31. The apparatus of claim 24, wherein the storage device is a hard disk.

32. The apparatus of claim 24, wherein the storage device is any type of removable storage media.

33. The apparatus of claim 24, wherein the storage device is any type of non-volatile and/or volatile storage medium.

34. The apparatus of claim 24, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

35. The apparatus of claim 34, wherein metadata associated with a pattern can describe actions relating to one or more locations.

36. The apparatus of claim 34, further comprising:

a module for receiving, at a second device, the event identification data;

a module for receiving a second content stream at the second device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

a module for processing, at the second device, the in-band data of the second content stream;

a module for determining that at least a portion of the in-band data of the second content stream matches a particular pattern among a plurality of patterns, wherein the event identification data associates the particular pattern with a location in the second content stream; and

a module for identifying the location within the second content stream.

37. The apparatus of claim 36, further comprising:

in response to identifying the location, a module for displaying the location within the second content stream to a user while playing the second content stream to the user.

38. The apparatus of claim 36, wherein the in-band data is closed-caption data, wherein the determining module calculates hash values from the closed caption data, and wherein the particular pattern is a sequence of hash values.

39. The apparatus of claim 36, wherein a user can skip to an identified location while playing the second content stream.

40. The apparatus of claim 36, wherein the second device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

41. The apparatus of claim 40, wherein metadata associated with the identified location instructs the second device to skip the segment of the second content stream.

42. The apparatus of claim 36, wherein metadata associated with the identified location instructs the second device to reorder two or more segments of the second content stream.

43. The apparatus of claim 42, further comprising:

a module for creating, at the second device, a customized multimedia content stream containing the two or more re-ordered segments; and

a module for displaying the customized content stream to a user.

44. The apparatus of claim 36, wherein metadata associated with the identified location instructs the second device to store one or more segments of the second content stream on a storage device, and wherein the storage device contains one or more extracted segments from one or more different content streams.

45. The apparatus of claim 44, further comprising:

a module for creating, at the second device, a customized multimedia content stream containing any combination of segments stored on the storage device,

a module for displaying the customized content stream to a user.

46. The apparatus of claim 24, wherein the first content stream is a television broadcast stream.

47. A computer-readable medium carrying one or more sequences of instructions for creating and viewing customized multimedia streams, which instructions, when executed by one or more processors, cause the one or more processors to carry out the steps of:

receiving a first content stream at a first device, wherein the first content stream contains in-band data;

receiving input from the user selecting one or more locations within the first content stream;

identifying a pattern of in-band data associated with a location within the first content stream;

identifying metadata associated with the location; and

storing the pattern and the associated metadata as event identification data on a storage device.

48. The computer-readable medium of claim 47, wherein the in-band data is closed-caption data.

49. The computer-readable medium of claim 48, wherein identifying a pattern of in-band data includes computing a sequence of hash values from the closed caption data.

50. The computer-readable medium of claim 47, wherein the in-band data is Enhanced Television (ETV) signaling data.

51. The computer-readable medium of claim 47, further comprising instructions for:

sending the event identification data to a second device.

52. The computer-readable medium of claim 51, wherein the user specifies the second device.

53. The computer-readable medium of claim 47, further comprising instructions for:

sending the event identification data to a server; and

wherein the server distributes the event identification to other devices.

54. The computer-readable medium of claim 47, wherein the storage device is a hard disk.

55. The computer-readable medium of claim 47, wherein the storage device is any type of removable storage media.

56. The computer-readable medium of claim 47, wherein the storage device is any type of non-volatile and/or volatile storage medium.

57. The computer-readable medium of claim 47, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data

58. The computer-readable medium of claim 57, wherein metadata associated with a pattern can describe actions relating to one or more locations.

59. The computer-readable medium of claim 47, further comprising instructions for:

receiving, at a second device, the event identification data;

receiving a second content stream at the second device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

processing, at the second device, the in-band data of the second content stream;

determining that at least a portion of the in-band data of the second content stream matches a particular pattern among a plurality of patterns, wherein the event identification data associates the particular pattern with a location in the second content stream; and

identifying the location within the second content stream.

60. The computer-readable medium of claim 59, further comprising instructions for, in response to identifying the location, displaying the location within the second content stream to a user while playing the second content stream to the user.

61. The computer-readable medium of claim 59, wherein the in-band data is closed-caption data, wherein the determining step calculates hash values from the closed caption data, and wherein the particular pattern is a sequence of hash values.

62. The computer-readable medium of claim 60, wherein a user can skip to an identified location while playing the second content stream.

63. The computer-readable medium of claim 60, wherein the second device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

64. The computer-readable medium of claim 63, wherein metadata associated with the identified location instructs the second device to skip the segment of the second content stream.

65. The computer-readable medium of claim 59, wherein metadata associated with the identified location instructs the second device to reorder two or more segments of the second content stream.

66. The computer-readable medium of claim 65, further comprising instructions for:

creating, at the second device, a customized multimedia content stream containing the two or more re-ordered segments; and

displaying the customized content stream to a user.

67. The computer-readable medium of claim 59, wherein metadata associated with the identified location instructs the second device to store one or more segments of the second content stream on a storage device, and wherein the storage device contains one or more extracted segments from one or more different content streams.

68. The computer-readable medium of claim 67, further comprising instructions for:

creating, at the second device, a customized multimedia content stream containing any combination of segments stored on the storage device,

displaying the customized content stream to a user.

69. The computer-readable medium of claim 47, wherein the first content stream is a television broadcast stream.

70. A method for creating and viewing customized multimedia streams, comprising:

reading a first content stream from a removable storage medium at a first device, wherein the first content stream contains in-band data;

displaying the first content stream to a user;

receiving input from the user selecting one or more locations within the first content stream;

identifying a pattern of in-band data associated with a location within the first content stream;

identifying metadata associated with the location; and

storing the pattern and associated metadata as event identification data on a storage device.

71. The method of claim 70, wherein the in-band data is closed-caption data.

72. The method of claim 71, wherein identifying a pattern of in-band data includes computing a sequence of hash values from the closed caption data.

73. The method of claim 70, wherein the in-band data is Enhanced Television (ETV) signaling data.

74. The method of claim 70, further comprising:

sending the event identification data to a second device.

75. The method of claim 74, wherein the user specifies the second device.

76. The method of claim 70, further comprising:

sending the event identification data to a server; and

wherein the server distributes the event identification data to other devices.

77. The method of claim 70, wherein the storage device is a hard disk.

78. The method of claim 70, wherein the storage device is any type of removable storage media.

79. The method of claim 70, wherein the storage device is any type of non-volatile and/or volatile storage medium.

80. The method of claim 70, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

81. The method of claim 80, wherein metadata associated with a pattern can describe actions relating to one or more locations.

82. The method of claim 70, further comprising:

receiving, at a second device, the event identification data;

reading a second content stream from a removable storage medium at the second device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

processing, at the second device, the in-band data of the second content stream;

determining that at least a portion of the in-band data of the second content stream matches a particular pattern among a plurality of patterns, wherein the event identification data associates the particular pattern with a location in the second content stream; and

identifying the location within the second content stream.

83. The method of claim 82, further comprising the step of:

in response to identifying the location, displaying the location within the second content stream to a user while playing the second content stream to the user.

84. The method of claim 82, wherein the in-band data is closed-caption data, wherein the determining step calculates hash values from the closed caption data, and wherein the particular pattern is a sequence of hash values.

85. The method of claim 82, wherein a user can skip to a displayed location while playing the second content stream.

86. The method of claim 82, wherein the second device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

87. The method of claim 86, wherein metadata associated with the identified location instructs the second device to skip the segment of the second content stream.

88. The method of claim 82, wherein metadata associated with the identified location instructs the second device to reorder two or more segments of the second content stream.

89. The method of claim 88, further comprising:

creating, at the second device, a customized multimedia content stream containing the two or more re-ordered segments; and

displaying the customized content stream to a user.

90. The method of claim 82, wherein metadata associated with the identified location instructs the second device to store one or more segments of the second content stream on a storage device, and wherein the storage device contains one or more extracted segments from one or more different content streams.

91. The method of claim 90, further comprising:

creating, at the second device, a customized multimedia content stream containing any combination of segments stored on the storage device,

displaying the customized content stream to a user.

92. The method of claim 70, wherein the first content stream is a television broadcast stream.

93. An apparatus for creating and viewing customized multimedia streams, comprising:

a module for reading a first content stream from a removable storage medium at a first device, wherein the first content stream contains in-band data;

a module for receiving input from the user selecting one or more locations within the first content stream;

a module for identifying a pattern of in-band data associated with a location within the first content stream;

a module for identifying metadata associated with the location;

a module for storing the pattern and associated metadata as event identification data on a storage device.

94. The apparatus of claim 93, wherein the in-band data is closed-caption data.

95. The apparatus of claim 94, wherein identifying a pattern of in-band data includes computing a sequence of hash values from the closed caption data.

96. The apparatus of claim 93, wherein the in-band data is Enhanced Television (ETV) signaling data.

97. The apparatus of claim 93, further comprising:

a module for sending the event identification data to a second device.

98. The apparatus of claim 97, wherein the user specifies the second device.

99. The apparatus of claim 93, further comprising:

a module for sending the event identification data to a server; and

wherein the server distributes the event identification data to other devices.

100. The apparatus of claim 93, wherein the storage device is a hard disk.

101. The apparatus of claim 93, wherein the storage device is any type of removable storage media.

102. The apparatus of claim 93, wherein the storage device is any type of non-volatile and/or volatile storage medium.

103. The apparatus of claim 93, wherein the user specifies an action associated with each selected location, and wherein the action associated with each selected location is saved as metadata in the event identification data.

104. The apparatus of claim 103, wherein metadata associated with a pattern can describe actions relating to one or more locations.

105. The apparatus of claim 103, further comprising:

a module for receiving, at a second device, the event identification data;

a module for reading a second content stream from a removable storage medium at the second device, the second content stream containing at least a portion of the in-band data contained in the first content stream;

a module for processing, at the second device, the in-band data of the second content stream;

a module for determining that at least a portion of the in-band data of the second content stream matches a particular pattern among a plurality of patterns, wherein the event identification data associates the particular pattern with a location in the second content stream; and

a module for identifying the location within the second content stream.

106. The apparatus of claim 105, further comprising:

in response to identifying the location, a module for displaying the location within the second content stream to a user while playing the second content stream to the user.

107. The apparatus of claim 105, wherein the in-band data is closed-caption data, wherein the determining module calculates hash values from the closed caption data, and wherein the particular pattern is a sequence of hash values.

108. The apparatus of claim 105, wherein a user can skip to an identified location while playing the second content stream.

109. The apparatus of claim 105, wherein the second device automatically skips a segment of the second content stream upon reaching an identified location when playing the second content stream to a user.

110. The apparatus of claim 109, wherein metadata associated with the identified location instructs the second device to skip the segment of the second content stream.

111. The apparatus of claim 105, wherein metadata associated with the identified location instructs the second device to reorder two or more segments of the second content stream.

112. The apparatus of claim 111, further comprising:

a module for creating, at the second device, a customized multimedia content stream containing the two or more re-ordered segments; and

a module for displaying the customized content stream to a user.

113. The apparatus of claim 105, wherein metadata associated with the identified location instructs the second device to store one or more segments of the second content stream on a storage device, and wherein the storage device contains one or more extracted segments from one or more different content streams.

114. The apparatus of claim 113, further comprising:

a module for creating, at the second device, a customized multimedia content stream containing any combination of segments stored on the storage device,

a module for displaying the customized content stream to a user.

115. The apparatus of claim 93, wherein the content stream is a television broadcast stream.