Digital video signature apparatus and methods for use with video program identification systems

• US 20060153296A1
• Filed: 03/10/2006
• Published: 07/13/2006
• Est. Priority Date: 09/12/2003
• Status: Active Grant

• Current Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Original Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Inventors: Deng, Kevin
• Granted Patent: US 7,793,318 B2
• Time in Patent Office: Days
• US Class Current: 375/240.120
• CPC Class Codes: G06F 16/7847 using low-level visual features of the video content
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: H04H 20/14 for monitoring programmes
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 17/004 for digital television systems
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/2547 Third Party Billing, e.g. billing of advertiser
• CPC Class Codes: H04N 21/25891 being end-user preferences retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 7/173 with two-way working, e.g. subscriber sending a programme selection signal

Abstract

Digital video signature apparatus and methods for use with video program identification systems are disclosed. The disclosed apparatus and methods identify a video program using a sequence of signatures. Each of the signatures includes a set of centroids corresponding to one of a plurality of frames of the video program. The apparatus and methods compare the sequence of signatures to a set of reference sequences of signatures and identify the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures.

Claims

1-67. -67. (canceled)

68. A method of generating a video signature, comprising:

receiving compressed video information;

selecting a frame from the compressed video information based on a number of intra-coded macro blocks within the frame;

calculating image component centroid information based on the selected frame; and

generating the video signature using the image component centroid information.

69. A method as defined in claim 68, wherein the number of intra-coded macro blocks within the frame is a substantial portion of all macro blocks within the frame.

70. A method as defined in claim 68, further comprising extracting scaled image information from the frame and using the scaled image information to calculate the image component centroid information.

71. A method as defined in claim 70, wherein extracting the scaled image information from the frame includes extracting only intra-coded macro blocks from the frame.

72. A method as defined in claim 71, wherein extracting the scaled image information includes extracting only a portion of each of the intra-coded macro blocks.

73. A method as defined in claim 72, wherein the portion of each of the intra-coded macro blocks includes only DC coefficients.

74. A method as defined in claim 70, further comprising converting the scaled image information to spatial domain information and using the spatial domain information to calculate the image component centroid information.

75. A method as defined in claim 70, further comprising weighting the scaled image information prior to generating the video signature.

76. A method as defined in claim 68, wherein the compressed video information includes at least one MPEG frame.

77. A method as defined in claim 68, wherein the image component centroid information includes normalized centroid data.

78. A method as defined in claim 68, wherein the image component centroid information includes centroid information associated with a plurality of image components.

79. A method as defined in claim 78, wherein the plurality of image components includes image components associated with a color model.

80. A method as defined in claim 68, wherein generating the video signature using the image component centroid information includes generating a set of image component centroids associated with a plurality of image components.

81. A method as defined in claim 68, wherein generating the video signature using the image component centroid information includes generating a set of image component centroids associated with a plurality of areas within an image associated with the compressed video information.

82. A method as defined in claim 68, further comprising inverting at least some of the compressed image information in response to a determination that the at least some of the compressed image information corresponds to relatively dark foreground information on a relatively light background.

83. A method as defined in claim 68, further comprising determining that at least a portion of the compressed video information is substantially uniform and generating the video signature using a constant associated with an image component value instead of the image centroid information.

84. A method as defined in claim 83, wherein the constant is one of an average and a trend value associated with a plurality of images.

85. A system for generating a video signature, comprising:

a memory; and

a processor coupled to the memory and programmed to;

receive compressed video information;

select a frame from the compressed video information based on a number of intra-coded macro blocks within the frame;

calculate image component centroid information based on the selected frame; and

generate the video signature using the image component centroid information.

86. A system as defined in claim 85, wherein the number of intra-coded macro blocks within the frame is a substantial portion of all macro blocks within the frame.

87. A system as defined in claim 85, wherein the processor is programmed to extract scaled image information from the frame and to use the scaled image information to calculate the image component centroid information.

88. A system as defined in claim 87, wherein the processor is programmed to extract the scaled image information from the frame by extracting only intra-coded macro blocks from the frame.

89. A system as defined in claim 88, wherein the processor is programmed to extract the scaled image information by extracting only a portion of each of the intra-coded macro blocks.

90. A system as defined in claim 89, wherein the portion of each of the intra-coded macro blocks includes only DC coefficients.

91. A system as defined in claim 87, wherein the processor is programmed to convert the scaled image information to spatial domain information and to use the spatial domain information to calculate the image component centroid information.

92. A system as defined in claim 87, wherein the processor is programmed to weight the scaled image information prior to generating the video signature.

93. A system as defined in claim 85, wherein the compressed video information includes at least one MPEG frame.

94. A system as defined in claim 85, wherein the image component centroid information includes normalized centroid data.

95. A system as defined in claim 85, wherein the image component centroid information includes centroid information associated with a plurality of image components.

96. A system as defined in claim 95, wherein the plurality of image components includes image components associated with a color model.

97. A system as defined in claim 85, wherein the processor is programmed to generate the video signature using the image component centroid information by generating a set of image component centroids associated with a plurality of image components.

98. A system as defined in claim 85, wherein the processor is programmed to generate the video signature using the image component centroid information by generating a set of image component centroids associated with a plurality of areas within an image associated with the compressed video information.

99. A system as defined in claim 85, wherein the processor is programmed to invert at least some of the compressed image information in response to a determination that the at least some of the compressed image information corresponds to relatively dark foreground information on a relatively light background.

100. A system as defined in claim 85, wherein the processor is programmed to determine that at least a portion of the compressed video information is substantially uniform and generate the video signature using a constant associated with an image component value instead of the image centroid information.

101. A system as defined in claim 100, wherein the constant is one of an average and a trend value associated with a plurality of images.

102. A machine readable medium having instructions stored thereon that, when executed, cause a machine to:

receive compressed video information;

select a frame from the compressed video information based on a number of intra-coded macro blocks within the frame;

calculate image component centroid information based on the selected frame; and

generating a video signature using the image component centroid information.

103. A machine readable medium as defined in claim 102, wherein the number of intra-coded macro blocks within the frame is a substantial portion of all macro blocks within the frame.

104. A machine readable medium as defined in claim 102 having instructions stored thereon that, when executed, cause the machine to extract scaled image information from the frame and use the scaled image information to calculate the image component centroid information.

105. A machine readable medium as defined in claim 104 having instructions stored thereon that, when executed, cause the machine to extract the scaled image information from the frame by extracting only intra-coded macro blocks from the frame.

106. A machine readable medium as defined in claim 105 having instructions stored thereon that, when executed, cause the machine to extract the scaled image information by extracting only a portion of each of the intra-coded macro blocks.

107. A machine readable medium as defined in claim 106, wherein the portion of each of the intra-coded macro blocks includes only DC coefficients.

108. A machine readable medium as defined in claim 104 having instructions stored thereon that, when executed, cause the machine to convert the scaled image information to spatial domain information and use the spatial domain information to calculate the image component centroid information.

109. A machine readable medium as defined in claim 104 having instructions stored thereon that, when executed, cause the machine to weight the scaled image information prior to generating the video signature.

110. A machine readable medium as defined in claim 102, wherein the compressed video information includes at least one MPEG frame.

111. A machine readable medium as defined in claim 102, wherein the image component centroid information includes normalized centroid data.

112. A machine readable medium as defined in claim 102, wherein the image component centroid information includes centroid information associated with a plurality of image components.

113. A machine readable medium as defined in claim 112, wherein the plurality of image components includes image components associated with a color model.

114. A machine readable medium as defined in claim 102 having instructions stored thereon that, when executed, cause the machine to generate the video signature using the image component centroid information by generating a set of image component centroids associated with a plurality of image components.

115. A machine readable medium as defined in claim 102 having instructions stored thereon that, when executed, cause the machine to generate the video signature using the image component centroid information by generating a set of image component centroids associated with a plurality of areas within an image associated with the compressed video information.

116. A machine readable medium as defined in claim 102 having instructions stored thereon that, when executed, cause the machine to invert at least some of the compressed image information in response to a determination that the at least some of the compressed image information corresponds to relatively dark foreground information on a relatively light background.

117. A machine readable medium as defined in claim 102 having instructions stored thereon that, when executed, cause the machine to determine that at least a portion of the compressed video information is substantially uniform and generate the video signature using a constant associated with an image component value instead of the image centroid information.

118. A machine readable medium as defined in claim 117, wherein the constant is one of an average and a trend value associated with a plurality of images.

119. A system for generating a signature associated with a video program, the system comprising:

a frame scanner to scan video frames associated with the video program, wherein the frame scanner is configured to select a video frame based on a number of intra-coded macro blocks within the video frame;

an intra-coded macro block extractor to extract intra-coded macro blocks from the video frame;

a scaled image extractor to extract a scaled image from the intra-coded macro blocks; and

a signature generator to calculate a plurality of centroids associated with video frame based on the scaled image and to form the signature based on the plurality of centroids.

120. A system as defined in claim 119, wherein the frame scanner is adapted to selectively send substantially intra-coded video frames to the intra-coded macro block extractor.

121. A system as defined in claim 119, wherein the video program includes one of a television commercial and a television show.

122. A system as defined in claim 119, wherein the video frame is composed according to a digital video compression standard.

123. A system as defined in claim 122, wherein the digital video compression standard is based on the MPEG standard.

124. A system as defined in claim 119, wherein the plurality of centroids are associated with a plurality of image components.

125. A system as defined in claim 124, wherein the plurality of image components are associated with a color model.

126. A system as defined in claim 119, wherein the plurality of centroids are associated with a plurality of areas within the video frame.

127. A method of identifying a video program, comprising:

receiving a sequence of signatures, wherein each of the signatures includes a set of centroids corresponding to one of a plurality of substantially intra-coded frames of the video program;

comparing the sequence of signatures to a set of reference sequences of signatures; and

identifying the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures.

128. A method as defined in claim 127, wherein each of the sets of centroids includes centroids associated with image components.

129. A method as defined in claim 127, wherein the image components are associated with a color model.

130. A method as defined in claim 127, wherein each of the sets of centroids includes centroids associated with a plurality of areas within its corresponding one of the plurality of frames.

131. A method as defined in claim 127, wherein comparing the sequence of signatures to the set of reference sequences of signatures includes comparing the sequence of signatures to each reference sequence of signatures.

132. A method as defined in claim 131, wherein comparing the sequence of signatures to each reference sequence of signatures includes sequentially comparing each signature within the sequence of signatures to each of the signatures within at least one of the reference sequences of signatures.

133. A method as defined in claim 127, wherein identifying the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures includes identifying the video program in response to identifying a matching sequence within the set of reference sequences of signatures.

134. A method as defined in claim 127, wherein identifying the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures includes providing textual information describing the video program.

135. A method as defined in claim 127, wherein identifying the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures includes manually identifying the video program in response to a failure to identify a matching sequence within the set of reference sequences of signatures.

136. A method as described in claim 135, further comprising adjusting at least one reference sequence of signatures from the set of reference sequences of signatures in response to different encoding methods.

137. A method as defined in claim 127, wherein video program is one of a television commercial and a television show.

138. A system for identifying a video program, comprising:

a memory; and

a processor coupled to the memory and programmed to;

receive a sequence of signatures, wherein each of the signatures includes a set of centroids corresponding to one of a plurality of substantially intra-coded frames of the video program;

compare the sequence of signatures to a set of reference sequences of signatures; and

identify the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures.

139. A system as defined in claim 138, wherein each of the sets of centroids includes centroids associated with image components.

140. A system as defined in claim 139, wherein the image components are associated with a color model.

141. A system as defined in claim 138, wherein each of the sets of centroids includes centroids associated with a plurality of areas within its corresponding one of the plurality of frames.

142. A system as defined in claim 138, wherein the processor is programmed to compare the sequence of signatures to the set of reference sequences of signatures by comparing the sequence of signatures to each reference sequence of signatures.

143. A system as defined in claim 142, wherein the processor is programmed to compare the sequence of signatures to each reference sequence of signatures by sequentially comparing each signature within the sequence of signatures to each of the signatures within at least one of the reference sequences of signatures.

144. A system as defined in claim 138, wherein the processor is programmed to identify the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures by identifying the video program in response to identifying a matching sequence within the set of reference sequences of signatures.

145. A computer readable medium having instructions stored thereon that, when executed, cause a machine to:

receive a sequence of signatures, wherein each of the signatures includes a set of centroids corresponding to one of a plurality of substantially intra-coded frames of a video program;

compare the sequence of signatures to a set of reference sequences of signatures; and

identify the video program based on the comparison of the sequence of signatures to the set of reference sequences of signatures.

146. A computer readable medium as defined in claim 145, wherein each of the sets of centroids includes centroids associated with image components.

147. A computer readable medium as defined in claim 145, wherein each of the sets of centroids includes centroids associated with a plurality of areas within its corresponding one of the plurality of frames.