System for identifying motion video/audio content

• US 8,577,077 B2
• Filed: 05/22/2008
• Issued: 11/05/2013
• Est. Priority Date: 05/22/2008
• Status: Active Grant

• Current Assignee: ZTE Corporation
• Original Assignee: Yuvad Technologies Co. Ltd
• Inventors: Zhang, Ji
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US12/085,752
• Publication Number:
  US 20100171879A1
• Time in Patent Office: 1,993 Days
• Field of Search: 382/100, 382/194, 725/19
• US Class Current: 382/100
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06V 20/48 Matching video sequences
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04H 2201/90 characterised by the use of signatures
• CPC Class Codes: H04H 60/37 for identifying segments of broadcast information, e.g. scenes or extracting programme ID
• CPC Class Codes: H04H 60/58 of audio determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00
• CPC Class Codes: H04H 60/59 of video recognising characters or patterns in general G06F18/00, G06V20/00
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A system for identifying motion video/audio content by means of comparing a video A to a registered video B so as to determine if they are originally the same as each other, wherein said system at least comprises a fingerprint extractor A for fingerprint A from the video A; a fingerprint database for the registered video B; and a matcher which is adapted to search from the fingerprint database for the registered fingerprint B of the video B by means of comparison of fingerprint A with a shifting window of a possible fingerprint B, so as to determine that the video A is visually identical to the video B if a match is found. According to the present invention, the method for extracting a fingerprint data from video/audio signals facilitates the automatic identification, archiving and search of video content, and can be of archiving without the need for human visual inspections.

Claims

1. A system for identifying motion video/audio content by means of comparing a video A to a registered video B so as to determine if they are originally the same as each other, wherein said system at least comprises

a fingerprint extractor A for fingerprint A from the video A;

a fingerprint database for the registered video B; and

a matcher which is adapted to search from the fingerprint database for the registered fingerprint B of the video B by means of comparison of fingerprint A with a shifting window of a possible fingerprint B, so as to determine that the video A is identical to the video B if a match is found,

wherein a SAD operation module and a SSAD operation module are provided,

the SAD operation module is adapted to calculate the sum of absolute difference (SAD) of every corresponding data samples between the fingerprint A and fingerprint B, the SAD operation is repeated for every video frames of the fingerprint A and fingerprint B, and

the SSAD operation module is adapted to add all the SAD operation results together to form a single sum of SAD (SSAD) number, the same process is then repeated by shifting the position of possible fingerprint B relative to A by one frame, each such shift results in a new SSAD value generated, so a series of SSAD values are generated and saved as a SSAD array, so a fingerprint match is identified by a sharp drop in the SSAD values just before the match and a sharp increase in SSAD values just after the match.

2. The system of claim 1, wherein a pattern array, a pattern extractor, a pattern store array, a pattern inspector, a pattern value collector, and a histogram collector are also provided,

the pattern array is adapted to store the SSAD values to detect the sharp drop pattern within the SSAD values;

the pattern extractor is adapted to inspect all neighboring SSAD values and obtains pattern values which are percentage of the change of neighboring SSAD values;

the pattern store array is adapted to store the extracted pattern values;

the pattern inspector is adapted to select the extracted pattern values within a window relative to a frame number;

the pattern value collector is adapted to perform the signed addition operations for all the extracted pattern values within the window together to yield a result value; and

the result value is compared with a user given threshold to determine if a match has been found between the fingerprint A and possible fingerprint B, if yes, the frame number is output to the histogram collector.

3. The system of claim 2, wherein the histogram collector is adapted to gather all of the pattern values that have exceeded the given threshold, count the number of times each value exceeds the threshold, and store them into an array, each item in the array holds the frame number, the pattern value, and the number of times that the pattern values have crossed the given threshold.

4. The system of claim 3, wherein a maximum value selector is adapted to inspect all such values within the histogram collector for the value that has appeared the most number of times, this value refers to the frame that is identified as the fingerprint matched frame, the output of the maximum value selector is then delivered to a formatter, which takes information from the relative position to determine on which frame position that a match has been identified.

5. The system of claim 3, wherein a threshold estimator is provided to determine if any pattern value is selected as an entry to the histogram collector, the estimator derives the threshold value from the fingerprint A array, it performs the SAD, SADD, pattern extractor operations between two identical fingerprint A arrays, and set the threshold value to be somewhere between 10% and 90% of the value when the two fingerprint arrays are offset by only one frame, the two groups of video A frames are frame-wise off-set by one frame, and the associated fingerprint A arrays then go through the SAD, SADD, pattern extractor and pattern value collector process to arrive at a pattern value.

6. The system of claim 1, wherein if the video B has less number of frames than the number of frames of video A plus 2M, the video images of the video B are repeated two, three or more times to merge into a single group of video images, where M is a constant which is chosen so that there are sufficient number of values of SSAD.

7. The system of claim 1, wherein the video B has less number of frames than the number of frames of video A plus 2M, it adds at least M frames to either sides of video B frames which have zero sample values, where M is a constant which is chosen so that there are sufficient number of values of SSAD.

8. The system of claim 1, wherein if the video B has the same number of frames as the video A, the roles of video A and video B are able to be reversed.

9. The system of claim 1, wherein a converter is used to make video A frames first go through it so as to form time-consecutive digitized video images if they are originally in analog format, and decompress the same if they are stored in a compressed data format.