METHOD AND APPARATUS FOR PROVIDING SIGNATURES OF AUDIO/VIDEO SIGNALS AND FOR MAKING USE THEREOF

• US 20110128445A1
• Filed: 11/30/2009
• Published: 06/02/2011
• Est. Priority Date: 11/30/2009
• Status: Active Grant

• Current Assignee: Grass Valley Canada (Belden Inc.)
• Original Assignee: Miranda Technologies Limited (Belden Inc.)
• Inventors: Carrières, Pascal
• Granted Patent: US 8,860,883 B2
• Time in Patent Office: Days
• US Class Current: 348/512
• CPC Class Codes: H04N 17/00 Diagnosis, testing or measuring for television systems or their details
• CPC Class Codes: H04N 5/04 Synchronising for television systems using pulse code modulation H04N7/56

Abstract

A method and apparatus are disclosed for providing a video signature representative of a content of a video signal. A method and apparatus are further disclosed for providing an audio signature representative of a content of an audio signal. A method and apparatus for detecting lip sync are further disclosed and take advantage of the method and apparatus disclosed for providing a video signature and an audio signature.

Claims

1. A method for generating a video signature representative of a content of a video signal, the method comprising:

receiving pixel data of each of a plurality of given pixels in a first image of the video signal,

receiving pixel data of each of the plurality of given pixels in a second image subsequent to the first image,

for each given pixel of the plurality of given pixels,

comparing corresponding pixel data of the given pixel of the first image with the corresponding pixel data of the given pixel in the second image to provide a corresponding indication of a difference between the pixel data of the given pixel of the first image and the pixel data of the given pixel in the second image;

if the difference is greater than a threshold, incrementing a counter;

providing an indication of the counter to thereby generate said video signature.

2. The method as claimed in claim 1, further comprising selecting a plurality of given pixels according to at least one criteria.

3. The method as claimed in claim 2, wherein the at least one criteria comprises a spatial criteria.

4. The method as claimed in claim 1, wherein the threshold is equal to thirty two (32).

5. The method as claimed in claim 1, wherein the providing of the indication of the counter comprises normalizing the indication of the counter.

6. A method for generating an audio signature representative of a content of an audio signal, the method comprising:

receiving an audio signal;

performing a first filtering of the audio signal to provide a first filtered audio signal;

performing a second filtering of the audio signal to provide a second filtered audio signal;

comparing the first filtered audio signal to the second filtered audio signal; and

assigning a value depending on the comparison to thereby generate the audio signature representative of the content of the audio signal.

7. The method as claimed in claim 6, wherein said first filtering comprises detecting an envelope signal of the received audio signal.

8. The method as claimed in claim 7, wherein said second filtering comprises computing an average value of the received audio signal.

9. The method as claimed in claim 8, wherein the assigning of a value depending on the comparison to thereby generate the audio signature representative of the content of the audio signal comprises performing a decimation.

10. An audio signature extraction unit for providing an audio signature signal AS₁ representative of a content of an audio signal A_i, the audio signature extraction unit comprising:

a first filtering unit for receiving the audio signal A_i and for providing a first filtered signal;

a second filtering unit for receiving the audio signal A_i and providing a second filtered signal;

a comparator for receiving the first filtered signal and the second filtered signal and for performing a comparison of the first filtered with the second filtered signal and for providing a comparison result to thereby provide the audio signature signal AS_i representative of a content of an audio signal.

11. The audio signature extraction unit as claimed in claim 10, wherein said first filtering unit comprises an envelope detector for providing a signal E_s indicative of an envelope of the audio signal A_i.

12. The audio signature extraction unit as claimed in claim 11, wherein said second filtering unit comprises a mean detector for providing a signal M_s indicative of a mean the audio signal A_i.

13. The audio signature extraction unit as claimed in claim 12, further comprising an absolute value providing unit for receiving the audio signal A_i and for providing a signal indicative of an absolute value of the value of the audio signal A_i;

wherein each of said mean detector and said envelope detector receives said signal indicative of an absolute value of the value of the audio signal A_i.

14. The audio signature extraction unit as claimed in claim 13, further comprising a decimator for receiving the audio signature signal AS_i and for decimating the audio signature signal AS_i.

15. A video signature extraction unit for providing a video signature signal VS_i representative of a content of an incoming image signal V_i, the video signature extraction unit comprising:

a first image pixel data providing unit for receiving the incoming image signal V_i and for providing a corresponding pixel data P_k of a pixel k of a plurality of given pixels 1 to N of a first image of the incoming image signal V_i;

a second image pixel data providing unit for receiving the incoming image signal V_i and for providing a corresponding pixel data C_k of the corresponding given pixel k in a second image of the incoming image signal V_i;

a comparator for receiving the corresponding pixel data P_k and the corresponding pixel data C_k and for comparing the corresponding pixel data P_k with the corresponding pixel data C_k to provide a result signal indicative of the comparison;

a counter for receiving and counting the result signal of each of the plurality of given pixels 1 to N and for providing a counter signal; and

a video signature providing unit for receiving the counter signal and for providing the video signature signal VS_i.

16. The video signature extraction unit as claimed in claim 15, wherein the result signal is a logic value one (1) if |P_k−

C_k| is greater than thirty two (32) and a logic value zero (0) otherwise.

17. The video signature extraction unit as claimed in claim 16, wherein the video signature providing unit receives the counter signal and divides the counter signal by N/15, wherein N is the number of given pixels.

18. The video signature extraction unit as claimed in claim 15, further comprising a windowing unit for receiving the incoming image signal V_i and for providing a selected signal W_i.

19. An apparatus for providing an indication of a lip sync, the apparatus comprising:

a first signature extraction unit for receiving a first audio signal and a first video signal and for providing a first video signature signal and a first audio signature signal;

a second signature extraction unit for receiving a second audio signal and a second video signal and for providing a second video signature signal and a second audio signature signal;

a signature analysis unit for receiving the first video signature signal, the first audio signature signal, the second video signature signal and the second audio signature signal and for correlating the first video signature signal with the second video signature signal to provide a video delay and for further correlating the first audio signature signal with the second audio signature signal to provide an audio delay, said signature analysis unit for further comparing said audio delay with said video delay and for providing an indication of a lip sync is said audio delay is different from said video delay.

20. The apparatus as claimed in claim 19, wherein said first signature extraction unit comprises a first audio signature extraction unit for receiving said first audio signal and for providing said first audio signature signal, further wherein said first signature extraction unit comprises a first video signature extraction unit for receiving said first video signal and for providing said first video signature signal.

21. The apparatus as claimed in claim 20, wherein said second signature extraction unit comprises a second audio signature extraction unit for receiving said second audio signal and for providing said second audio signature signal, further wherein said second signature extraction unit comprises a second video signature unit for receiving said second video signal and for providing said second video signature signal.

22. The apparatus as claimed in claim 20, wherein said first audio signature extraction unit comprises:

a first filtering unit for receiving the first audio signal and for providing a first filtered signal;

a second filtering unit for receiving the first audio signal and for providing a second filtered signal;

a comparator for receiving the first filtered signal and the second filtered signal and for performing a comparison of the first filtered with the second filtered signal and for providing a comparison result to thereby provide the first audio signature signal.

23. The apparatus as claimed in claim 20, wherein said first video signature extraction unit comprises:

a first image pixel data providing unit for receiving said first video signal and for providing a corresponding pixel data P_k of a pixel k of a plurality of given pixels 1 to N of a first image of said first video signal;

a second image pixel data providing unit for receiving said first video signal and for providing a corresponding pixel data C_k of the corresponding given pixel k in a second image of said first video signal;

a comparator for receiving the corresponding pixel data P_k and the corresponding pixel data C_k and for comparing the corresponding pixel data P_k with the corresponding pixel data C_k to provide a result signal indicative of the comparison;

a counter for receiving and counting the result signal of each of the plurality of given pixels 1 to N and for providing a counter signal; and

a video signature providing unit for receiving the counter signal and for providing the first video signature signal.

24. The apparatus as claimed in claim 21, wherein said second audio signature extraction unit comprises:

a first filtering unit for receiving the second audio signal and for providing a first filtered signal;

a second filtering unit for receiving the second audio signal and for providing a second filtered signal;

a comparator for receiving the first filtered signal and the second filtered signal and for performing a comparison of the first filtered with the second filtered signal and for providing a comparison result to thereby provide the second audio signature signal.

25. The apparatus as claimed in claim 21, wherein said second video signature extraction unit comprises:

a first image pixel data providing unit for receiving said second video signal and for providing a corresponding pixel data P_k of a pixel k of a plurality of given pixels 1 to N of a first image of said second video signal;

a second image pixel data providing unit for receiving said second video signal and for providing a corresponding pixel data C_k of the corresponding given pixel k in a second image of said second video signal;

a comparator for receiving the corresponding pixel data P_k and the corresponding pixel data C_k and for comparing the corresponding pixel data P_k with the corresponding pixel data C_k to provide a result signal indicative of the comparison;

a counter for receiving and counting the result signal of each of the plurality of given pixels 1 to N and for providing a counter signal; and

a video signature providing unit for receiving the counter signal and for providing the second video signature signal.