Automatic video glitch detection and audio-video synchronization assessment

• US 8,379,735 B2
• Filed: 05/15/2006
• Issued: 02/19/2013
• Est. Priority Date: 05/15/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Lee, Mingtzong, Yildiz, Huseyin S.
• Primary Examiner(s): Patel, Jay Au
• Assistant Examiner(s): TORRENTE, RICHARD T
• Application Number:
  US11/383,444
• Publication Number:
  US 20080037954A1
• Time in Patent Office: 2,472 Days
• Field of Search: 375/240.28, 348/512
• US Class Current: 375/240.28
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23892 involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

A process to detect video glitches and audio-video synchronization errors arising during playback or capture. The process encodes an external time stamp into video and audio streams directly at the output of a digital media playback device. The process captures the stamped video and audio streams and analyzes the external stamp by comparing it with a presentation time stamp and a capture time stamp.

Claims

1. A method, comprising:

adding, by a computing device, a first time stamp to a first area of a video frame, to a second area of the video frame, and to a first portion of an audio signal corresponding to the video frame;

adding, by the computing device, a first time code to a third area of the video frame and to a second portion of the audio signal;

recognizing, by the computing device, the first time stamp and the first time code;

creating, by the computing device, a data array that includes information about the video frame and any other related frames included in a series of video frames;

adding, by the computing device, the first time stamp and the first time code into an entry of the data array; and

searching, by the computing device, the data array to find an entry where the first time stamp is equal to the first time code.

2. The method of claim 1, further comprising determining a time-based offset using the first time stamp and the first time code.

3. The method of claim 1, wherein the first time stamp includes a bar code representing a time value.

4. The method of claim 1, wherein the first time stamp includes a bar code representing a time value.

5. The method of claim 1, wherein the adding a first time stamp is performed before the video frame is converted to an analog form.

6. The method of claim 1, wherein the adding a first time stamp is performed before the audio signal is converted to an analog form.

7. The method of claim 1, wherein the first time code is added after the video frame has been converted to an analog form.

8. The method of claim 1, wherein the first time code is added after the audio frame has been converted to an analog form.

9. The method of claim 1, wherein the recognizing is performed by an optical character recognition.

10. The method of claim 1, wherein the first area and second area of the video frame are non-visible.

11. The method of claim 1, wherein the first portion of the audio signal is represented with an audio signal encoding scheme which is not perceivable by human beings.

12. The method of claim 1, further comprising analyzing at least one difference between the first time stamp and first time code in a different entry of the data array.

13. A system, comprising:

a video rendering component for rendering a first time stamp to a first area and to a second area of a frame of video;

an audio rendering component for rendering the first time stamp to a first portion of audio information corresponding to the frame of video;

a time code generator for adding a first time code to a third area of the frame of video and to a second portion of the audio information;

an analysis component for;

determining the first time stamp and the first time code;

converting the first time stamp to a time-based value based at least in part on an expected duration of time that the frame of video is to be displayed and a sequential number value of the first time stamp; and

calculating a difference of the time-based value and a second time-based value associated with the audio information.

14. The system of claim 13, wherein the first time stamp is added to the first area and the second area of the frame of video before the frame of video is received by the system.

15. The system of claim 13, wherein the first time stamp was added to the first portion of audio information corresponding to the frame of video before the first portion of audio is received by the system.

16. The system of claim 13, wherein the first area is left of a vertical center point of the frame of video and is above a horizontal center point of the frame of video.

17. The system of claim 13, wherein the second area is right of a vertical center point of the frame of video and is below a horizontal center point of the frame of video.

18. A frame of video, comprising:

a first time stamp corresponding to a presentation time of the frame of video and corresponding to a first area and a second area of the frame of video, the first time stamp being rendered to a first portion of audio information corresponding to the frame of video;

a second time stamp corresponding to the presentation time of the frame of video; and

a first time code corresponding to a display time of the frame of video, the first time code being added to a third area of the frame of video and to a second portion of the audio information,

wherein the first time stamp is converted to a time-based value based at least in part on an expected duration of time that the frame of video is to be displayed and a sequential number value of the first time stamp, a difference of the time-based value and a second time-based value associated with the audio information being calculated.

19. The frame of video of claim 18, wherein the first time stamp is an undetectable watermark and the second time stamp is an undetectable watermark.

20. The frame of video of claim 18, wherein the first time code is a Society of Motion Picture Television Engineers (SMPTE) signal.