System and method to estimate end-to-end video frame delays

• US 9,369,742 B2
• Filed: 12/06/2012
• Issued: 06/14/2016
• Est. Priority Date: 12/06/2012
• Status: Active Grant

• Current Assignee: Arlington Technologies, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Avaya Incorporated
• Inventors: Karacali-Akyamac, Bengi
• Primary Examiner(s): Tran, Hai V
• Application Number:
  US13/706,383
• Publication Number:
  US 20140165085A1
• Time in Patent Office: 1,286 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/24 Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests monitoring of server performance or load G06F11/34; arrangements for observation, testing or troubleshooting for broadcast or for distribution combined with broadcast H04H20/12
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/632 using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices broadcast-related systems characterised by transmission among terminal devices H04H60/80; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/64738 Monitoring network characteristics, e.g. bandwidth, congestion level data switched network analysis H04L41/14; monitoring functioning in data switched networks H04L43/0817; flow control in packet networks H04L47/10

Abstract

System and method to calculate video frame delay in a video stream received by a telecommunications endpoint, the method including: locating reference features characteristic of content in the received video stream; calculating, by use of a processor, reduced reference features from the located reference features; receiving reduced reference features of a transmitted video stream, the transmitted video stream corresponding to the received video stream; calculating, by use of a processor, a received trajectory of the reduced reference features from the received video stream; calculating, by use of a processor, a transmitted trajectory of the reduced reference features from the transmitted video stream; and calculating, by use of a processor, video frame delay as a time shift between the received trajectory and the transmitted trajectory.

Claims

1. A method to calculate video frame delay in a video stream received by a telecommunications endpoint, the method comprising:

locating reference features that are characteristic of content in the received video stream;

calculating, by use of a processor, reduced reference features from the located reference features;

receiving reduced reference features of a transmitted video stream, the transmitted video stream corresponding to the received video stream;

calculating, by use of a processor, a received trajectory of the reduced reference features from the received video stream;

calculating, by use of a processor, a transmitted trajectory of the reduced reference features from the transmitted video stream; and

calculating, by use of a processor, video frame delay as a time shift between the received trajectory and the transmitted trajectory.

2. The method of claim 1, wherein the reference features that are characteristic of content comprise faces.

3. The method of claim 1, wherein the reduced reference features comprise locations of the reference features.

4. The method of claim 1, wherein the reduced reference features comprise a location of rectangular areas surrounding the reference features.

5. The method of claim 1, wherein the reduced reference features are calculated at less than a full frame rate of the received video stream.

6. The method of claim 1, wherein the reduced reference features are calculated at no more than five frames per second.

7. The method of claim 1, wherein the reduced reference features of the transmitted video stream are received via a communication channel that is separate from a communication channel used to transport the video stream.

8. The method of claim 1, further comprising:

receiving a frame of the transmitted video stream to the telecommunications endpoint, once per second, via a reliable channel; and

calculating reduced reference features from the frame.

9. The method of claim 1, wherein the video stream originates from a remotely-located camera accessible via a communication path.

10. The method of claim 1, further comprising:

detecting an absence of reference features from the received video stream.

11. A system to calculate video frame delay in a video stream received by a telecommunications endpoint, the system comprising:

a computer-readable storage medium, storing executable instructions; and

a plurality of processors coupled to the computer-readable storage medium, the processors, when executing the executable instructions;

locate reference features that are characteristic of content in the received video stream;

calculate reduced reference features from the located reference features;

receive reduced reference features of a transmitted video stream, the transmitted video stream corresponding to the received video stream;

calculate a distance between the reduced reference features in the received video stream and the reduced reference features of the transmitted video stream;

calculate a received trajectory of the reduced reference features from the received video stream;

calculate a transmitted trajectory of the reduced reference features from the transmitted video stream; and

calculate video frame delay as a time shift between the received trajectory and the transmitted trajectory.

12. The system of claim 11, wherein the reference features that are characteristic of content comprise faces.

13. The system of claim 11, wherein the reduced reference features comprise a location of rectangular areas surrounding the reference features.

14. The system of claim 11, wherein the reduced reference features are calculated at less than a full frame rate of the received video stream.

15. The system of claim 11, wherein the reduced reference features are calculated at no more than five frames per second.

16. The system of claim 11, wherein the reduced reference features of the transmitted video stream are received via a communication channel that is separate from a communication channel used to transport the video stream.

17. The system of claim 11, wherein the video stream originates from a remotely-located camera accessible via a communication path.

18. The system of claim 11, wherein the processors, when executing the executable instructions:

detect an absence of reference features from the received video stream.