Inter-network and inter-protocol video conference privacy method, apparatus, and computer program product

• US 7,509,553 B2
• Filed: 11/04/2003
• Issued: 03/24/2009
• Est. Priority Date: 11/04/2002
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Tandberg Telecom AS (Cisco Systems, Inc.)
• Inventors: Fuldseth, Arild, Kaljord, Andreas, Midtskogen, Steinar, Johansen, Tom-Ivar
• Primary Examiner(s): Lamarre; Guy J
• Application Number:
  US10/699,850
• Publication Number:
  US 20040168110A1
• Time in Patent Office: 1,967 Days
• Field of Search: 714/746, 714/747, 375/240.16, 375/240.13, 375/240.27
• US Class Current: 714/747
• CPC Class Codes: H04L 2209/30 Compression, e.g. Merkle-Damgard construction
• CPC Class Codes: H04L 2209/56 Financial cryptography, e.g. electronic payment or e-cash
• CPC Class Codes: H04L 2209/80 Wireless
• CPC Class Codes: H04L 2463/062 applying encryption of the keys
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 9/083 involving central third party, e.g. key distribution center [KDC] or trusted third party [TTP]
• CPC Class Codes: H04L 9/0841 involving Diffie-Hellman or related key agreement protocols
• CPC Class Codes: H04L 9/12 Transmitting and receiving encryption devices synchronised or initially set up in a particular manner
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/895 in combination with error concealment
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A system, apparatus, method, and computer program product for detecting and correcting packet losses that adversely affect video quality. This packet loss feature may be employed as part of the secure video conference system, method and computer program product, or be employed in a non-secure system, method and computer program product.

Claims

1. A video-conferencing system, comprising:

a decoder;

an encoder connected to the decoder by a network;

a packet loss detection mechanism co-located with the decoder, the packet loss detection mechanism configured to detect a lost packet and to transmit one of a packet loss notification message and a lost packet identification message; and

one of

a sustained packet loss detection mechanism co-located with one of the encoder and the packet loss detection mechanism, the sustained packet loss detection mechanism configured to receive the packet loss notification message and to inform the encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined packet loss scenario, and

an error concealment device co-located with the decoder and connected to the packet loss mechanism, the error concealment device configured to replace motion vector data lost from a current frame with motion vector data from a previous frame in response to the lost packet identification message;

wherein the encoder is further configured to determine one of a reliability of a reference block and a change in a reference block mean value;

wherein the encoder is configured to determine a reliability of a reference block on the basis of a period since the reference block was last updated as INTRA; and

wherein the encoder is configured to determine a reliability R_n of a reference block in a frame n is determined as follows;

R_n=1-p for INTRA blocks;

R_n=(1-p) R_n-1 for INTER blocks with non-zero difference signal; and

R_n=R_n-1 for INTER blocks with zero difference signals, where

p is a packet loss probability and R_n-1 is one of a reliability of a same block in a previous frame and an expression dependent on a reliability of neighbor blocks that includes non-zero motion vectors effects.

2. The video-conferencing system of claim 1, wherein

the encoder is configured to insert an INTER or INTRA block according to a first and a second insertion method if the packet loss notification message indicates the presence of the first and the second predetermined packet loss scenarios, respectively.

3. The video-conferencing system of claim 1, wherein the first predetermined packet loss scenario comprises:

a periodicity in a last N packet loss notifications that exceeds a predetermined packet loss periodicity value.

4. The video-conferencing system of claim 3, wherein the first predetermined packet loss scenario comprises:

at least K of N×

N ratios which are less than a predetermined threshold t, where

T₀, T₁, . . . and T_N are a set of time instances corresponding to a set of N most recent packet loss notifications;

d_i is a plurality of time differences d₀=T₁−

T₀, d₁=T₂−

T₁, . . . d<sub>N−

1</sub>=T_N−

T<sub>N−

1</sub> between packet loss notifications; and

the N×

N ratios equal r_i,j=|d_i|/|d_j| corresponding to every possible combination of two time differences.

5. The video-conferencing system of claim 1, wherein

the encoder is configured to insert a plurality of INTRA blocks separated by a predetermined distance corresponding to a packet loss occurrence rate.

6. The video-conferencing system of claim 5, wherein

the predetermined distance is configured to vary such that a higher packet loss occurrence rate results in a closer spacing of inserted INTRA blocks.

7. The video-conferencing system of claim 5, wherein

the encoder is configured to update an entire picture using INTRA blocks during a predetermined number of frames.

8. The video-conferencing system of claim 7, wherein

the encoder is configured to prevent a motion vector from pointing into an area of the entire picture that has not yet been updated by an INTRA block.

9. The video-conferencing system of claim 1, wherein

the encoder is configured to take anticipated packet losses into account when making a decision between inserting an INTRA block and inserting an INTER block.

10. The video-conferencing system of claim 1, wherein the encoder is configured to determine a change in mean value, MD_n for a block in a frame n as follows:

MD_n=|M_n−

M_n-1 |

where M_n and M_n-1 are mean values of a block in frame n and n-1, respectively.

11. The video-conferencing system of claim 10, wherein

the encoder is configured to determine a sum of absolute differences between original block pixels, c(i,j) and pixels pointed to by a chosen motion vector, p(i,j) as follows

12. The video-conferencing system of claim 11, wherein

the encoder is configured to choose constants a, b, c, and d such that a penalty term increases with a change in mean value MD_n, the packet loss rate p, and the bit rate r, and decreases with reliability R_n.

13. The video-conferencing system of claim 1, wherein

the error concealment device is configured to estimate a motion vector V′

_n(k,l) of a lost frame n in block position (k,l) with an actual motion vector V<sub>n−

1</sub>(k,l) of a previous frame n−

1 at block position (k,l).

14. The video-conferencing system of claim 13, wherein

the error concealment device is configured to set a residual of the lost frame n to a predetermined residual value.

15. The video-conferencing system of claim 14, wherein

the error concealment device is configured to set the residual of the lost frame n to zero.

16. A video-conferencing interface device connecting two networks, comprising:

a decoder connected to an upstream encoder by a first of the two networks;

an encoder connecting the decoder to a downstream decoder by a second of the two networks;

a packet loss detection mechanism connected to the decoder configured to detect a lost packet and to transmit one of a packet loss notification message and a lost packet identifier; and

one of a sustained packet loss detection mechanism connected to the packet loss detection mechanism and the upstream encoder, the sustained packet loss detection mechanism configured to receive the packet loss notification message and to inform the upstream encoder that the packet loss notification message indicates a presence of one of a first and a second predetermined upstream packet loss scenario, and an error concealment device connected to the decoder and the packet loss detection mechanism, the error concealment device configured to replace a lost motion vector with a replacement motion vector from a previous frame;

wherein the encoder is further configured to determine one of a reliability of a reference block and a change in a reference block mean value;

wherein the encoder is configured to determine a reliability of a reference block on the basis of a period since the reference block was last updated as INTRA; and

wherein the encoder is configured to determine a reliability R_n of a reference block in a frame n is determined as follows;

R_n=1-p for INTRA blocks;

R_n=(1-p) R_n-1 for INTER blocks with non-zero difference signal; and

R_n=R_n-1 for INTER blocks with zero difference signals, where

p is a packet loss probability and R_n-1 is one of a reliability of a same block in a previous frame and an expression dependent on a reliability of neighbor blocks that includes non-zero motion vectors effects.

17. The video-conferencing gateway device of claim 16, wherein

the encoder is connected to a second sustained packet loss notification device configured to identify a presence of a first and a second predetermined downstream packet loss scenario, the encoder further configured to insert an INTER or INTRA block according to one of a first and a second method corresponding to the presence of the first and the second predetermined downstream packet loss scenario.