Layered DCT video coder for packet switched ATM networks

• US 5,260,783 A
• Filed: 02/21/1991
• Issued: 11/09/1993
• Est. Priority Date: 02/21/1991
• Status: Expired due to Term

• Current Assignee: Verizon Laboratories Incorporated (Verizon Communications Inc.)
• Original Assignee: GTE Laboratories Incorporated (Lumen Technologies, Inc.)
• Inventors: Dixit, Sudhir S.
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): LEE, MICHAEL
• Application Number:
  US07/659,191
• Time in Patent Office: 992 Days
• Field of Search: 358/133, 358/135, 358/136, 358/105, 358/11, 358/12, 358/13, 358/141, 382/56, 382/49, 370/60, 370/94.1
• US Class Current: 375/240.13
• CPC Class Codes: H04L 2012/5616 Terminal equipment, e.g. codecs, synch.
• CPC Class Codes: H04L 2012/5652 Cell construction, e.g. including header, packetisation, depacketisation, assembly, reassembly
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04Q 11/0478 Provisions for broadband connections

Abstract

A digital video encoder encodes a video frame into a differential video frame for transmission over a packet switched network. The video encoder includes an inter-frame encoder, an intra-frame encoder, and an encoding selector for selecting between the inter-frame and intra-frame encoder depending on the relative motion between the video frame being encoded and the previous video frame.

A composite frame combiner provides a composite intra/inter-frame encoded difference frame having one set of pixels encoded by the inter-frame encoder, and another set of pixels encoded by the intra-frame encoder. The set of intra-frame encoded pixels includes at least one square or rectangular pixel block, a vertical strip of pixel blocks, and a horizontal strip of pixel blocks.

Difference frames are encoded into separable data sets representing video information within a particular range of image resolution. A discrete cosine transform (DCT) is used to transform the difference pixels into corresponding DCT coefficients which are separable, by resolution, into the data sets providing coefficient layers.

A packetizer formats the data sets into asynchronous transfer mode (ATM) packets for transmission over network.

Claims

1. An apparatus for encoding a digital video frame f_i, having a matrix with N rows and M columns of pixels Δ

x_i (n,m), for transmission over a digital communications channel, comprising

an inter-frame encoder for encoding a said pixel x_i (n,m) of said video frame f_i into a corresponding differentially encoded pixel Δ

x_i (n,m)=x_i (n,m)-x_i (n,m) of a difference frame Δ

f_i dependent on the corresponding pixel x_i-1 (n,m) of a previous video frame f_i-1,

an intra-frame encoder for encoding a said pixel x_i (n,m) of frame f_i into a corresponding differentially encoded pixel Δ

x_i (n,m) of said difference frame Δ

f_i (n,m) dependent on other said pixels of the same said video frame f_i, and

an encoding selector for selecting between said inter-frame encoder and said intra-frame encoder for differentially encoding pixels x_i (n,m) of frame f_i into corresponding differentially encoded pixels Δ

x_i (n,m) of said difference frame Δ

f_i, said encoding selector being responsive to the relative motion between said video frame f_i and said previous video frame f_i-1,

said coding selector further including a motion detector for detecting the relative motion between said video frame f_i and said previous vide frame f_i-1, and for providing a decision parameter K representing the level of detected relative motion, K being computed by

space="preserve" listing-type="equation">K=k.sub.1 Δ

x.sub.i +k.sub.2 σ

where Δ

x_i and σ

² are an estimate of the mean and variance, respectively, of difference pixel value Δ

x_i (n,m), k1 and k2 are weighing coefficients each having a value between 0 and 1 such that k₁ +k₂ =1,

a layered resolution encoder for encoding said differential pixels Δ

x_i (n,m) of frame Δ

f_i into a plurality of separable data sets, each said data set representing video information, within a particular range of video image resolution, about said differential pixels Δ

x_i (n,m),

a packetizer for formatting said plurality of data sets, into at least one asynchronous transfer mode (ATM) packet for transmission over the digital communications channel, said ATM packet comprising a header field portion having data for establishing a virtual communications channel between selected devices on the digital communications channel, and an information field portion for transferring said plurality of data sets between said selected devices,

wherein said encoding selector responds to decision parameter K by selecting said inter-frame encoder when K<

T and by selecting said intra-frame encoder when K≧

T, where T represents a threshold parameter T having a value between 0 and 1.

2. The apparatus of claim 1, wherein

said estimate of means difference pixel value Δ

x_i is computed by ##EQU3## and, said estimate of means difference pixel variance σ

² is computed by ##EQU4## where Δ

x_i (j) is a short notation for J randomly selected pixels Δ

x_i (m,n) of difference frame Δ

f_i.

3. The apparatus of claim 1 wherein said encoding selector comprises

a composite frame combiner for providing a differentially encoded difference frame Δ

f_i having a first set of difference pixels Δ

x_i (m,n) encoded by said inter-frame encoder, and a second set of pixels Δ

x_i (m,n) encoded by said intra-frame encoder.

4. The apparatus of claim 3 wherein said second set of pixels Δ

x_i (m,n) comprises

at least one square pixel block having P pixels Δ

x_i (m,n) on each side, where N/P, and M/P are integer values.

5. The apparatus of claim 4 wherein said second set of pixels Δ

x_i (m,n) comprises

a vertical strip portion having a width of P pixels and a length of N pixels such that said vertical strip portion comprises a quantity N/P of said pixel blocks arranged vertically aligned and non-overlapping.

6. The apparatus of claim 5 wherein

said vertical strip portion of said second set of pixels Δ

x_i+1 (m,n) of frame f_i+1 is offset horizontally in position from said vertical strip portion of said second set of pixels Δ

x_i (m,n) of frame f_i by at least P said pixels.

7. The apparatus of claim 4 wherein said second set of pixels Δ

x_i (m,n) comprises

a horizontal strip portion having a width of M pixels and a length of P pixels such that said horizontal strip portion comprises a quantity M/P of said pixel blocks arranged horizontally aligned and non-overlapping.

8. The apparatus of claim 7 wherein

said horizontal strip portion of said second set of pixels Δ

x_i+1 (m,n) of frame f_i+1 is offset vertically in position from said horizontal strip portion of said second set of pixels Δ

x_i (m,n) of frame f_i by at least P said pixels.

9. The apparatus of claim 1 wherein

said layered resolution encoder comprises a discrete cosine transform (DCT) for transforming said differential pixels Δ

x_i (n,m) into a plurality of DCT coefficients representing said differential pixels, said DCT coefficients separable into said plurality of said data sets providing coefficient layers.

10. The apparatus of claim 9 wherein

said difference frame Δ

f_i comprises a plurality of square pixel blocks having P differential pixels Δ

x_i (m,n) on each side, where N/P, and M/P are integer values, and

said layered resolution encoder transforms said differential pixels within each said pixel block into a corresponding square coefficient block of DCT coefficients, having P coefficients on each side, representing video information about said differential pixels Δ

x_i (n,m) within said corresponding pixel block.

12. The apparatus of claim 1 wherein said information field portion comprises an adaptation overhead field portion comprising

a cell sequence number for indicating the temporal relationship of said ATM packet relative to other said ATM packets, and

a sync flag taking one of a plurality of states for indicating the composition of the remainder of said information field portion, said sync flag taking a first state when said information field portion includes a first type of information field, and a second state when said information field portion includes a second type of information field.

13. The apparatus of claim 12 wherein said first type of information field comprises an adaptation overhead field portion further comprising

a coding mode field for indicating whether said data sets are encoded in said inter-frame or said intra-frame coding mode, and

a component type field for indicating to which said color video component said data sets belong.

14. The apparatus of claim 12 wherein

said difference frame Δ

f_i comprises a plurality of square pixel blocks having P differential pixels Δ

x_i (m,n) on each side, where N/P, and M/P are integer values,

said layered resolution encoder comprises a discrete cosine transform (DCT) for transforming said differential pixels Δ

x_i (n,m) in each said pixel block into a corresponding block of DCT coefficients representing said differential pixels in said corresponding pixel block, said DCT coefficients separable into said plurality of said data sets providing coefficient layers for each said DCT coefficient block, and

said first type of information field comprises an adaptation overhead field portion further comprising a resolution information field for indicating which said coefficient layers are available to be transferred between said devices for each said coefficient block.

15. The apparatus of claim 14 wherein said resolution information field comprises

a data bit corresponding to each said coefficient block, each said data bit taking a first state when no said coefficient layers are available to be transferred for said corresponding coefficient block, and each said data bit taking a second state when all said coefficient layers are available to be transferred for said corresponding coefficient block.

16. The apparatus of claim 14 wherein said resolution information field comprises

a block resolution field corresponding to each said coefficient block, each said block resolution field taking one of a plurality of states for indicating which said coefficient layers are available to be transferred for said corresponding coefficient block.

11. The apparatus of claim 11 wherein said DCT coefficients within each said coefficient block are separated into said coefficient layers.

17. A method for encoding a digital video frame f_i, having N rows and M columns of pixels x_i (n,m), for transmission over a digital communications channel, comprising the steps of

selecting between an inter-frame encoding mode and an intra-frame encoding mode for differentially encoding pixels x_i (n,m) of a video frame f_i into corresponding differentially encoded pixels Δ

x_i (n,m) of a difference frame Δ

f_i, said selection being determined by the relative motion between said video frame f_i and a previous video frame f_i-1,

said step of selecting between an inter-frame encoding mode and an intra-frame encoding mode further including detecting the relative motion between said video frame f_i and said previous video frame f_i-1, and computing a decision parameter K representing the level of detected relative motion, K being computed by

space="preserve" listing-type="equation">K=k.sub.1 Δ

x.sub.i +k.sub.2 σ

where Δ

x_i and σ

² are an estimate of the mean and variance, respectively, of difference pixel values Δ

x_i (n,m), k1 and k2 are weighting coefficients each having a value between 0 and 1 such that k₁ +k₂ =1, selecting said inter-frame encoder when K<

T, selecting said intra-frame encoder when K≧

T, where T represents a threshold parameter T having a value between 0 and 1,

if said inter-frame encoding mode is selected, then encoding a said pixel x_i (n,m) of frame f_i into a corresponding differentially encoded pixel Δ

x_i (n,m) of said difference frame Δ

f_i, dependent on the corresponding pixel x_i-1 (n,m) of said previous vide frame f_i-1, and

if said intra-frame encoding mode is selected, then encoding a said pixel x_i (n,m) of said frame f_i into a corresponding differentially encoded pixel Δ

x_i (n,m) computed by Δ

x_i (n,m)=x_i (n,m)-x_i-1 (n,m) dependent on other said pixels of the same said video frame f_i,

layer encoding said differential pixels Δ

x_i (n,m) of frame Δ

f_i into a plurality of separable data sets, each said data set representing video information, within a particular range of video image resolution, about said differential pixels Δ

x_i (n,m),

formatting said plurality of data sets, into at least one asynchronous transfer mode (ATM) packet for transmission over the digital communications channel, said ATM packet comprising a header field portion having data for establishing a virtual communications channel between selected devices on the digital communications channel, and an information field portion for transferring said plurality of data sets between said selected devices.

18. The method of claim 17, wherein

said estimate of mean difference pixel value Δ

x_i is computed by ##EQU5## and, said estimate of mean difference pixel variance σ

² is computed by ##EQU6## where Δ

x_i (j) is a short notation for J randomly selected pixels Δ

x_i (m,n) of difference frame Δ

f_i.

19. The method of claim 17 wherein said selecting step comprises

combining said inter-frame and intra-frame encoding modes to provide a differentially encoded difference frame Δ

f_i having a first set of pixels Δ

x_i (m,n) encoded by said inter-frame encoding mode, and a second set of pixels Δ

x_i (m,n) encoded by said intra-frame encoding mode.

20. The method of claim 19 wherein said second set of pixels Δ

x_i (m,n) comprises

at least one square pixel block having P pixels Δ

x_i (m,n) on each side, where N/P, and M/P are integer values.

21. The method of claim 20 wherein said second set of pixels Δ

x_i (m,n) comprises

a vertical strip portion having a width of P pixels and a length of N pixels such that said vertical strip portion comprises a quantity N/P of said pixel blocks arranged vertically aligned and non-overlapping.

22. The method of claim 21 further comprising the step of

horizontally offsetting the position of said vertical strip portion of said second set of pixels Δ

x_i+1 (m,n) of frame f_i+1 from the position of said vertical strip portion of said second set of pixels Δ

x_i (m,n) of frame f_i by at least P said pixels.

23. The method of claim 20 wherein said second set of pixels Δ

x_i (m,n) comprises

a horizontal strip portion having a width of M pixels and a length of P pixels such that said horizontal strip portion comprises a quantity M/P of said pixel blocks arranged horizontally aligned and non-overlapping.

24. The method of claim 23 further comprising the step of

vertically offsetting the position of said horizontal strip portion of said second set of pixels Δ

x_i+1 (m,n) of frame f_i+1 from the position of said horizontal strip portion of said second set of pixels Δ

x_i (m,n) of frame f_i by at least P said pixels.

25. The method of claim 17 wherein said layer encoding step comprises

transforming said differential pixels Δ

x_i (n,m) into a plurality of discrete cosine transform (DCT) coefficients representing said differential pixels, and

separating said DCT coefficients into said plurality of said data sets providing coefficient layers.

26. The method of claim 25 wherein

said difference frame Δ

f_i comprises a plurality of square pixel blocks having P differential pixels Δ

x_i (m,n) on each side, where N/P, and M/P are integer values, and

said transforming step transforms said differential pixels within each said pixel block into a corresponding square coefficient block of DCT coefficients, having P coefficients on each side, representing video information about said differential pixels Δ

x_i (n,m) within said corresponding pixel block.

28. The method of claim 17 wherein said information field portion comprises an adaptation overhead field portion comprising

a cell sequence number for indicating the temporal relationship of said ATM packet relative to other said ATM packets, and

a sync flag taking one of a plurality of states for indicating the composition of the remainder of said information field portion, said sync flag taking a first state when said information field portion includes a first type of information field, and a second state when said information field portion includes a second type of information field.

29. The method of claim 28 wherein said first type of information field comprises an adaptation overhead field portion further comprising

a coding mode field for indicating whether said data sets are encoded in said inter-frame or said intra-frame coding mode, and

a component type field for indicating to which said color video component said data sets belong.

30. The method of claim 28 wherein

said difference frame Δ

f_i comprises a plurality of square pixel blocks having P differential pixels Δ

x_i (m,n) on each side, where N/P, and M/P are integer values,

said layer encoding step comprises transforming said differential pixels Δ

x_i (n,m) in each said pixel block into a corresponding block of discrete cosine transform (DCT) coefficients representing said differential pixels in said corresponding pixel block, and separating said DCT coefficients into said plurality of said data sets providing coefficient layers for each said DCT coefficient block, and

wherein said first type of information field comprises an adaptation overhead field portion further comprising a resolution information field for indicating which said coefficient layers are available to be transferred between said devices for each said coefficient block.

31. The method of claim 20 wherein said resolution information field comprises

a data bit corresponding to each said coefficient block, each said data bit taking a first state when no said coefficient layers are available to be transferred for said corresponding coefficient block, and each said data bit taking a second state when all said coefficient layers are available to be transferred for said corresponding coefficient block.

32. The method of claim 30 wherein said resolution information field comprises

a block resolution field corresponding to each said coefficient block, each said block resolution field taking one of a plurality of states for indicating which said coefficient layers are available to be transferred for said corresponding coefficient block.

33. An apparatus for transferring a digital video frame f_i, having N rows and M columns of pixels x_i (n,m), for transmission over a digital communications channel, comprising

an inter-frame encoder for encoding a said pixel x_i (n,m) of a video frame f_i into a corresponding differentially encoded pixel Δ

x_i (n,m) of a difference frame Δ

f_i dependent on the corresponding pixel x_i-1 (n,m) of a previous video frame f_i-1,

an intra-frame encoder for encoding a said pixel x_i (n,m) of a video frame f_i into a corresponding differentially encoded pixel Δ

x_i (n,m) dependent on other pixels of the same said frame f_i,

an encoding selector for selecting between said inter-frame encoder and said intra-frame encoder for differentially encoding pixels Δ

x_i (n,m) of said frame f_i into corresponding differentially encoded pixels Δ

x_i (n,m) of said frame Δ

f_i, said encoding selector being responsive to the relative motion between said video frame f_i and said previous vide frame f_i-1, and

a layered resolution encoder for encoding said differential pixels Δ

x_i (n,m) of frames Δ

f_i into a plurality of separable data sets, each said data set representing video information, within a particular range of video image resolution, about said differential pixels Δ

x_i (n,m),

a packetizer for formatting said plurality of data sets into at least one asynchronous transfer mode (ATM) packet for transmission over the digital communications channel, said ATM packet comprising a header field portion having data for establishing a virtual communications channel between selected devices on the digital communications channel, and an information field portion for transferring said plurality of data sets between said selected devices,

said information field portion having an adaptation overhead field portion including a cell sequence number for indicating the temporal relationship of said ATM packet relative to other said ATM packets, and

a sync flag taking one of a plurality of states for indicating the composition of the remainder of said information field portion, said sync flag taking a first state when said information field portion includes a first type of information field, and a second state when said information field portion includes a second type of information field,

said encoding selector comprises a composite frame combiner for providing a differentially encoded difference frame Δ

f_i having a first set of pixels Δ

x_i (n,m) encoded by said inter-frame encoder, and a second set of pixels Δ

x_i (n,m) encoded by said intra-frame encoder, and

said first type of information field comprises and adaptation overhead field portion further comprising

a strip location field for indicating the location of said second set of pixels relative to said first set of pixels within said encoded difference frame Δ

f_i.

27. The method of claim 33 further comprising the step of separating said DCT coefficients within each said coefficient block into said coefficient layers.