Motion vector quantizing selection system

• US 5,844,612 A
• Filed: 11/09/1995
• Issued: 12/01/1998
• Est. Priority Date: 11/09/1995
• Status: Expired due to Term

• Current Assignee: Utah Valley University
• Original Assignee: Utah Valley University
• Inventors: Israelsen, Paul D.
• Primary Examiner(s): Au, Amelia
• Assistant Examiner(s): Le, Vu
• Application Number:
  US08/555,435
• Time in Patent Office: 1,118 Days
• Field of Search: 348/414, 348/417, 348/418, 348/422, 348/405, 348/699, 348/415, 348/416, 348/419
• US Class Current: 375/240.14
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A video telephone system includes a camera that produces video images. Processing circuitry performs vector quantization on the video images to represent the video images with a relatively small amount of data for transmission to a remote location or memory. The video images are represented by frames which may be further divided into a blocks. A current block is the block that is being considered at a given time. A conditional previous block occupies the position in a previous frame that the current block occupies in the current frame. A motion previous block is a block of the previous frame containing data like that of the current block. To further reduce the amount of data needed to represent the video images, the processing circuitry determines what if any quantization is needed with respect to the current block. The processing circuitry may (1) perform vector quantization on the current block, (2) perform vector quantization on a residual between the current block and motion previous block, or (3) pass bits indicating that the vector quantization of the conditional previous block is adequate. The decision of which occurs depends on various distances related to the current block, the motion previous block, and the conditional previous block. The invention has application for images other than those used in video telephone systems.

Claims

1. An image processing system that processes image data between a source of the image data and a port connectable to a transmission link that is in turn connected to a decoder at a remote location, said image data including a current block which is a block of image data currently under consideration, the current block occupying a position in a current frame of image data, a previous frame which is the frame of image data immediately prior to the current frame, and a conditional previous block which is a block in the previous frame collocated with the current block in the current frame, the image processing system comprising:

source means for generating video image data in frame by frame sequence, said source means including means to generate a current frame and a previous frame;

a memory connectable to said source means to receive said video image data therefrom in said frame by frame sequence, each frame of said frame by frame sequence being comprised of a plurality of blocks, said memory being operable to hold at least a current block of a plurality of blocks of said current frame and at least a portion of the previous frame;

process means for processing said video image data, said process means being connected to said memory to receive said current block and to receive said portion of said previous frame, said process means being configured

to select a conditional previous block of said previous frame,

to select a motion detected block of said current frame relative to said conditional previous block, and

to calculate a motion residual between said motion detected block and said conditional previous block;

a vector quantizer for performing vector quantization of video image data supplied to it and for generating a vector quantized signal reflective of said current frame;

arbiter means connected to said process means to receive therefrom said conditional previous block, said motion residual and said current block and connected to said vector quantizer to supply a selected one of said conditional previous block, said motion residual and said current block, said arbiter means being operable for selecting and supplying to said vector quantizer (1) said current block, (2) said motion residual between said current block and said motion detected block, or (3) a signal that the conditional previous block is acceptable as said current block, the selected of (1), (2) and (3) being supplied for assembly into the current frame; and

a port connected to the vector quantizer for receiving therefrom and for supplying to a remote decoder said vector quantized signal reflective of said current frame.

2. An image processing system that processes image data between a source of the image data and a port connectable to a transmission link for transmission of said image data to a remote location, said image data including a current block which is a block of image data currently under consideration, the current block occupying a position in a current frame of image data, and said image data including a previous frame which is the frame of image data immediately prior to the current frame, said image processing system comprising:

source means for generating video image data in frame by frame sequence, said source means including means to generate a current frame and a previous frame;

a memory connectable to said source means to receive said video image data therefrom in said frame by frame sequence, each frame of said frame by frame sequence being comprised of a plurality of blocks, said memory being operable to hold at least a current block and at least a portion of said previous frame;

means connected to receive said video image data and operable for selecting a motion previous block which best matches the current block from the blocks of said previous frame;

means connected to receive said motion previous block and said current block and operable for determining a motion residual between said current block and said motion previous block;

means connected to receive said motion previous block and said motion residual and operable for determining a motion residual distance measurement based at least in part based on said motion residual;

means connected to receive said video image data and to select a conditional previous block which is a block that occupies a position in said previous frame that corresponds to the position the current block;

means connected to receive said current block and said conditional previous block and operable for determining a conditional update residual between said current block and said conditional previous block;

means connected to receive said conditional update residual, said motion previous block and said conditional previous block, said means being operable for determining a conditional update distance measurement based at least in part on said conditional update residual;

means connected to receive said current block and said motion previous block and operable for determining a current block distance measurement based at least in part on pixel values of said current block;

a vector quantizer for performing vector quantization of video image data supplied to and for generating a vector quantized signal reflective of said current frame;

means connected to receive said video image data and said conditional previous block, said means being operable for determining and providing conditional update indicating bits to indicate that said conditional previous block is acceptable for purposes of decoding at a remote location;

arbiter means connected to receive said current block, said motion residual, said conditional update indicating bits, said arbiter being operable for selecting and supplying to said vector quantizer one of (1) said current block, (2) said motion residual, or (3) said conditional update indicating bits; and

a port connected to the vector quantizer to receive and to supply to a remote decoder said vector quantized signals reflective of said current frame.

3. The system of claim 2 further comprising means for overriding said arbiter and for causing said arbiter to said current block to said vector quantizer.

4. The system of claim 2 wherein said means for selecting a motion previous block, said means for determining a motion residual between the current block and the motion previous block, said means for determining a motion residual distance measurement, said means for determining a conditional update residual, said means for determining a conditional update distance measurement, means for determining a current block distance measurement, said vector quantizer, and said means for providing conditional update indicating bits to the port are each a microprocessor that interacts with said memory.

5. The system of claim 2 wherein at least some of the said means for selecting a motion previous block, said means for determining a motion residual between the current block and the motion previous block, said means for determining a motion residual distance measurement, said means for determining a conditional update residual, said means for determining a conditional update distance measurement, said means for determining a current block distance measurement, said vector quantizer and said means for providing conditional update indicating bits to the port are each a microprocessor that interacts with said memory.

6. The system of claim 2 wherein said memory includes a first memory in which said current block is held and a separate second memory in which the at least a portion of said previous frame is held.

7. The system of claim 2 wherein said motion residual equals Curpix_i -Motprevpix_i, for i=0 to N, where Curpix_i are pixels of said current block, Motprevpix_i are pixels of said motion previous block, and N is a number equal to one less than the number of pixels in said current block.

8. The system of claim 2 wherein said conditional update residual equals Curpix_i -Condprevpix_i, for i=0 to N, where Curpix_i are pixels of said current block, Condprevpix_i are pixels of said conditional previous block, and N is a number equal to one less than the number of pixels in said current block.

9. The system of claim 2 wherein said means for determining a conditional update distance is also responsive to the maximum of absolute values of the conditional update residual.

10. An image processing system that processes image data, said image data including a current block which is a block of image data currently under consideration, said current block occupying a position in a current frame of image data, and said image data including a previous frame which is the frame of image data immediately prior to said current frame, said image processing system comprising:

source means for generating video image data in frame by frame sequence, said source means including means to generate a current frame and a previous frame;

a memory connectable to said source means to receive said video image data therefrom in said frame by frame sequence, each frame of said frame by frame sequence being comprised of a plurality of blocks, said memory being operable to hold at least a current block and at least a portion of said previous frame;

means connected to receive said video image data and operable for selecting a motion previous block which best matches said current block from the blocks of said previous frame;

means connected to receive said motion previous block and said current block and operable for determining a motion residual between said current block and said motion previous block;

means connected to receive said motion previous block and said motion residual and operable for determining a motion residual distance measurement based at least in part based on said motion residual;

means connected to receive said video image data and to select a conditional previous block which is a block that occupies a position in said previous frame that corresponds to the position the current block;

means connected to receive said current block and said conditional previous block and operable for determining a conditional update residual between said current block and said conditional previous block;

means connected to receive said conditional update residual, said motion previous block and said conditional previous block, said means being operable for determining a conditional update distance measurement based at least in part on said conditional update residual;

means connected to receive said current block and said motion previous block and operable for determining a current block distance measurement based at least in part on pixel values of said current block;

a vector quantizer for performing vector quantization of video image data supplied to in and for generating a vector quantized signal reflective of said current frame;

means connected to receive said video image data and said conditional previous block, said means being operable for determining and providing conditional update indicating bits to indicate that said conditional previous block is acceptable for purposes of decoding at a remote location;

arbiter means connected to receive said current block, said motion residual, said conditional update indicating bits, said arbiter being operable for selecting and supplying to said vector quantizer one of (1) said current block, (2) said motion residual, or (3) said conditional update indicating bits;

a port connected to the vector quantizer to receive and to supply to a remote decoder said vector quantized signals reflective of said current frame; and

a remote decoder connected to said port through a transmission link, said remote decoder being operable to select a conditional previous block in assembling at a remote location said current block corresponding to the current block in the memory.