Image encoding and transmitting system

• US 5,510,839 A
• Filed: 07/06/1992
• Issued: 04/23/1996
• Est. Priority Date: 07/04/1991
• Status: Expired due to Term

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Hamano, Takashi, Sakai, Kiyoshi, Matsuda, Kiichi
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US07/909,044
• Time in Patent Office: 1,387 Days
• Field of Search: 358/105, 358/136, 358/141, 358/167
• US Class Current: 375/240.16
• CPC Class Codes: H04L 2012/5616 Terminal equipment, e.g. codecs, synch.
• CPC Class Codes: H04L 2012/5646 Cell characteristics, e.g. loss, delay, jitter, sequence integrity
• CPC Class Codes: H04L 2012/5651 Priority, marking, classes
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/895 in combination with error concealment
• CPC Class Codes: H04N 21/236 Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream multiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04Q 11/0478 Provisions for broadband connections

Abstract

An image encoding and transmitting system is an image transmission system for transmitting digital data such as TV telephones. The system for dividing input frame data into a plurality of blocks before transmitting them comprises on its sending side an intra-frame/inter-frame determiner for receiving intra-frame data and inter-frame data and selecting either of them for output, and a difference collator for having the intra-frame/inter-frame determiner compulsorily output intra-frame data at the time of data input for the next frame when one or more data in each block forming inter-frame data are larger than a predetermined threshold. The system also comprises on its receiving side an picture element domain frame data storage for storing received data in the picture element domain, and a motion vector detector for detecting according to the contents in the storage a motion vector using the data in the block surrounding the discarded block in order to guess the data in the discarded block.

Claims

1. An image encoding and transmitting system used in an image transmission system for dividing inputted frame image data into blocks comprising a plurality of picture elements, encoding said data in block units, and transmitting said encoded data in packet units, said system comprising:

a motion vector detector on a sending side of said image transmission system for detecting a motion vector for each of said blocks and sending said motion vector to a receiving side,

an inverse discrete cosine transform unit on said receiving side for performing an inverse discrete cosine transform on received data,

an adder for adding the present frame data as an output from said inverse discrete cosine transform unit to data obtained by multiplying the delay result obtained, by delaying data in the preceding frame according to a motion vector received from said sending side by a leak coefficient α

,

a frame memory for storing an output of said adder,

a motion compensation replacement unit (MC) for replacing the data for a block whose encoded data are not received due to packet-discard using a motion vector received from said sending side and the content stored in said frame memory when a packet discard detection signal is inputted,

a variable delay unit for delaying an output from said frame memory according to a motion vector received from said sending side,

a leak coefficient unit for multiplying an output from said variable delay unit by a leak coefficient α and

outputting-the product to said adder.

2. The image encoding and transmitting system according to claim 1, further comprising before said inverse discrete cosine transform unit:

a header remover for removing a header from a received packet,

a separator for separating a packet whose header is removed into a sequence number and encoded data,

a counter to be stepped up when said packet is received,

a comparator for comparing a count value outputted from said counter with a sequence number outputted by said separator, and outputting said detection signal when the discard of a packet is detected,

a memory for storing encoded data outputted by said separator, and

a read controller for outputting the content stored in said memory when said comparator does not detect the discard of a packet, and outputting dummy data "0" as encoded data to said inverse discrete cosine transform unit.

3. The image encoding and transmitting system according to claim 1, wherein

said motion compensation replacement unit comprises:

a 2-block-line delay unit and a 1-block-line delay unit for delaying said packet discard detection signal by 2 block lines and 1 block line,

a B(i,j-8) discard detector for detecting the discard of a block above the target block to be replaced according to an output from said 2-block-line delay unit,

a B(i,j) discard detector for detecting the discard of said target block to be replaced according to an output from said 1-block-line delay unit,

a B(i,j+n) discard detector for detecting the discard of a block below said target block to be replaced according to an input from said packet-discard detection signal,

1-block-line and 2-block-line delay units for delaying a motion vector received from said sending side by 1 block and 2 blocks respectively,

a motion vector detector for detecting a motion vector using the outputs from said three discard detectors and two delay units, and the present value of a motion vector received from said sending side, and

a replacement controller for replacing data using said detected motion vector.

4. The image encoding and transmitting system according to claim 1, wherein

in a data replacement process using said motion vector, determination is made as to whether or not block B(i,j) having its upper left corner point (i,j) is discarded,

if no, the process flow is terminated,

if yes, the variable y indicating the vertical coordinate is set to "0",

the variable x indicating the horizontal coordinate is set to "0",

the horizontal coordinate (i+x) and the vertical coordinate (j+y) in the present frame are replaced with the horizontal coordinate (i+x+x component of the motion vector) and the vertical coordinate (j+y+y component of the motion vector) in the preceding frame,

the value of the variable x indicating the horizontal coordinate is stepped up,

determination is made as to whether or not "x" has reached m,

if "x" has not reached m, the processes are repeated from the process in which said present frame data are replaced with the preceding frame data,

if "x" has reached m, the value of the variable y is stepped up,

determination is made a to whether or not "y" has reached n,

if "y" has not reached n, the processes are repeated from the process in which the variable x indicating said horizontal coordinate is set to "0", and

if "y" has reached n, the process flow is terminated.

5. The image encoding and transmitting system according to claim 1, wherein

an average value of motion vectors of one or more blocks not affected by packet-discard among blocks surrounding the block whose encoded data are not received is used as a motion vector in replacing said block data.

6. The image encoding and transmitting system according to claim 5, wherein

in a moving vector average value calculating process, determination is made as to whether or not block B(i,j) having its upper left corner point (i,j) is discarded,

if no, the process flow is terminated,

if yes, the variable blkct indicating the number of significant blocks for use in detecting a motion vector is set to "0",

determination is made as to whether or not block B(i,j-n) above said discarded block B (i,j) is discarded,

if said block B(i,j-n) is not discarded, the x component of said motion vector for said block B(i,j-n) received from the sending side is set to vector 1x, and the y component of said motion vector is set to vector 1y,

the value of the variable blkct as said number of significant blocks is stepped up, said vector 1x is set as a vector x as the x component of a motion vector to be processed in said average value calculation, and said vector 1y is set as a vector y as the y component of a motion vector to be processed in said average value calculation,

if said block B(i,j-n) is discarded, no actions are taken,

determination is made whether of not block B(i,j+n) below said discarded block B(i,j) is discarded,

if said block B(i,j+n) is not discarded, the x component of the motion vector for said block B(i,j+n) received from the sending side is set to vector 2x, and the y component of the motion vector is set to vector 2y,

a value of the variable blkct indicating said number of significant blocks is stepped up, said vector 2x is added to the x component of the motion vector to be processed in said average value calculation to obtain the x component of a new motion vector, said vector 2y is added to the y component of the motion vector to be processed in said average value calculation to obtain the y component of a new motion vector,

if said block B(i,i+n) is discarded, no actions are taken,

determination is made as to whether or not said variable blkct equals "0",

if said variable blkct is "0", the x and y components of the motion vector are set to "0" and the process flow is terminated, and

if said variable blkct is not "0", the x component of the motion vector to be processed in said average value calculation is divided by the variable blkct, and the result is assigned as the x component of the motion vector, the y component of the motion vector to be processed in said average value calculation is divided by the variable blkct, and the result is assigned as the y component of the motion vector, and the process flow is terminated.