Method and apparatus for transmitting and utilizing analog encoded information

• US 5,802,211 A
• Filed: 08/19/1996
• Issued: 09/01/1998
• Est. Priority Date: 12/30/1994
• Status: Expired due to Term

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: King, Larry
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): Bella, Matthew C.
• Application Number:
  US08/699,411
• Time in Patent Office: 743 Days
• Field of Search: 382/236, 382/232, 348/409, 348/412, 348/413, 348/416, 348/430, 348/699
• US Class Current: 382/236
• CPC Class Codes: G10L 19/04 using predictive techniques
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/543 using regions
• CPC Class Codes: H04N 21/23113 involving housekeeping operations for stored content, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; snloading stored programs G06F9/445; housekeeping operations in file systems, e.g. deletion policies G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/4335 Housekeeping operations, e.g. prioritizing content for deletion because of storage space restrictions storage management, e.g. defragmentation G06F3/0604; unloading stored programs G06F9/445; storage management in file systems G06F16/10; buffering arrangements in a network node or in an end terminal in packet networks H04L49/90
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs

Abstract

Methods and apparatus for reducing the bandwidth required to transmit analog information. The analog information is converted to digital data organized in time defined blocks of information called frames. The content of a frame may be modeled as a composite of superimposed objects, which are subsets of information in the frame, that are largely unchanged frame to frame. Each object in a present frame may be represented by an identification of the object and a description of the changes in object-to-frame relationship that occurred from a previous frame to the present frame. The objects in the present frame may then be reconstructed by adjusting the objects in the previous frame in accordance with the change in object-to-frame relationships.

Claims

1. In a system for communicating plural frames of image information, each frame representing a separate image existing during a respective one of a sequence of time periods, a method of transmitting the image information from a transmitter to a receiver comprising the steps of:

(a) storing information corresponding to plural frames in memory at the transmitter;

(b) storing at least two partial images from one of said frames in memory at the transmitter and at the receiver as objects prior to the transmission of a present frame;

(c) dividing each object into rectangular tiles;

(d) defining motion vectors which describe position and orientation changes to each tile in the objects between a present frame and a former frame;

(e) applying the motion vectors to the respective tiles in the former frame to displace the tiles in accordance with the position and orientation changes;

(f) comparing the luminance and chrominance of the displaced tiles with image regions in the present frame having the same coordinates for a match within a preset variance;

(g) repeating steps (d)-(f) until the motion vector which produces the best match for each tile, within the present variance, is determined;

(h) generating object transformations for each object related to (1) the collective motion vectors which produced the best matches, and (2) depth placement changes of the tiles in the objects;

(i) applying the object transformations to the pels in each object in the former frame to displace the objects accordingly;

(j) comparing the luminance and chrominance of the pels of the displaced objects with the pels in the present frame having the same coordinates for a match within a preset variance, wherein the tiles matched within the preset variance define the object and the corresponding object transformation describes the change between the former frame and the present frame in the relationship of the object to the frames;

(k) designating pels not matched within the preset variance as being in a new object;

(l) repeating steps (h)-(j) for the new object;

(m) storing the defined objects and the corresponding object transformations in memory at the transmitter; and

(n) transmitting the object transformations and corresponding object identifications for the defined objects.

2. The method of claim 1 further comprising the step of:

o) transmitting the information corresponding to pels for which no object transformation was generated to the receiver.

3. The method of claim 2 further comprising the steps of:

p) deleting the objects from the memory at the transmitter and at the receiver that are not identified in some preset number of consecutive present frames; and

q) merging plural objects having the same object transformations into one object.

4. The method of claim 3 wherein the comparing of pel luminance and chrominance for a match in step (j) further comprises the steps of:

(j)(1) determining if the pels having the same coordinates in the two frames have matching luminances within a preset limit;

(j)(2) for those pels which did not have matching luminances, determining if the pels having the same coordinates in the two frames have matching chrominances within a preset limit;

(j)(3) for those pels which did not have matching luminances or chrominances and for which an adjacent pel did have matching luminance or chrominance, repeating steps (j)(1)-(j)(2) after increasing the preset limits, said preset limits having a maximum beyond which no increases are made; and

(j)(4) disregarding groups of adjacent pels having less than a predetermined area which did not have matching luminances or chrominances.

5. The method of claim 2 further comprising the steps of:

p) receiving the information corresponding to the pels for which no object transformation produced a match in step (j), the object transformations and the corresponding object identifications; and

q) combining the received object transformations and object identifications to reconstruct the objects in the present frame;

r) reconstructing the present frame by integrating the reconstructed objects of step (q) with the received information in step (p); and

s) storing the reconstructed objects and received information in memory at the receiver.

6. In a system for communicating plural frames of image information, each frame representing a separate image existing during a respective one of a sequence of time periods, a method of transmitting the image information from a transmitter to a receiver comprising the steps of:

(a) storing information corresponding to plural frames in memory at the transmitter;

(b) storing at least two partial images from one of said frames in memory at the transmitter and at the receiver as objects prior to the transmission of a present frame;

(c) dividing each object into rectangular tiles;

(d) defining motion vectors which describe position and orientation changes to each tile in the objects between a present frame and a former frame;

(e) applying the motion vectors to the respective tiles in the former frame to displace the tiles in accordance with the position and orientation changes;

(f) comparing the luminance and chrominance of the displaced tiles with pels in the present frame having the same coordinates for a match within a preset variance;

(g) repeating steps (d)-(f) until the motion vector which produces the best match for each tile is determined;

(h) generating object transformations for each object related to (1) the collective motion vectors which produced the best matches, and (2) depth placement changes of the tiles in the objects;

(i) applying the object transformations to the pels in each object in the former frame to displace the objects accordingly;

(j) (1) determining if the displaced pels and the pels of the present frame having the same coordinates have matching luminances within a preset limit;

(j)(2) for those pels which did not have matching luminances, determining if the pels have matching chrominances within a preset limit;

(j)(3) for those pels which did not have matching luminances or chrominances and for which an adjacent pel did have matching luminance or chrominance, repeating steps (j)(1)-(j)(2) after increasing the preset limits, said preset limits having a maximum beyond which no increases are made;

(j)(4) disregarding groups of adjacent pels having less than a predetermined area which did not have matching luminances or chrominances, wherein the tiles matched within the preset variance define the object and the corresponding object transformation describes the change between the former frame and the present frame in the relationship of the object to the frames;

k) merging plural objects having the same object transformations into one object;

l) designating tiles not matched within the preset variance as being in a new object;

(m) repeating steps (h)-(j) for the new object;

(n) storing the defined objects and the corresponding object transformations in memory at the transmitter;

o) transmitting the information corresponding to the pels for which no object transformation produced a match within the preset variance of step (j)(4), and transmitting the object transformations and corresponding object identifications for the defined objects to the receiver;

p) receiving the information corresponding to the pels for which no object transformation produced a match within the preset variance of step (j)(4), the object transformations and the corresponding object identifications;

q) combining the received object transformations and object identifications to reconstruct the objects in the present frame;

r) reconstructing the present frame at the receiver by integrating the reconstructed objects of step (q) with the received information of step (p);

s) storing the reconstructed objects and received information in memory at the receiver; and

(t) deleting the objects from the memory at the transmitter and at the receiver which are absent from some preset number of consecutive frames.

7. In a system for communicating plural frames of information, each frame representing the state of the information during a respective one of a sequence of time periods, said information comprising a plurality of image data values, a method of transmitting the information from a transmitter to a receiver comprising the steps of:

(a) identifying within the information at the transmitter one or more objects, each defined by a subset of substantially all the data values in a frame which, (i) have not changed substantially, and (ii) have changed collectively as a subset relative to a former one of the frames;

(b) determining for each object the change between at least two of the frames in the relationship of the object to the frames, by;

(b)(1) dividing an object into rectangular tiles;

(b)(2) assuming a motion vector which describes position changes to the object between two different frames;

(b)(3) applying the motion vector to pels in the object in a former frame to effect the position changes;

(b)(4) comparing the luminance and chrominance of the pels in the two frames having the same coordinates for a match within a preset variance;

(b)(5) repeating steps (a)(2)-(a)(4) until the motion vector which produces the best statistical measure of matched pels is determined;

(b)(6) generating an object transformation related to the collective motion vectors and depth placement changes of the tiles in the object;

(b)(7) applying the object transformation to the object pels in the former frame to displace the pels in accordance with the object transformation; and

(b)(8) comparing the luminance and chrominance of the displaced pels in the former frame with the pels of the present frame having the same coordinates for a match within a preset variance, wherein the matching pels define the object and the object transformation comprises the object change;

(c) transmitting for each object an identification of the object and said determined chance; and

(d) reconstructing at the receiver the objects in one of the frames from the identifications and the determined changes.

8. The method of claim 7 further comprising the step of:

(b)(9) designating the tiles having pels which did not match within the preset variance of step (b)(3) as being in a new object; and

(b)(10) repeating steps (b)(6)-(b)(8) for the new object.

9. The method of claim 7 further comprising the steps of:

(c)(1) transmitting portions of said information corresponding to the pels for which no object transformation is generated.

10. The method of claim 7 further comprising the step of storing in memory at the transmitter and at the receiver the information corresponding to each object.

11. The method of claim 10 further comprising the step of deleting from said memory the information corresponding to objects which are present in fewer than a predetermined number of the frames.

12. The method of claim 9 further comprising the step of merging two or more objects in the same frame which have substantially the same object transformations, into one object.