Method and apparatus for coding and decoding image data

• US 20030076881A1
• Filed: 04/24/2002
• Published: 04/24/2003
• Est. Priority Date: 04/24/2001
• Status: Abandoned Application

• Current Assignee: Monolith Company Limited
• Original Assignee: Monolith Company Limited
• Inventors: Shinagawa, Yoshihisa, Akiyoshi, Nobuo, Akiyoshi, Kozo
• Application Number:
  US10/128,342
• Publication Number:
  US 20030076881A1
• Time in Patent Office: Days
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/50 using predictive coding H04N19/61 takes precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/523 with sub-pixel accuracy
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/54 using feature points or meshes
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards

Abstract

An apparatus and method for coding and decoding image data in which image data are input, and the input data are separated into key frames and intermediate frames (which are frames other than the key frames). A pixel by pixel matching is then performed between the key frames to allow generation of both virtual key frames and intermediate frames between the key frames by interpolating the matching results. Actual frames, which may be key frames or intermediate frames, are then coded by determining a difference between virtual frames and actual frames, so that the actual frames can be coded based on the small amount of difference data.

Claims

1. A method of coding image data, comprising:

computing a primary matching between a first key frame and a second key frame included in the image data;

generating a virtual third key frame based on a result of the primary matching;

coding an actual third key frame included in the image data, by utilizing the virtual third key frame; and

computing a secondary matching between adjacent key frames among the first, second and actual third key frames.

2. A method according to claim 1, wherein said computing a primary matching includes computing, pixel by pixel, a matching between the first key frame and the second key frame, and said generating includes generating the virtual third key frame by performing, pixel by pixel, an interpolation computation based on a correspondence relation of position and intensity of pixels between the first and second key frames.

3. A method according to claim 1, further comprising:

outputting, as a coded data stream, the first and second key frames, the coded third key frame and corresponding point data obtained as a result of said secondary matching.

4. A method according to claim 3, wherein the coded third key frame is generated in such a manner that the coded third key frame includes difference data related to a difference between the virtual third key frame and the actual third key frame.

5. A method according to claim 4, wherein the coded third key frame is generated in such a manner that the coded third key frame further includes corresponding point data obtained as a result of said primary matching.

6. A method of coding image data in which image frame data are separated into a key frame and an intermediate frame so as to be coded, the method characterized in that the intermediate frame is coded based on a result of matching between key frames, and at least one of the key frames is also coded based on a result of matching between other key frames.

7. An image data coding apparatus, comprising:

a unit which acquires image data that includes a plurality of frames;

a unit which computes a primary matching between first and second key frames included in the acquired image data;

a unit which generates a virtual third key frame based on a result of the primary matching;

a unit which codes an actual third key frame by utilizing the virtual third key frame; and

a unit which computes a secondary matching between adjacent key frames among the first, second and actual third key frames.

8. An image data coding apparatus according to claim 7, wherein the first, second and third key frames are arranged in this temporal order, and said generating unit generates the virtual third key frame by extrapolation.

9. An image data coding apparatus according to claim 7, wherein the first, third and second key frames are arranged in this temporal order, and said generating unit generates the virtual third key frame by interpolation.

10. An image data coding apparatus according to claim 7, wherein said coding unit codes a difference between the virtual third key frame and the actual third key frame.

11. An image data coding apparatus according to claim 7, wherein said secondary-matching computing unit computes a pixel-by-pixel matching between the adjacent key frames.

12. An image data coding apparatus according to claim 7, wherein said generating unit computes a pixel-by-pixel matching between the first and second key frames, and generates the virtual third key frame by performing an interpolation computation based on a result thereof.

13. An image data coding apparatus according to claim 7, further comprising:

a unit which outputs the first and second key frames, the coded third key frame and data obtained as a result of the secondary matching as a coded data stream.

14. An image data coding apparatus according to claim 7, wherein the coded third key frame is generated in such a manner that the coded third key frame includes difference data related to a difference between the virtual third key frame and the actual third key frame.

15. An image data coding apparatus according to claim 7, wherein the coded third key frame is generated in such a manner that the coded third key frame further includes corresponding point data obtained as a result of the primary matching.

16. An image data coding apparatus according to claim 13, wherein the coded data stream stores a result of the secondary matching as corresponding point data.

17. An image data coding apparatus according to claim 7, wherein said coding unit further codes an actual intermediate frame by utilizing a virtual intermediate frame generated based on a result of the secondary matching.

18. An image data coding apparatus according to claim 17, wherein said coding unit codes a difference between the virtual intermediate frame and the actual intermediate frame.

19. A computer program executable by a computer, the program comprising the functions of:

computing a primary matching between a first key frame and a second key frame included in the image data;

generating a virtual third key frame based on a result of the primary matching;

coding an actual third key frame included in the image data, by utilizing the virtual third key frame; and

computing a secondary matching between adjacent key frames among the first, second and actual third key frames.

20. A computer program, executable by a computer, for coding image data in which image frame data are separated into a key frame and an intermediate frame so as to be coded, the program including the functions of:

coding the intermediate frame based on a result of matching between key frames, and also coding at least one of the key frames based on a result of matching between other key frames.

21. An image decoding method, comprising:

acquiring a coded data stream which includes data of first and second key frames and data of a third key frame coded based on a result of a matching between the first and second key frames;

decoding the third key frame from the acquired coded data stream; and

computing a matching between adjacent key frames among the first, second and third key frames, and thereby generating an intermediate frame.

23. A method according to claim 21, wherein the coded third key frame data includes coded data of a difference between a virtual third key frame generated based on a matching computed between the first and second key frames and an actual third key frame.

24. A method according to claim 23, wherein, in said decoding, after the virtual third key frame is generated by computing the matching between the first and second key frames, the actual third key frame is decoded based on the thus generated virtual third key frame.

25. A method according to claim 21, wherein the coded third key frame data includes corresponding point data which is a result of the matching computed between the first and second key frames and coded data of a difference between a virtual third key frame to be generated based on the corresponding point data and an actual third key frame.

26. A method according to claim 25, wherein, in said decoding, after the virtual third key frame is generated based on the corresponding point data, the actual third key frame is decoded based on the thus generated virtual third key frame.

22. An image decoding method, comprising:

acquiring a coded data stream which includes data of first and second key frames, data of a third key frame coded based on a result of a matching therebetween, and corresponding point data obtained as a result of computation of a matching between adjacent key frames among the first, second and third key frames;

decoding the third key frame from the acquired coded data stream; and

generating an intermediate frame based on the corresponding point data.

27. An image decoding apparatus, comprising:

a unit which acquires a coded data stream that includes data of first and second key frames and data of a third key frame coded based on a result of a matching between the first and second key frames;

a unit which decodes the third key frame from the acquired coded data stream; and

a unit which computes a matching between adjacent key frames among the first, second and third key frames, and thereby generates an intermediate frame.

28. An image decoding apparatus, comprising:

a unit which acquires a coded data stream that includes data of first and second key frames, data of a third key frame coded based on a result of a matching therebetween, and corresponding point data obtained as a result of computation of a matching between adjacent key frames among the first, second and third key frames;

a unit which decodes the third key frame from the acquired coded data stream; and

a unit which generates an intermediate frame based on the corresponding point data.

29. An image decoding apparatus according to claim 28, wherein the coded third key frame data includes coded data of a difference between a virtual third key frame generated based on a matching computed between the first and second key frames, and an actual third key frame.

30. An image decoding apparatus according to claim 29, wherein after the virtual third key frame is generated by computing the matching between the first and second key frames, said decoding unit decodes the actual third key frame based on the virtual third key frame.

31. An image decoding apparatus according to claim 28, wherein the coded third key frame data includes corresponding point data which is a result of the matching computed between the first and second key frames and coded data of a difference between a virtual third key frame to be generated based on the corresponding point data and an actual third key frame.

32. An image decoding apparatus according to claim 31, wherein after the virtual third key frame is generated based on the corresponding point data, said decoding unit decodes the actual third key frame based on the virtual third key frame.

33. A method of coding image data, comprising:

separating frames included in the image data into key frames and intermediate frames;

generating a series of source hierarchical images of different resolutions by operating a multiresolutional critical point filter on a first key frame obtained by said separating;

generating a series of destination hierarchical images of different resolutions by operating the multiresolutional critical point filter on a second key frame obtained by said separating;

computing a matching of the source hierarchical images and the destination hierarchical images in a resolutional level hierarchy;

generating a virtual third key frame based on a result of the matching; and

coding an actual third key frame included in the image data, by utilizing the virtual third key frame.

34. An image data coding apparatus, comprising:

a first functional block which acquires a virtual key frame generated based on a result of a matching performed between key frames included in image data; and

a second functional block which codes an actual key frame included in the image data, by utilizing the virtual key frame.

35. An image data coding apparatus according to claim 34, further comprising:

a third functional block which computes a matching between adjacent key frames including the actual key frame and which codes an intermediate frame that is other than the key frames.

36. An image decoding method, comprising:

acquiring, from a coded data stream of image data, first and second key frames and a third key frame which is coded based on a result of a processing performed between the first and second key frames and which is different from the first and second key frames;

decoding the thus acquired coded third key frame; and

generating an intermediate frame, which is not a key frame, by performing a processing between a plurality of key frames including the third key frame obtained as a result of said decoding.

37. An image decoding apparatus, comprising:

a first functional block which acquires, from a coded data stream of image data, first and second key frames and a third key frame which is coded based on a result of a processing performed between the first and second key frames and which is different from the first and second key frames;

a second functional block which decodes the thus acquired coded third key frame; and

a third functional block which generates an intermediate frame, which is not a key frame, by performing a processing between a plurality of key frames including the third key frame obtained in said second functional block.

38. A computer program executable by a computer, the program comprising the functions of:

acquiring a coded data stream that includes data of first and second key frames and data of a third key frame coded based on a result of a matching between the first and second key frames;

decoding the third key frame from the acquired coded data stream; and

computing a matching between adjacent key frames among the first, second and third key frames, and thereby generating an intermediate frame.

39. A computer program executable by a computer, the program comprising the functions of:

acquiring a coded data stream that includes data of first and second key frames, data of a third key frame coded based on a result of a matching therebetween, and corresponding point data obtained as a result of computation of a matching between adjacent key frames among the first, second and third key frames;

decoding the third key frame from the acquired coded data stream; and

generating an intermediate frame based on the corresponding point data.

40. A method of coding image data, comprising:

computing a matching between first and second key frames included in the image data;

generating a virtual second key frame based on a result of the matching and the first key frame; and

coding an actual second key frame by utilizing the virtual second key frame.

41. A method according to claim 40, wherein said coding includes compressing a difference between the actual second key frame and the virtual second key frame.

42. A method according to claim 40, further comprising:

incorporating the first key frame, coded second key frame and corresponding point data obtained as a result of the matching into a coded data stream, so as to be output.

43. An image data coding apparatus, comprising:

a unit which acquires image data including a plurality of frames;

a matching unit which computes a matching between first and second key frames included in the acquired image data;

a generating unit which generates a virtual second key frame based on a result of the matching and the first key frame; and

a coding unit which codes an actual second key frame by utilizing the virtual second key frame.

44. A computer program executable by a computer, the program comprising the functions of:

computing a matching between first and second key frames included in image data;

generating a virtual second key frame based on a result of the matching and the first key frame; and

coding an actual second key frame by utilizing the virtual second key frame.

45. An image decoding method, comprising:

acquiring a coded data stream that includes data of a first key frame and a second key frame which is coded based on a result of a matching between the first and second key frames;

decoding the second key frame from the acquired coded data stream; and

generating an intermediate frame between the first key frame and the second key frame by utilizing the first key frame, decoded second key frame and a result of the matching therebetween.

46. An image decoding apparatus, comprising:

a unit which acquires a coded data stream that includes data of a first key frame and a second key frame which is coded based on a result of a matching between the first and second key frames;

a unit which decodes the second key frame from the coded data stream acquired by said acquiring unit; and

a unit which generates an intermediate frame between the first key frame and the second key frame by utilizing the first key frame, decoded second key frame and a result of the matching therebetween.

47. A computer program executable by a computer, the program comprising the functions of:

acquiring a coded data stream that includes data of a first key frame and a second key frame which is coded based on a result of a matching between the first and second key frames;

decoding the second key frame from the acquired coded data stream; and

generating an intermediate frame between the first key frame and the second key frame by utilizing the first key frame, decoded second key frame and a result of the matching therebetween.

48. A coded image data structure, comprising:

an index region which identifies image data;

a reference data region which includes data used in a decoding processing;

an independent frame data region which includes data relating to independent frames which are decoded independent of other frames; and

a coded frame region which includes data related to dependent frames which are decoded depending on other frames,

wherein said regions are integrated to form the coded image data.

49. A coded image data structure according to claim 48, wherein said coded frame region includes coded data of a difference between an actual dependent frame and a virtual dependent frame determined based on data related to an independent frame.