Motion vector detecting method, record medium on which motion vector calculating program has been recorded, motion detecting apparatus, motion detecting method, picture encoding apparatus, picture encoding method, motion vector calculating method, record medium on which motion vector calculating program has been recorded

• US 20020015513A1
• Filed: 07/23/2001
• Published: 02/07/2002
• Est. Priority Date: 07/15/1998
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ando, Yuji, Yasuda, Hiroyuki
• Application Number:
  US09/910,065
• Publication Number:
  US 20020015513A1
• Time in Patent Office: Days
• US Class Current: 382/107
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A motion vector calculating method is disclosed which includes the steps of: (a) extracting a reference block from a reference picture corresponding to a current block of a current picture to be processed, the size and origin of the reference block matching those of the current block; (b) while moving the reference block in a predetermined search area, obtaining a residual between the current block; (c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector; (d) orthogonally transforming pixel data of a reference block and pixel data of a current block of the current picture and (e) obtaining a residual between orthogonally transformed data of the reference block and orthogonally transformed data of each block of the current picture. In some embodiments, the motion vector calculation stops when a residual is larger than a predetermined value, which may be based on a characteristic of a picture. A motion vector for an entire picture may be calculated based on a motion vector detected in a plurality of macro blocks, or vice versa. The orthogonal transformation may be skipped if the residual is smaller than a predetermined value. Various methods of increasing the speed of calculation by using fewer than all pixels in a macro block are provided, such as using only those pixels on the circumference of a macro block. Media with computer programs according to the foregoing methods and apparatuses for performing the foregoing methods are provided.

Claims

1. A motion vector calculating method, comprising the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector;

(d) orthogonally transforming pixel data of a block of the reference picture and pixel data of a block of the current picture; and

(e) obtaining a residual between orthogonally transformed data of the block of the reference picture and orthogonally transformed data of each block of the current picture.

2. The motion vector calculating method as set forth in claim 1,

wherein step (d) is performed by Hadamard transforming method.

3. The motion vector calculating method as set forth in claim 1, further comprising the steps of:

dividing each of a block of the reference picture and a block of the current picture into a plurality of blocks; and

orthogonally transforming each of the divided blocks.

4. A record medium on which a motion vector calculating program has been recorded, the motion vector calculating program causing a system that has the record medium to perform the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector;

(d) orthogonally transforming pixel data of a block of the reference picture and pixel data of a block of the current picture; and

(e) obtaining a residual between orthogonally transformed data of the block of the reference picture and orthogonally transformed data of each block of the current picture.

5. The record medium as set forth in claim 4, wherein step (d) is performed by Hadamard transforming method.

6. The record medium as set forth in claim 4, further comprising the steps of:

dividing each of a block of the reference picture and a block of the current picture into a plurality of blocks; and

orthogonally transforming each of the divided blocks.

7. A motion vector calculating method, comprising the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector;

(d) while calculating a residual between pixels of a block of the reference picture and pixels of a block of the current picture, comparing the obtained residual with a predetermined threshold value; and

(e) when the residual is larger than the predetermined threshold value, stopping the calculation of the motion vector; and

(f) setting the initial value of the predetermined threshold value corresponding to a characteristic of a picture.

8. The motion vector calculating method as set forth in claim 7,

wherein the initial value of the predetermined threshold value is set corresponding to the sum of the absolute values of the difference values between values of pixels of the same picture and the mean value of the pixels thereof.

9. The motion vector calculating method as set forth in claim 7,

wherein the initial value of the predetermined threshold value is set corresponding to a residual at the origin.

10. The motion vector calculating method as set forth in claim 7,

wherein the initial value of the predetermined threshold value is set corresponding to the sum of the absolute values of the difference values between values of pixels of the same picture and the mean value of the pixels thereof and a residual at the origin.

11. The motion vector calculating method as set forth in claim 7,

wherein the predetermined threshold value is the minimum value of residuals that have been obtained so far.

13. The record medium as set forth in claim 12,

wherein the initial value of the predetermined threshold value is set corresponding to the sum of the absolute values of the difference values between values of pixels of the same picture and the mean value of the pixels thereof.

14. The record medium as set forth in claim 12,

wherein the initial value of the predetermined threshold value is set corresponding to a residual at the origin.

15. The record medium as set forth in claim 12,

wherein the initial value of the predetermined threshold value is set corresponding to the sum of the absolute values of the difference values between values of pixels of the same picture and the mean value of the pixels thereof and a residual at the origin.

16. The record medium as set forth in claim 12,

wherein the predetermined threshold value is the minimum value of residuals that have been obtained so far.

18. The motion detecting apparatus as set forth in claim 17,

wherein said extracting means extracts a plurality of adjacent macro blocks.

19. The motion detecting apparatus as set forth in claim 17,

wherein said extracting means extracts a plurality of macro blocks from each of areas into which the entire picture is divided.

12. A record medium on which a motion vector calculating program has been recorded, the motion vector calculating program causing a system that has the record medium to perform the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector;

(d) while calculating a residual between pixels of a block of the reference picture and pixels of a block of the current picture, comparing the obtained residual with a predetermined threshold value; and

(e) when the residual is larger than the predetermined threshold value, stopping the calculation of the motion vector; and

(f) setting the initial value of the predetermined threshold value corresponding to a characteristic of a picture.

17. A motion detecting apparatus, comprising:

extracting means for extracting a plurality of macro blocks from a picture;

first motion detecting means for detecting a motion vector of each of the plurality of macro blocks extracted by said extracting means;

motion calculating means for calculating a motion vector of the entire picture with motion vectors of individual macro blocks detected by said first motion detecting means; and

second motion detecting means for calculating a motion vector of each macro block with the motion vector calculated by said motion calculating means.

20. A motion detecting method, comprising the steps of:

(a) extracting a plurality of macro blocks from a picture;

(b) detecting a motion vector of each of the plurality of macro blocks that have been extracted;

(c) calculating a motion vector of the entire picture with motion vectors of individual macro blocks that have been detected; and

(d) calculating a motion vector of each macro block with the motion vector that have been calculated.

21. The motion detecting method as set forth in claim 20,

wherein step (a) is performed by extracting a plurality of adjacent macro blocks.

22. The motion detecting method as set forth in claim 20,

wherein step (a) is performed by extracting a plurality of macro blocks from each of areas into which the entire picture is divided.

24. The picture encoding apparatus as set forth in claim 23,

wherein said motion detecting means calculates a mean discrete residual of each pixel block, and

wherein said determining means compares the mean discrete residual with the predetermined value.

25. The picture encoding apparatus as set forth in claim 23, further comprising:

value determining means for determining the predetermined value with information obtained in the encoding process,

wherein said determining means generates the determined result with the predetermined value determined by said value determining means.

27. The picture encoding method as set forth in claim 26,

wherein step (a) is performed by calculating a mean discrete residual of each pixel block, and

wherein step (b) is performed by comparing the mean discrete residual with the predetermined value.

28. The picture encoding method as set forth in claim 26, further comprising the step of:

determining the predetermined value with information obtained in the encoding process,

wherein step (b) is performed by generating the determined result with the predetermined value that has been determined.

30. The vector calculating method as set forth in claim 29,

wherein the residual is calculated with an instruction that causes a plurality of successive data pieces to be processed at a time.

31. The vector calculating method as set forth in claim 29,

wherein N pixels of the current picture and N pixels of the reference picture are extracted checkerwise at a time.

23. A picture encoding apparatus, comprising:

motion detecting means for detecting a motion vector of a predetermined pixel block of input picture data and generating motion residual information;

determining means for comparing the motion residual information received from said motion detecting means with a predetermined value and generating a determined result;

picture data process means for performing a predetermined process for picture data, the predetermined process being required for an encoding process;

encoding means for performing the encoding process for picture data; and

controlling means for skipping the predetermined process performed by said picture data process means corresponding to the determined result of said determining means and causing said encoding means to perform the encoding process.

26. A picture encoding method, comprising the steps of:

(a) detecting a motion vector of a predetermined pixel block of input picture data and generating motion residual information;

(b) comparing the motion residual information with a predetermined value and generating a determined result;

(c) performing a predetermined process for picture data, the predetermined process being required for an encoding process; and

(d) skipping the predetermined process corresponding to the determined result and performing the encoding process for the picture data.

29. A motion vector calculating method, comprising the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector;

(d) extracting N pixels of the current picture and N pixels of the reference picture at a time (where N is an integer);

(e) storing the N pixels of the current picture and the N pixels of the reference picture as successive data to a memory; and

(f) reading pixels of the block of the current picture and pixels of the block of the reference picture as successive data from the memory so as to obtain a residual.

32. A record medium on which a motion vector calculating program has been recorded, the motion vector calculating program causing a system that has the record medium to perform the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector;

(d) extracting N pixels of the current picture and N pixels of the reference picture at a time (where N is an integer);

(e) storing the N pixels of the current picture and the N pixels of the reference picture as successive data to a memory; and

(f) reading pixels of the block of the current picture and pixels of the block of the reference picture as successive data from the memory so as to obtain a residual.

33. The record medium as set forth in claim 32, wherein the residual is calculated with an instruction that causes a plurality of successive data pieces to be processed at a time.

34. The record medium as set forth in claim 32,

wherein N pixels of the current picture and N pixels of the reference picture are extracted checkerwise at a time.

36. The motion vector calculating method as set forth in claim 35,

wherein the residual is calculated with an instruction that causes a plurality of successive data pieces to be processed at a time.

37. The motion vector calculating method as set forth in claim 35,

wherein N pixels of the current picture and N pixels of the reference picture are extracted checkerwise at a time.

39. The record medium as set forth in claim 38,

wherein the residual is calculated with an instruction that causes a plurality of successive data pieces to be processed at a time.

40. The record medium as set forth in claim 38,

wherein N pixels of the current picture and N pixels of the reference picture are extracted checkerwise at a time.

35. A motion vector calculating method, comprising the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a coarse motion vector;

(d) while moving the block of the reference picture in the vicinity of the coarse motion vector obtained at step (c), obtaining a residual between the block of the current picture and the block of the reference picture;

(e) detecting a block with the minimum residual from the reference picture so as to detect a fine motion vector;

(f) storing pixels of the current picture and pixels of the reference picture to a first memory;

(g) extracting N pixels of the current picture and N pixels of the reference picture at a time (where N is an integer); and

(h) storing the N pixels of the current picture and the N pixels of the reference picture as successive data to a second memory,

wherein step (c) is performed with the N pixels of the current picture and the N pixels of the reference picture stored as successive data in the second memory, and

wherein step (e) is performed with the pixels of the current picture and the pixels of the reference picture stored in the first memory.

38. A record medium on which a motion vector calculating program has been recorded, the motion vector calculating program causing a system that has the record medium to perform the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a coarse motion vector;

(d) while moving the block of the reference picture in the vicinity of the coarse motion vector obtained at step (c), obtaining a residual between the block of the current picture and the block of the reference picture;

(e) detecting a block with the minimum residual from the reference picture so as to detect a fine motion vector;

(f) storing pixels of the current picture and pixels of the reference picture to a first memory;

(g) extracting N pixels of the current picture and N pixels of the reference picture at a time (where N is an integer); and

(h) storing the N pixels of the current picture and the N pixels of the reference picture as successive data to a second memory,

wherein step (c) is performed with the N pixels of the current picture and the N pixels of the reference picture stored as successive data in the second memory, and

wherein step (e) is performed with the pixels of the current picture and the pixels of the reference picture stored in the first memory.

41. A motion vector calculating method, comprising the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector; and

(d) comparing contour pixels of the block of the reference picture with contour pixels of the block of the current picture so as to obtain a residual therebetween.

42. The motion vector calculating method as set forth in claim 41,

wherein step (d) includes the steps of:

cumulating the absolute values of the difference values between the upper contour pixels of the block of the current picture and the upper contour pixels of the block of the reference picture in a horizontal scanning direction so as to obtain the sum thereof;

cumulating the absolute values of the difference values between the left contour pixels of the block of the current picture and the left contour pixels of the block of the reference picture in a vertical scanning direction so as to obtain the sum thereof;

cumulating the absolute values of the difference values between the right contour pixels of the block of the current picture and the right contour pixels of the block of the reference picture in a vertical scanning direction so as to obtain the sum thereof; and

cumulating the absolute values of the difference values between the lower contour pixels of the block of the current picture and the lower contour pixels of the block of the reference picture in a horizontal scanning direction so as to obtain the sum thereof.

43. A record medium on which a motion vector calculating program has been recorded, the motion vector calculating program causing a system that has the record medium to perform the steps of:

(a) extracting a block from a reference picture corresponding to a block of a current picture to be processed, the size of the block of the reference picture being the same as the size of the block of the current picture, the origin of the block of the reference picture matching the origin of the block of the current picture;

(b) while moving the block of the reference picture in a predetermined search area, obtaining a residual between the block of the current picture and the block of the reference picture;

(c) detecting a block with the minimum residual from the reference picture so as to calculate a motion vector; and

(d) comparing contour pixels of the block of the reference picture with contour pixels of the block of the current picture so as to obtain a residual therebetween.

44. The record medium as set forth in claim 43 wherein step (d) includes the steps of:

cumulating the absolute values of the difference values between the upper contour pixels of the block of the current picture and the upper contour pixels of the block of the reference picture in a horizontal scanning direction so as to obtain the sum thereof;

cumulating the absolute values of the difference values between the left contour pixels of the block of the current picture and the left contour pixels of the block of the reference picture in a vertical scanning direction so as to obtain the sum thereof;

cumulating the absolute values of the difference values between the right contour pixels of the block of the current picture and the right contour pixels of the block of the reference picture in a vertical scanning direction so as to obtain the sum thereof; and

cumulating the absolute values of the difference values between the lower contour pixels of the block of the current picture and the lower contour pixels of the block of the reference picture in a horizontal scanning direction so as to obtain the sum thereof.