Motion vector detection apparatus, motion vector processing method and program

• US 8,619,863 B2
• Filed: 10/13/2009
• Issued: 12/31/2013
• Est. Priority Date: 11/05/2008
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yokoyama, Masayuki
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Xu, Xiaolan
• Application Number:
  US12/588,329
• Publication Number:
  US 20100110302A1
• Time in Patent Office: 1,540 Days
• Field of Search: None
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/53 Multi-resolution motion estimation; Hierarchical motion estimation
• CPC Class Codes: H04N 19/567 Motion estimation based on rate distortion criteria
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Disclosed herein is a motion vector detection apparatus, including: a spatial luminance difference sum total calculation section; a temporal luminance difference sum total calculation section; a search range calculation section; and a block matching calculation section.

Claims

1. A motion vector detection apparatus, comprising:

a spatial luminance difference sum total calculation section that calculates a sum total of absolute values of spatial luminance differences, which are luminance differences of pixels of a base block set in a base frame with respect to a predetermined pixel;

a temporal luminance difference sum total calculation section that calculates a sum total of absolute values of temporal luminance differences, which are luminance differences between corresponding pixels of the base block, and a corresponding block in a reference frame corresponding to the base block;

a search range calculation section that calculates a search range of the reference frame for searching for a motion vector with respect to the base block based on the sum total of the absolute values of the spatial luminance differences and the sum total of the absolute values of the temporal luminance differences; and

a block matching calculation section that sets a reference block in the search range of the reference frame determined depending upon the search range and performs block matching based on the correlation degree between the set reference block and the base block to calculate the motion vector,

wherein said spatial luminance difference sum total calculation section calculates, as the sum total of the absolute values of the spatial luminance differences, sum totals of absolute values of the spatial luminance differences in both of a horizontal direction and a vertical direction with respect to the base block, and

said search range calculation section calculates search ranges in the horizontal and vertical directions based on the sum total of the absolute values of the temporal luminance differences and the sum totals of the absolute values of the spatial luminance differences in the horizontal and vertical directions.

2. The motion vector detection apparatus according to claim 1, wherein said search range calculation section calculates the search range such that, as the value of the sum total of the absolute values of the spatial luminance differences increases, the size of the search range decreases, but as value of the sum total of the absolute values of the temporal luminance differences increases, the size of the search range increases.

3. The motion vector detection apparatus according to claim 2, wherein said search range calculation section calculates the search range such that the size of the search range increases in inverse proportion to the sum total of the absolute values of the spatial luminance differences but increases in proportion to the sum total of the absolute values of the temporal luminance differences.

4. The motion vector detection apparatus according to claim 1, wherein said spatial luminance difference sum total calculation section adds the absolute values of the spatial luminance differences in the horizontal and vertical directions to a positive side spatial luminance difference sum total or totals or a negative side spatial luminance difference sum total or totals based on the sign of the temporal luminance difference with respect to the predetermined pixel in the base block and the signs of the spatial luminance differences in the horizontal and vertical directions.

5. The motion vector detection apparatus according to claim 4, wherein said temporal luminance difference sum total calculation section adds the absolute values of the temporal luminance differences in the horizontal and vertical directions to the positive side temporal luminance difference sum total or totals or the negative side temporal luminance difference sum total or totals based on the sign of the temporal luminance difference with respect to the predetermined pixel in the base block and the signs of the spatial luminance differences in the horizontal and vertical directions.

6. The motion vector detection apparatus according to claim 5, wherein said search range calculation section includes

a positive side search range calculation section configured to calculate the search ranges on the positive side in the horizontal and vertical directions based on the positive side spatial luminance difference sum totals in the horizontal and vertical directions and the positive side temporal luminance difference sum totals in the horizontal and vertical directions, and

a negative side search range calculation section configured to calculate the search ranges on the negative side in the horizontal and vertical directions based on the negative side spatial luminance difference sum totals in the horizontal and vertical directions and the negative side temporal luminance difference sum totals in the horizontal and vertical directions.

7. The motion vector detection apparatus according to claim 4, further comprising a likelihood calculation section configured to calculate a positive side likelihood and a negative side likelihood based on the positive side spatial luminance difference sum totals and the negative side spatial luminance difference sum totals, respectively, in the horizontal and vertical directions.

8. The motion vector detection apparatus according to claim 7, wherein said search range calculation section includes

a positive side search range calculation section configured to calculate the search ranges on the positive side in the horizontal and vertical directions based on the sum totals of the absolute values of the spatial luminance differences, the sum totals of the absolute values of the temporal luminance differences and the positive side likelihoods in the horizontal and vertical directions, and

a negative side search range calculation section configured to calculate the search ranges on the negative side in the horizontal and vertical directions based on the sum totals of the absolute values of the spatial luminance differences, the sum totals of the absolute values of the temporal luminance differences and the negative side likelihoods in the horizontal and vertical directions.

9. The motion vector detection apparatus according to claim 1, further comprising:

a base luminance value production section configured to produce a base luminance value of the base block based on the luminance value of the predetermined pixel in the base block; and

an effectiveness calculation section configured to calculate an effectiveness degree of each of the pixels of the base block based on the luminance values of the pixels of the base block and the base luminance value;

said block matching calculation section including a correlation calculation section configured to set a plurality of reference blocks in a search region in the reference frame which is determined depending upon the search range and calculate, for each of the reference blocks, the correlation degree of the reference block with respect to the base block based on the luminance values of the pixels in the set reference block which correspond to the pixel positions of the base block and the effective degrees corresponding to the pixel positions of the base block.

10. The motion vector detection apparatus according to claim 1, wherein said search range calculation section determines one of the search range calculated based on the sum total of the absolute values of the spatial luminance differences and the sum total of the absolute values of the temporal luminance differences and a predetermined range as the search range.

11. A motion vector detection method, comprising the steps of:

calculating a sum total of absolute values of spatial luminance differences which are luminance differences of pixels of a base block set in a base frame with respect to a predetermined pixel;

calculating a sum total of absolute values of temporal luminance differences which are luminance differences between corresponding pixels of the base block and a corresponding block in a reference frame corresponding to the base block;

calculating a search range of the reference frame for searching for a motion vector with respect to the base block based on the sum total of the absolute values of the spatial luminance differences and the sum total of the absolute values of the temporal luminance differences; and

setting a reference block in the search range of the reference frame determined depending upon the search range and carrying out block matching based on the correlation degree between the set reference block and the base block to calculate the motion vector,

wherein calculating the sum total of the absolute values of the spatial luminance differences includes calculating, as the sum total of the absolute values of the spatial luminance differences, sum totals of absolute values of the spatial luminance differences in both of a horizontal direction and a vertical direction with respect to the base block, and

calculating the search range of the reference frame includes calculating search ranges in the horizontal and vertical directions based on the sum total of the absolute values of the temporal luminance differences and the sum totals of the absolute values of the spatial luminance differences in the horizontal and vertical directions.

14. The method according to claim 11, further includes adding the absolute values of the spatial luminance differences in the horizontal and vertical directions to a positive side spatial luminance difference sum total or totals or a negative side spatial luminance difference sum total or totals based on the sign of the temporal luminance difference with respect to the predetermined pixel in the base block and the signs of the spatial luminance differences in the horizontal and vertical directions.

12. A non-transitory computer readable medium storing program code, the program code being executable by a processor to perform operations comprising:

calculating a sum total of absolute values of spatial luminance differences which are luminance differences of pixels of a base block set in a base frame with respect to a predetermined pixel;

calculating a sum total of absolute values of temporal luminance differences which are luminance differences between corresponding pixels of the base block and a corresponding block in a reference frame corresponding to the base block;

calculating a search range of the reference frame for searching for a motion vector with respect to the base block based on the sum total of the absolute values of the spatial luminance differences and the sum total of the absolute values of the temporal luminance differences; and

setting a reference block in the search range of the reference frame determined depending upon the search range and carrying out block matching based on the correlation degree between the set reference block and the base block to calculate the motion vector,

wherein calculating the sum total of the absolute values of the spatial luminance differences includes calculating. as the sum total of the absolute values of the spatial luminance differences, sum totals of absolute values of the spatial luminance differences in both of a horizontal direction and a vertical direction with respect to the base block, and

calculating the search range of the reference frame includes calculating search ranges in the horizontal and vertical directions based on the sum total of the absolute values of the temporal luminance differences and the sum totals of the absolute values of the spatial luminance differences in the horizontal and vertical directions.

15. The non-transitory computer readable medium according to claim 12, wherein the operations further comprise adding the absolute values of the spatial luminance differences in the horizontal and vertical directions to a positive side spatial luminance difference sum total or totals or a negative side spatial luminance difference sum total or totals based on the sign of the temporal luminance difference with respect to the predetermined pixel in the base block and the signs of the spatial luminance differences in the horizontal and vertical directions.

13. A motion vector detection apparatus, comprising:

spatial luminance difference sum total calculation means for calculating a sum total of absolute values of spatial luminance differences which are luminance differences of pixels of a base block set in a base frame with respect to a predetermined pixel;

temporal luminance difference sum total calculation means for calculating a sum total of absolute values of temporal luminance differences which are luminance differences between corresponding pixels of the base block and a corresponding block in a reference frame corresponding to the base block;

search range calculation means for calculating a search range of the reference frame for searching for a motion vector with respect to the base block based on the sum total of the absolute values of the spatial luminance differences and the sum total of the absolute values of the temporal luminance differences; and

block matching calculation means for setting a reference block in the search range of the reference frame determined depending upon the search range and carrying out block matching based on the correlation degree between the set reference block and the base block to calculate the motion vector,

wherein said spatial luminance difference sum total calculation means calculates, as the sum total of the absolute values of the spatial luminance differences, sum totals of absolute values of the spatial luminance differences in both of a horizontal direction and a vertical direction with respect to the base block, and

said search range calculation means calculates search ranges in the horizontal and vertical directions based on the sum total of the absolute vlues of the temporal luminance differences and the sum totals of the absolute values of the spatial luminance differences in the horizontal and vertical directions.

16. The apparatus according to claim 13, wherein said spatial luminance difference sum total calculation means adds the absolute values of the spatial luminance differences in the horizontal and vertical directions to a positive side spatial luminance difference sum total or totals or a negative side spatial luminance difference sum total or totals based on the sign of the temporal luminance difference with respect to the predetermined pixel in the base block and the signs of the spatial luminance differences in the horizontal and vertical directions.