Image processing device, image processing method, and storage medium

• US 6,956,960 B2
• Filed: 02/07/2003
• Issued: 10/18/2005
• Est. Priority Date: 03/29/1999
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Tange, Akira
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US10/360,419
• Publication Number:
  US 20030118105A1
• Time in Patent Office: 984 Days
• Field of Search: 392/103, 392/104, 392/107, 392192-194, 734/88, 348/154, 348/155, 356/27
• US Class Current: 382/107
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 19/537 Motion estimation other than block-based
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20

Abstract

An image processing device for calculating the motion direction of each pixel in a predetermined screen of interest from among a plurality of time series screens. The image processing device includes a score calculator for calculating a score corresponding to each of a plurality of motion directions of each pixel in the predetermined screen of interest, based on pixels present in the motion direction across the plurality of the time series screens, and a motion direction calculator for calculating the motion direction of each pixel in the predetermined screen of interest, by processing the plurality of the motion directions based on the scores corresponding to the plurality of the motion directions. The image processing device thus performs a highly accurate motion direction detection process within a short calculation time.

Claims

1. An image processing device for determining the presence or absence of a moving area in a predetermined screen of interest from among a plurality of time series screens, the image processing device comprising:

score calculating means for calculating a score corresponding to each of a plurality of motion directions of pixels in the predetermined screen of interest, based on pixels present in the motion direction across the plurality of the time series screens so that a plurality of scores are calculated for each pixel;

motion direction calculating means for calculating the motion direction of each pixel in the predetermined screen of interest, by processing the plurality of the motion directions based on the plurality of scores corresponding to the plurality of the motion directions respectively; and

determining means for determining the presence or absence of the moving area based on the motion direction calculated by the motion direction calculating means.

2. An image processing device according to claim 1, further comprising:

direction setting means for setting the plurality of the motion directions of each pixel in the screen of interest; and

range setting means for setting a range of extraction for extracting the pixels in the plurality of the screens present in the motion direction set by the direction setting means to calculate the score for each motion direction,

wherein the score calculating means calculates the score of the motion direction, based on the pixels within the range set by the range setting means, from among the pixels of the plurality of screens present in the motion direction set by the direction setting means.

3. An image processing device according to 2, wherein the range setting means shifts, in the motion direction, the range of the pixels which is used for calculating the score of the motion direction, and

the score calculating means calculates the score for each range of the pixels shifted in the motion direction, and calculates a final score of the motion direction based on the resulting scores.

4. An image processing device according to claim 3, wherein the calculating means calculates the score for each range of the pixels shifted in the motion direction, and sums the scores, thereby calculating a final score for the motion direction.

5. An image processing device according to claim 2, further comprising score memory means for storing the score of each pixel in the screen of interest, and

score updating means for updating the score stored in the score memory means, based on the score determined from the pixels present in the motion direction set by the direction setting means.

6. An image processing device according to claim 5, wherein the score updating means updates the score stored in the score memory means by summing thereto the score that is determined from the pixels present in the motion direction set by the direction setting means.

7. An image processing device according to claim 2, wherein the direction setting means for successively setting the motion directions, comprises:

motion direction memory means for storing the motion direction of each pixel in the screen of interest, and

motion direction updating means for updating the motion direction stored in the motion direction memory means with the motion direction newly set by the direction setting means.

8. An image processing device according to claim 2, wherein the score calculating means regards, as the score of the motion direction, the reciprocal of a variance of the pixel values of the pixels within a range set by the range setting means, from among the pixels of the plurality of the screens present in the motion direction set by the direction setting means.

9. An image processing device according to claim 1, wherein the determining means detects the moving area in the screen of interest by binarizing the motion direction.

10. An image processing device according to claim 1, wherein the determining means detects the still area in the screen of interest by binarizing the motion direction.

11. An image processing device according to claim 1, wherein the determining means determines the presence or absence of the moving area by binarizing the mean of the motion directions of the pixels in the screen of interest.

12. An image processing device according to claim 1, wherein the determining means determines the presence or absence of the moving area, based on the motion directions of the pixels having the scores thereof higher than a predetermined threshold, in the screen of interest.

13. An image processing device according to claim 1, the score calculating means calculates the scores of the plurality of the motion direction of each pixel in the predetermined screen of interest, based on a plurality of image blocks in each motion direction of the plurality of the time series screens.

14. An image processing device according to claim 1, further comprising memory means for storing moving image data composed of a plurality of time series screens, wherein the score calculating means calculates the scores of the plurality of the motion directions of the pixels in the predetermined screen of interest, based on the pixels present in the motion direction in all of the time series screens stored in the memory means.

15. An image processing device for determining the presence or absence of a moving area in a predetermined screen of interest from among a plurality of time series screens, the image processing device comprising:

score calculating means for calculating a score corresponding to each of a plurality of motion directions of pixels in the predetermined screen of interest, based on pixels present in the motion direction across the plurality of the time series screens;

motion direction calculating means for calculating the motion direction of each pixel in the predetermined screen of interest, by processing the plurality of the motion directions based on the scores corresponding to the plurality of the motion directions;

determining means for determining the presence or absence of the moving area based on the motion direction calculated by the motion direction calculating means;

direction setting means for setting the plurality of the motion directions of each pixel in the screen of interest; and

range setting means for setting a range of extraction for extracting the pixels in the plurality of the screens present in the motion direction set by the direction setting means to calculate the score for each motion direction,

wherein the score calculating means calculates the score of the motion direction, based on the pixels within the range set by the range setting means, from among the pixels of the plurality of screens present in the motion direction set by the direction setting means,

wherein the direction setting means for successively setting the motion directions, comprises;

motion direction memory means for storing the motion direction of each pixel in the screen of interest, and

motion direction updating means for updating the motion direction stored in the motion direction memory means with the motion direction newly set by the direction setting means, and

wherein the motion direction updating means updates the motion direction stored in the motion direction memory means by forming a weighted sum of the motion direction stored in the motion direction memory means and the motion direction newly set by the direction setting means, with the scores corresponding to the respective motion directions accounted for as the weights therefor.

16. An image processing method for determining the presence or absence of a moving area in a predetermined screen of interest from among a plurality of time series screens, the image processing method comprising the steps of:

calculating a score corresponding to each of a plurality of motion directions of pixels in the predetermined screen of interest, based on pixels present in the motion direction across the plurality of the time series screens so that a plurality of scores are calculated for each pixel;

calculating the motion direction of each pixel in the predetermined screen of interest, by processing the plurality of the motion directions based on the plurality of scores corresponding to the plurality of the motion directions respectively; and

determining the presence or absence of the moving area based on the calculated motion direction.

17. A storage medium for storing a computer executable program for determining the presence or absence of a moving area in a predetermined screen of interest from among a plurality of time series screens, the program comprising the steps of:

calculating a score corresponding to each of a plurality of motion directions of pixels in the predetermined screen of interest, based on pixels present in the motion direction across the plurality of the time series screens so that a plurality of scores are calculated for each pixel;

calculating the motion direction of each pixel in the predetermined screen of interest, by processing the plurality of the motion directions based on the plurality of scores corresponding to the plurality of the motion directions respectively, and

determining the presence or absence of a moving area based on the calculated motion direction.