Image processing device
First Claim
1. An image processing device for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said image processing device comprising:
- weighted difference image data calculating means for calculating a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on weighting which is to correspond to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and outputting as weighted difference image data corresponding to said frame of interest;
inter-weighted-difference-image correlation data computing means for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector which is to indicate relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest containing each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for, with regard to each predetermined increment of said inter-weighted-difference-image correlation data, detecting at least one of said weighting and said movement vector where the correlation between said weighted difference image data is maximum, setting said weighting to said mixture ratio corresponding to said increment of said frame of interest in the event that said weighting is detected, and setting said movement vector to said movement vector corresponding to said increment of said frame of interest in the event that said movement vector is detected.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention enables a movement vector of a moving object to the detected, by detecting a mixture ratio indicating the proportion of mixing with an image, or taking into consideration a region mixed with the image. A weighted frame difference computation unit 3423 outputs weighted difference. A movement compensation unit 3424 and a correlation value computing unit 3427 compute the correlation between a block of interest containing each pixel of interest of weighted difference image data, and a corresponding block of weighted difference image data of an adjacent frame. A maximum value judgment unit 3428 detects at least one of weight and a movement vector wherein correlation between weighted difference image data is greatest. The present invention can be applied to signals processing device s for processing images.
-
Citations
131 Claims
-
1. An image processing device for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said image processing device comprising:
-
weighted difference image data calculating means for calculating a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on weighting which is to correspond to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and outputting as weighted difference image data corresponding to said frame of interest;
inter-weighted-difference-image correlation data computing means for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector which is to indicate relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest containing each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for, with regard to each predetermined increment of said inter-weighted-difference-image correlation data, detecting at least one of said weighting and said movement vector where the correlation between said weighted difference image data is maximum, setting said weighting to said mixture ratio corresponding to said increment of said frame of interest in the event that said weighting is detected, and setting said movement vector to said movement vector corresponding to said increment of said frame of interest in the event that said movement vector is detected. - View Dependent Claims (2, 3, 4, 5)
-
-
6. An image processing method for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said method comprising:
-
a weighted difference image data calculating step for calculating a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on weighting which is to correspond to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and outputting as weighted difference image data corresponding to said frame of interest;
an inter-weighted-difference-image correlation data computing step for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector which is to indicate relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest containing each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for, with regard to each predetermined increment of said inter-weighted-difference-image correlation data, detecting at least one of said weighting and said movement vector where the correlation between said weighted difference image data is maximum, setting said weighting to said mixture ratio corresponding to said increment of said frame of interest in the event that said weighting is detected, and setting said movement vector to said movement vector corresponding to said increment of said frame of interest in the event that said movement vector is detected. - View Dependent Claims (7, 8, 9, 10)
-
-
11. A recording medium storing a computer-readable program for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said program comprising:
-
a weighted difference image data calculating step for calculating a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on weighting which is to correspond to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and outputting as weighted difference image data corresponding to said frame of interest;
an inter-weighted-difference-image correlation data computing step for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector which is to indicate relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest containing each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for, with regard to each predetermined increment of said inter-weighted-difference-image correlation data, detecting at least one of said weighting and said movement vector where the correlation between said weighted difference image data is maximum, setting said weighting to said mixture ratio corresponding to said increment of said frame of interest in the event that said weighting is detected, and setting said movement vector to said movement vector corresponding to said increment of said frame of interest in the event that said movement vector is detected. - View Dependent Claims (12, 13, 14, 15)
-
-
16. A program for causing a computer, which processes image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, to execute:
-
a weighted difference image data calculating step for calculating a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on weighting which is to correspond to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and outputting as weighted difference image data corresponding to said frame of interest;
an inter-weighted-difference-image correlation data computing step for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector which is to indicate relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest containing each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for, with regard to each predetermined increment of said inter-weighted-difference-image correlation data, detecting at least one of said weighting and said movement vector where the correlation between said weighted difference image data is maximum, setting said weighting to said mixture ratio corresponding to said increment of said frame of interest in the event that said weighting is detected, and setting said movement vector to said movement vector corresponding to said increment of said frame of interest in the event that said movement vector is detected. - View Dependent Claims (17, 18, 19, 20)
-
-
21. An image-taking device, comprising:
-
image-taking means for outputting a subject image taken by an image-taking device having a predetermined number of pixels having time-integration effects as image data made up of a predetermined number of pieces of pixel data;
weighted difference image data calculating means for calculating a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on weighting which is to correspond to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and outputting as weighted difference image data corresponding to said frame of interest;
inter-weighted-difference-image correlation data computing means for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector which is to indicate relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest containing each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for, with regard to each predetermined increment of said inter-weighted-difference-image correlation data, detecting at least one of said weighting and said movement vector where the correlation between said weighted difference image data is maximum, setting said weighting to said mixture ratio corresponding to said increment of said frame of interest in the event that said weighting is detected, and setting said movement vector to said movement vector corresponding to said increment of said frame of interest in the event that said movement vector is detected. - View Dependent Claims (22, 23, 24, 25)
-
-
26. An image processing device for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said image processing device comprising:
-
weighting information generating means for generating a plurality of weights corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
weighted difference image data detecting means for detecting a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on said weight indicated by said weighting information, and outputting as weighted difference image data corresponding to said frame of interest;
inter-weighted-difference-image correlation data detecting means for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector for indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest made up of at least one pixel and centered on each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
mixture ratio detecting means for, with regard to each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, detecting said weighting where the correlation between said weighted difference image data is maximum, and outputting as said mixture ratio corresponding to said increment of said frame of interest. - View Dependent Claims (27, 28, 29, 30, 31, 32, 33, 34, 35, 36)
-
-
37. An image processing method for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said method comprising:
-
a weighting information generating step for generating a plurality of weights corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
a weighted difference image data detecting step for detecting a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on said weight indicated by said weighting information, and outputting as weighted difference image data corresponding to said frame of interest;
an inter-weighted-difference-image correlation data detecting step for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector for indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest made up of at least one pixel and centered on each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a mixture ratio detecting step for, with regard to each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, detecting said weighting where the correlation between said weighted difference image data is maximum, and outputting as said mixture ratio corresponding to said increment of said frame of interest. - View Dependent Claims (38, 39, 40, 41, 42, 43, 44, 45, 46, 47)
-
-
48. A recording medium storing a computer-readable program for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said program comprising:
-
a weighting information generating step for generating a plurality of weights corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
a weighted difference image data detecting step for detecting a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on said weight indicated by said weighting information, and outputting as weighted difference image data corresponding to said frame of interest;
an inter-weighted-difference-image correlation data detecting step for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector for indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest made up of at least one pixel and centered on each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a mixture ratio detecting step for, with regard to each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, detecting said weighting where the correlation between said weighted difference image data is maximum, and outputting as said mixture ratio corresponding to said increment of said frame of interest. - View Dependent Claims (49, 50, 51, 52, 53, 54, 55, 56, 57, 58)
-
-
59. A program for causing a computer, which processes image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, to execute:
-
a weighting information generating step for generating a plurality of weights corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
a weighted difference image data detecting step for detecting a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on said weight indicated by said weighting information, and outputting as weighted difference image data corresponding to said frame of interest;
an inter-weighted-difference-image correlation data detecting step for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector for indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest made up of at least one pixel and centered on each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a mixture ratio detecting step for, with regard to each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, detecting said weighting where the correlation between said weighted difference image data is maximum, and outputting as said mixture ratio corresponding to said increment of said frame of interest. - View Dependent Claims (60, 61, 62, 63, 64, 65, 66, 67, 68, 69)
-
-
70. An image-taking device, comprising:
-
image-taking means for outputting a subject image taken by an image-taking device having a predetermined number of pixels having time-integration effects as image data made up of a predetermined number of pieces of pixel data;
weighting information generating means for generating a plurality of weights corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
weighted difference image data detecting means for detecting a weighted difference between each pixel in a frame of interest in said image data and each pixel in an adjacent frame which is adjacent to said frame of interest in said image data, based on said weight indicated by said weighting information, and outputting as weighted difference image data corresponding to said frame of interest;
inter-weighted-difference-image correlation data detecting means for matching a relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to a movement vector for indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, computing the correlation between a block of interest made up of at least one pixel and centered on each pixel of interest of said weighted difference image data of said frame of interest and a corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
mixture ratio detecting means for, with regard to each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, detecting said weighting where the correlation between said weighted difference image data is maximum, and outputting as said mixture ratio corresponding to said increment of said frame of interest. - View Dependent Claims (71, 72, 73, 74, 75, 76, 77, 78, 79, 80)
-
-
81. An image processing device for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said image processing device comprising:
-
region specifying means for specifying a mixed region made up of a mixture of a foreground object component configuring foreground objects and a background object component configuring background objects based on said image data, and outputting region information indicating the specified mixed region;
mixture ratio estimating means for estimating mixture ratio indicating ratio of mixture between said foreground object component and said background object component in said pixel data belonging to said mixed region, based on said image data and said region information;
estimated movement vector generating means for generating a plurality of estimated movement vectors, and outputting each of said estimated movement vectors that have been generated;
difference image data calculating means for calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a first adjacent frame adjacent to said frame of interest of said image data and outputting as first difference image data, and calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a second adjacent frame adjacent to said frame of interest of said image data and outputting as second difference image data;
inter-difference-image correlation data computing means for matching the relative position of said first difference image data and said second difference image data based on said estimated movement vector, computing the correlation between said block of interest made up of at least one pixel centered a pixel of said first difference image data corresponding to said pixel of interest of said frame of interest, and said corresponding block made up of at least one pixel of said second difference image data, and outputting as inter-difference-image correlation data; and
detecting means for detecting said estimated movement vector where the correlation is maximum, based on said inter-difference-image correlation data, and outputting as said movement vector corresponding to said pixel of interest of said frame of interest. - View Dependent Claims (82, 83, 84, 85, 86)
-
-
87. An image processing method for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said method comprising:
-
a region specifying step for specifying a mixed region made up of a mixture of a foreground object component configuring foreground objects and a background object component configuring background objects based on said image data, and outputting region information indicating said specified mixed region;
a mixture ratio estimating step for estimating mixture ratio indicating ratio of mixture between said foreground object component and said background object component in said pixel data belonging to said mixed region, based on said image data and said region information;
an estimated movement vector generating step for generating a plurality of estimated movement vectors, and outputting each of said estimated movement vectors that have been generated;
a difference image data calculating step for calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a first adjacent frame adjacent to said frame of interest of said image data and outputting as first difference image data, and calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a second adjacent frame adjacent to said frame of interest of said image data and outputting as second difference image data;
an inter-difference-image correlation data computing step for matching the relative position of said first difference image data and said second difference image data based on said estimated movement vector, computing the correlation between said block of interest made up of at least one pixel centered a pixel of said first difference image data corresponding to said pixel of interest of said frame of interest, and said corresponding block made up of at least one pixel of said second difference image data, and outputting as inter-difference-image correlation data; and
a detecting step for detecting said estimated movement vector where the correlation is maximum, based on said inter-difference-image correlation data, and outputting as said movement vector corresponding to said pixel of interest of said frame of interest.
-
-
88. A recording medium storing a computer-readable image processing program for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said program comprising:
-
a region specifying step for specifying a mixed region made up of a mixture of a foreground object component configuring foreground objects and a background object component configuring background objects based on said image data, and outputting region information indicating the specified mixed region;
a mixture ratio estimating step for estimating mixture ratio indicating ratio of mixture between said foreground object component and said background object component in said pixel data belonging to said mixed region, based on said image data and said region information;
an estimated movement vector generating step for generating a plurality of estimated movement vectors, and outputting each of said estimated movement vectors that have been generated;
a difference image data calculating step for calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of a frame of interest of said image data and each pixel of a first adjacent frame adjacent to said frame of interest of said image data and outputting as first difference image data, and calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a second adjacent frame adjacent to said frame of interest of said image data and outputting as second difference image data;
an inter-difference-image correlation data computing step for matching the relative position of said first difference image data and said second difference image data based on said estimated movement vector, computing the correlation between said block of interest made up of at least one pixel centered a pixel of said first difference image data corresponding to said pixel of interest of said frame of interest, and said corresponding block made up of at least one pixel of said second difference image data, and outputting as inter-difference-image correlation data; and
a detecting step for detecting said estimated movement vector where the correlation is maximum, based on said inter-difference-image correlation data, and outputting as said movement vector corresponding to said pixel of interest of said frame of interest.
-
-
89. A program for causing a computer, which processes image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, to execute:
-
a region specifying step for specifying a mixed region made up of a mixture of a foreground object component configuring foreground objects and a background object component configuring background objects based on said image data, and outputting region information indicating said specified mixed region;
a mixture ratio estimating step for estimating mixture ratio indicating ratio of mixture between said foreground object component and said background object component in said pixel data belonging to said mixed region, based on said image data and said region information;
an estimated movement vector generating step for generating a plurality of estimated movement vectors, and outputting each of said estimated movement vectors that have been generated;
a difference image data calculating step for calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a first adjacent frame adjacent to said frame of interest of said image data and outputting as first difference image data, and calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a second adjacent frame adjacent to said frame of interest of said image data and outputting as second difference image data;
an inter-difference-image correlation data computing step for matching the relative position of said first difference image data and said second difference image data based on said estimated movement vector, computing the correlation between said block of interest made up of at least one pixel centered a pixel of said first difference image data corresponding to said pixel of interest of said frame of interest, and said corresponding block made up of at least one pixel of said second difference image data, and outputting as inter-difference-image correlation data; and
a detecting step for detecting said estimated movement vector where the correlation is maximum, based on said inter-difference-image correlation data, and outputting as said movement vector corresponding to said pixel of interest of said frame of interest.
-
-
90. An image-taking device, comprising:
-
image-taking means for outputting a subject image taken by an image-taking device having a predetermined number of pixels having time-integration effects as image data made up of a predetermined number of pieces of pixel data;
region specifying means for specifying a mixed region made up of a mixture of a foreground object component configuring foreground objects and a background object component configuring background objects based on said image data, and outputting region information indicating said specified mixed region;
mixture ratio estimating means for estimating mixture ratio indicating ratio of mixture between said foreground object component and said background object component in said pixel data belonging to said mixed region, based on said image data and said region information;
estimated movement vector generating means for generating a plurality of estimated movement vectors, and outputting each of said estimated movement vectors that have been generated;
difference image data calculating means for calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a first adjacent frame adjacent to said frame of interest of said image data and outputting as first difference image data, and calculating difference based on said mixture ratio of said pixel of interest of said frame of interest between each pixel of said frame of interest of said image data and each pixel of a second adjacent frame adjacent to said frame of interest of said image data and outputting as second difference image data;
inter-difference-image correlation data computing means for matching the relative position of said first difference image data and said second difference image data based on said estimated movement vector, computing the correlation between said block of interest made up of at least one pixel centered a pixel of said first difference image data corresponding to said pixel of interest of said frame of interest, and said corresponding block made up of at least one pixel of said second difference image data, and outputting as inter-difference-image correlation data; and
detecting means for detecting said estimated movement vector where the correlation is maximum, based on said inter-difference-image correlation data, and outputting as said movement vector corresponding to said pixel of interest of said frame of interest. - View Dependent Claims (91, 92, 93, 94, 95)
-
-
96. An image processing device for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said image processing device comprising:
-
weighting information generating means for generating a plurality of weights each with differing values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating said weight of each of the generated weights;
weighted difference image data calculating means for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of said frame of interest of said image data and each pixel of said adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
movement vector information generating means for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
inter-weighted-difference-image correlation data computing means for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between block of interest made up of at least one pixel centered on each pixel of interest of said weighted difference image data of said frame of interest and said corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector. - View Dependent Claims (97, 98, 99, 100, 101, 102, 103, 104, 105)
-
-
106. An image processing method for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said method comprising:
-
a weighting information generating step for generating a plurality of weights each with differing values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating said weight of each of the generated weights;
a weighted difference image data calculating step for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of said frame of interest of said image data and each pixel of said adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
a movement vector information generating step for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
an inter-weighted-difference-image correlation data computing step for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between block of interest made up of at least one pixel centered on each pixel of interest of said weighted difference image data of said frame of interest and said corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector.
-
-
107. A recording medium storing a computer-readable image processing program for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said program comprising:
-
a weighting information generating step for generating a plurality of weights each with differing values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating said weight of each of the generated weights;
a weighted difference image data calculating step for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of said frame of interest of said image data and each pixel of said adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
a movement vector information generating step for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
an inter-weighted-difference-image correlation data computing step for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between block of interest made up of at least one pixel centered on each pixel of interest of said weighted difference image data of said frame of interest and said corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector.
-
-
108. A program for causing a computer, which processes image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, to execute:
-
a weighting information generating step for generating a plurality of weights each with differing values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating said weight of each of the generated weights;
a weighted difference image data calculating step for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of said frame of interest of said image data and each pixel of said adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
a movement vector information generating step for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
an inter-weighted-difference-image correlation data computing step for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between block of interest made up of at least one pixel centered on each pixel of interest of said weighted difference image data of said frame of interest and said corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector.
-
-
109. An image-taking device, comprising:
-
image-taking means for outputting a subject image taken by an image-taking device having a predetermined number of pixels having time-integration effects as image data made up of a predetermined number of pieces of pixel data;
weighting information generating means for generating a plurality of weights each with differing values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating said weight of each of the generated weights;
weighted difference image data calculating means for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of said frame of interest of said image data and each pixel of said adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
movement vector information generating means for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
inter-weighted-difference-image correlation data computing means for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between block of interest made up of at least one pixel centered on each pixel of interest of said weighted difference image data of said frame of interest and said corresponding block made up of at least one pixel of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of at least one pixel of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector. - View Dependent Claims (110, 111, 112, 113, 114, 115, 116, 117, 118)
-
-
119. An image processing device for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said image processing device comprising:
-
weighting information generating means for generating a plurality of weights each with different values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
weighted difference image data calculating means for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of a frame of interest of said image data and each pixel of an adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
movement vector information generating means for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
inter-weighted-difference-image correlation data computing means for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between a first block of interest made up of a plurality of pixels centered on each pixel of interest of said weighted difference image data of said frame of interest and a first corresponding block made up of a plurality of pixels of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of a plurality of pixels of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector. - View Dependent Claims (120, 121, 122, 123)
-
-
124. An image processing method for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said method comprising:
-
a weighting information generating step for generating a plurality of weights each with different values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
a weighted difference image data calculating step for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of a frame of interest of said image data and each pixel of an adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
a movement vector information generating step for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
an inter-weighted-difference-image correlation data computing step for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between a first block of interest made up of a plurality of pixels centered on each pixel of interest of said weighted difference image data of said frame of interest and a first corresponding block made up of a plurality of pixels of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of a plurality of pixels of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector.
-
-
125. A recording medium storing a computer-readable image processing program for processing image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, said program comprising:
-
a weighting information generating step for generating a plurality of weights each with different values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
a weighted difference image data calculating step for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of a frame of interest of said image data and each pixel of an adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
a movement vector information generating step for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
an inter-weighted-difference-image correlation data computing step for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between a first block of interest made up of a plurality of pixels centered on each pixel of interest of said weighted difference image data of said frame of interest and a first corresponding block made up of a plurality of pixels of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of a plurality of pixels of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector.
-
-
126. A program for causing a computer, which processes image data made up of a predetermined number of pieces of pixel data obtained by an image-taking device having a predetermined number of pixels having time-integration effects, to execute:
-
a weighting information generating step for generating a plurality of weights each with different values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
a weighted difference image data calculating step for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of a frame of interest of said image data and each pixel of an adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
a movement vector information generating step for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
an inter-weighted-difference-image correlation data computing step for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between a first block of interest made up of a plurality of pixels centered on each pixel of interest of said weighted difference image data of said frame of interest and a first corresponding block made up of a plurality of pixels of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
a detecting step for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of a plurality of pixels of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector.
-
-
127. An image-taking device, comprising:
-
image-taking means for outputting a subject image taken by an image-taking device having a predetermined number of pixels having time-integration effects as image data made up of a predetermined number of pieces of pixel data;
weighting information generating means for generating a plurality of weights each with different values, corresponding to a mixture ratio indicating a mixed state in said pixel data of objects which are multiple in the real world, and generating weighting information indicating each of said generated weights;
weighted difference image data calculating means for calculating weighted difference based on said weighting indicated by said weighting information between each pixel of a frame of interest of said image data and each pixel of an adjacent frame which is adjacent to said frame of interest of said image data, and outputting as weighted difference image data corresponding to said frame of interest;
movement vector information generating means for generating a plurality of movement vectors each having different values indicating relative movement between said pixel data of said frame of interest and said pixel data of said adjacent frame, and generating movement vector information indicating each of said generated movement vectors;
inter-weighted-difference-image correlation data computing means for matching the relative position of said weighted difference image data of said frame of interest and said weighted difference image data of said adjacent frame according to said movement vector indicated by said movement vector information, computing the correlation between a first block of interest made up of a plurality of pixels centered on each pixel of interest of said weighted difference image data of said frame of interest and a first corresponding block made up of a plurality of pixels of said weighted difference image data of said adjacent frame, and outputting as inter-weighted-difference-image correlation data; and
detecting means for detecting said weighting and said movement vector where the correlation between said weighted difference image data is maximum, for each predetermined increment made up of a plurality of pixels of said inter-weighted-difference-image correlation data, setting said detected weight to said mixture ratio corresponding to said increment of said frame of interest, setting said detected movement vector to said movement vector corresponding to said increment of said frame of interest, and outputting at least one of said mixture ratio and said movement vector. - View Dependent Claims (128, 129, 130, 131)
-
Specification