Image processing apparatus suitable for measuring depth of given point with respect to reference point using two stereoscopic images

• US 4,837,616 A
• Filed: 06/17/1988
• Issued: 06/06/1989
• Est. Priority Date: 06/19/1987
• Status: Expired due to Term

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Inoue, Yoichi, Kasano, Akira
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US07/207,831
• Time in Patent Office: 354 Days
• Field of Search: 358/107, 358/88, 356/1, 382/23
• US Class Current: 348/139
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area

Abstract

First and second images as stereoscopic images of an object are obtained. The second image is shifted by a value dx. Absolute values of differences between image data of the shifted image and the first image are calculated, thus obtaining a third image. The third image is subjected to average value filtering to obtain a fourth image. A fifth image which has the same size as that of the fourth image and has a maximum gray level in an initial state is prepared. Image data having a smaller value is selected from image data of the fourth and fifth images to obtain a sixth image. Absolute values of differences between image data of the sixth and fifth images are calculated, thus obtaining a seventh image having the calculated absolute values as image data. A ninth image which has the same size as that of the seventh image. When the gray level of the pixel of the sevength image is 0, an eighth image is obtained to have the image data of the ninth image. When the gray level of the pixel of the seventh image is other than 0, the eighth image is obtained to have image data representing a gray level corresponding the value dx as the image data. After the eighth image is obtained, the sixth image is set as the fifth image, the eighth image is set as the ninth image and value dx is updated, and the eighth image is repetitively obtained. The finally obtained eighth image is determined as a parallax image representing a deviation of positions of corresponding pixels on the first and second images by gray levels. Based on the parallax image, a to an arbitrary point with respect to a reference point is determined.

Claims

1. An image processing apparatus comprising:

first frame memory means for storing a first image as a two-dimensional gray-level image obtained when a three-dimensional object to be measured is viewed from a predetermined direction, a position of each pixel of said first image being expressed by coordinates defined by a first direction and a second direction perpendicular to the first direction;

second frame memory means for storing a second image as a two-dimensional gray-level image obtained when said three-dimensional object to be measured is viewed from a direction different from said predetermined direction, a position of each pixel of said second image being expressed by coordinates defined along said first and second directions, and a deviation between said first and second images being generated in one direction of said first and second directions;

shifted image generating means for shifting, by a value, said second image within a predetermined range in said one direction to obtain a shifted image;

third image generating means for calculating the absolute values representing differences between image data of pixels located at identical coordinate positions in said shifted and first images, thereby obtaining a third image, the absolute values being image data representing gray levels of pixels located in said third image at coordinate positions which correspond to the pixel positions of said first and shifted images;

average value filtering means for performing average value filtering for each pixel of said third image so as to obtain a fourth image;

holding means for holding a fifth image having the same size as said fourth image, each pixel of said fifth image having a maximum gray level in an initial state;

sixth image generating means for selecting image data representing a smaller gray level from image data of pixels at identical coordinate positions of said fourth and fifth images, thereby obtaining a sixth image, the selected image data being image data representing gray levels of pixels in said sixth image located at coordinate positions which correspond to the pixel position of said fourth and fifth images;

seventh image generating means for calculating absolute values of differences between image data of pixels at identical coordinate positions in said sixth and fifth images, and processing the absolute value, thereby obtaining a seventh image, the absolute values being image data representing gray levels of pixels in said seventh image located at coordinate positions which correspond the pixel positions in said fifth and sixth images;

means for holding an eighth image having the same size as that of said seventh image;

ninth image generating means for detecting a gray level of each pixel of said seventh image, and obtaining a ninth image, when the detected gray level is 0, from the image data of pixel located at the corresponding coordinate position in the eighth image, which is used as image data representing a gray level of pixel located at the corresponding coordinate position in the ninth image, and, when the detected gray level is other than 0, from image data representing a gray level corresponding to said value dx, which is used as image data representing a gray level of pixel located at the corresponding coordinate position in said ninth image; and

updating means for, after said ninth image has been obtained, setting said sixth image as said fifth image and said ninth image as said eighth image, updating the value dx, for repetitively obtaining said ninth image, and determining the finally obtained ninth image as a parallax image representing, by gray levels, a deviation of corresponding pixels in said first and second images by gray levels.

2. An apparatus according to claim 1, wherein said ninth image generating means comprises:

a first table for storing first data in an entry at address "0" and storing second data in other entries;

means for addressing said first table using the image data of each pixel of said seventh image;

means for obtaining a tenth image represented by values of addressed entries of said first table used as image data of pixels in said tenth image located at corresponding coordinate positions which correspond the pixel positions in said seventh image;

means for coupling image data of pixels at identical coordinate positions in said tenth and eighth images, such that image data of said tenth image serve as upper address portions, and image data of said eighth image serve as lower address portions;

a second table having a plurality of entries, for storing, in each entry, an upper address of which has the first data, a value identical to that of the lower address portion, and storing, in an entry, an upper address of which has the second data, image data representing a gray level corresponding to the value dx; and

means for addressing said second table using the addresses coupled by said coupling means and obtaining said ninth image represented by the contents of designated entries used as image data of corresponding pixels in said ninth image.

3. An apparatus according to claim 1, wherein said average value filtering means calculates an average value of the gray levels of the predetermined pixels around each pixel of said second image, and determines the average value as image data representing a gray level of a pixel in said third image having a coordinate position which is identical to the pixel in said second image.

4. An apparatus according to claim 1, further comprising means for obtaining a distance to an arbitrary point with respect to a given reference point on a deviation of positions of corresponding pixels on said first and second images represented by said parallax image.

5. An image processing apparatus comprising:

first frame memory means for storing a first image as a two-dimensional gray-level image obtained when a three-dimensional object to be measured is viewed from a predetermined direction, a position of each pixel of said first image being expressed by coordinates defined by a first direction and a second direction perpendicular to the first direction;

second frame memory means for storing a second image as a two-dimensional gray-level image obtained when said three-dimensional object to be measured is viewed from a different direction from said predetermined direction, a position of each pixel of said second image being expressed by coordinates defined along said first and second directions, and a deviation between said first and second images being generated in only one direction of said first and second directions;

means for shifting, by a value dx, said second image within a predetermined range in said one direction so as to obtain shifted image;

third image generating means for calculating similarities of image within predetermined area around the pixels located at identical coordinate positions in said shifted and first images and obtaining a third image represented by data indicating the similarities used as image data of pixels located at the corresponding coordinate positions of said third image;

means for holding a fourth image, said fourth image having substantially the same size as that of said third image, and represented by image data exhibiting minimum similarities in an initial state;

fifth image generating means for comparing image data of pixels located at identical coordinate positions in said third and fourth images, for, when the similarity indicated by the image data of said third image is larger than that of said fourth image, obtaining a fifth image being represented by the image data in said third image as image data of pixels at the corresponding coordinate positions of said fifth image, and when the similarity indicated by the image data of said third image is smaller than that of said fourth image, obtaining said fifth image represented by the image data in said fourth image as image data of pixels at the corresponding coordinate positions of said fifth image;

means for holding a sixth image having substantially the same size as that of said fifth image;

seventh image generating means for obtaining a seventh image represented by data indicating a gray level corresponding to a present value dx as image data of said seventh image of pixels at identical coordinate positions to the pixels for which said seventh image generating means determines that the similarity represented by the image data of said third image is larger than that of said fourth image data, and for obtaining said seventh image having the image data of said sixth image as image data of pixels at the corresponding coordinate positions of said seventh image for pixels at the identical coordinate positions to the pixels for which said seventh image generating means determines that the similarity represented by the image data of said third image is smaller than that of said fourth image data; and

means for, after said seventh image is obtained, setting said fifth image as said fourth image and said seventh image as said sixth image and updating the value dx, for repetitively obtaining said seventh image, and for determining the finally obtained seventh image as a parallax image representing a deviation of corresponding pixels on said first and second images by gray levels.

6. An apparatus according to claim 5, wherein said third image generating means has:

means for calculating absolute values of differences between image data of pixels at identical coordinate positions of said shifted and first images and obtaining a tenth image to have the calculated absolute values as image data of pixels at identical coordinate positions thereof; and

average value filtering means for performing average value filtering processing for each pixel of said tenth image to obtain said third image, the smaller image data of each pixel of said third image becomes, the higher the similarity becomes.

7. An apparatus according to claim 6, wherein said seventh image generating means has:

eleventh image generating means for calculating absolute values of differences between image data of pixels at identical coordinate positions of said fifth and fourth images and obtaining an eleventh image to have the calculated absolute values as image data of pixels at identical coordinate positions thereof; and

means for detecting for detecting a gray level of each pixel of said eleventh image, and for, when the detected gray level is 0, obtaining said seventh image to have the image data of pixels at the corresponding coordinate positions of said six image as image data of pixels at the corresponding coordinate positions thereof, and when the detected gray level of a given pixel of said eleventh image is other than 0, obtaining said seventh image to have the image data representing a gray level corresponding to the value dx as image data of pixels at the corresponding coordinate positions thereof.

8. An image processing method comprising:

obtaining a first image as a two-dimensional gray-level image when a three-dimensional object to be measured is viewed from a predetermined direction, a position of each pixel of said first image being expressed by coordinates defined by a first direction and a second direction perpendicular to the first direction;

obtaining a second image as a two-dimensional gray-level image obtained when said three-dimensional object to be measured is viewed from a direction different from said predetermined direction, a position of each pixel of said second image being expressed by coordinates defined by said first and second directions, and a deviation between said first and second images being generated in one of said first and second directions;

shifting said second image by a value in said one direction within a predetermined range to obtain an image and calculating absolute values of differences between image data of pixels at identical coordinate positions of said shifted and first images so as to obtain a third image which is represented by the absolute values as the image data of pixels at the corresponding coordinate positions of said third image;

performing average value filtering processing for each pixel of said third image to obtain a fourth image;

preparing a fifth image which has the same size as that of said fourth image and each pixel of which has a maximum gray level in an initial state;

selecting image data having a smaller value from image data of pixels at identical positions of said fourth and fifth images, and obtaining a sixth image represented by the selected image data as image data of pixels at the corresponding coordinate positions of said sixth image;

calculating absolute values of differences between image data of pixels at identical coordinate positions of said sixth and fifth images, and obtaining a seventh image represented by the absolute values as image data of pixels at the corresponding coordinate positions of said seventh image;

preparing an eighth image having the same size as that of said seventh image;

detecting a gray level of each pixel of said seventh image, and when the detected gray level is 0, obtaining a ninth image having the image data of pixels located at the corresponding coordinate positions in said eighth image as image data of pixels at the corresponding coordinate positions of said eighth image, and when the detected gray level is other than 0, obtaining said ninth image having data representing a gray level corresponding to the value dx as image data of pixels at the corresponding coordinate positions of said eighth image; and

after said ninth image is obtained, setting said sixth image as said fifth image and said ninth image as said eighth image and updating the value dx, repetitively obtaining said ninth image, and determining the finally obtained ninth image as a parallax image representing a deviation of corresponding pixels on said first and second images by gray levels.

9. A method according to claim 8, wherein the step of obtaining said ninth image comprises:

preparing a first table for storing first data in an entry of address "0" and storing second data in other entries;

addressing said first table using the image data of each pixel of said seventh image;

obtaining a tenth image which has values of addressed entries of said first table as image data of pixels at corresponding coordinate positions;

coupling image data of pixels at identical coordinate positions of said tenth and eighth images, such that image data of said tenth image serve as upper address portions, and image data of said eighth image serve as lower address portions;

preparing a second table for storing, in an entry, an upper address of which has the first data, a value identical to that of the lower address portion, and storing, in an entry, an upper address of which has the second data, image data representing a gray level corresponding to the value dx; and

addressing said second table using the addresses coupled by the step of coupling the image data and obtaining said ninth image to have the contents of designated entries as image data of corresponding pixels thereof.

10. A method according to claim 8, wherein the step of performing average value filtering processing is to calculate an average value of image data of predetermined m×

n pixels around each pixel of said second image, and to determine the average value as image data of a pixel of said third image at the identical coordinate position to the pixel of said second image.

11. A method according to claim 8, further comprising the step of obtaining a distance to an arbitrary point with respect to a given reference point based on a deviation of positions of corresponding pixels on said first and second images represented by said parallax image.