Picture generating apparatus and picture generating method

• US 7,015,951 B1
• Filed: 05/07/1999
• Issued: 03/21/2006
• Est. Priority Date: 05/08/1998
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yoshigahara, Takayuki, Fujita, Toshifumi
• Primary Examiner(s): Vu, Ngoc-Yen
• Assistant Examiner(s): JERABEK, KELLY L
• Application Number:
  US09/462,345
• Time in Patent Office: 2,510 Days
• Field of Search: 348/187, 348135-137, 348/633, 348 42- 48, 348140-143, 348/153, 348/159, 348/207.99, 348/169, 382/154, 382/278, 382284-285, 382219-220, 396/89, 396/121, 396/128
• US Class Current: 348/207.99
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area

Abstract

A picture generating apparatus according to this invention comprises: two image pick-up devices 3a, 3b or more adapted for picking up image of an object 2 to be imaged to generate picture data and respectively disposed at different positions; a correlation detecting section 5 for comparing, with each other, respective picture data generated by the respective image pick-up devices 3a, 3b on epipolar line determined by connecting correspondence points of line of sight connecting virtual position A and the object 2 to be imaged and line of sight connecting position of each of the image pick-up devices 3a, 3b and the object 2 to be imaged to detect correlation therebetween; and a distance picture generating section 5 for generating distance picture indicating distance between virtual position and the object to be imaged on the basis of correlation detected by the correlation detecting section 5.

Claims

1. A picture generating apparatus, comprising:

at least two image pick-up means, for picking up an image of an object to be imaged, respectively disposed at different positions;

correlation detecting means for comparing, with each other, on an epipolar line which is an intersection line of an image pick-up plane of the image pick-up means and a plane determined by a line of sight connecting a virtual position and the object to be imaged, and a line connecting an optical center of the virtual position and an optical center of a detection camera, respective picture data generated by the respective image pick-up means to detect correlation therebetween; and

distance picture generating means for generating distance picture indicating distance between a virtual position and the object to be imaged on the basis of the correlation detected by the correlation detecting means.

2. A picture generating apparatus as set forth in claim 1, wherein the correlation detecting means compares, with each other, picture data of small areas consisting of plural pixel data positioned on the epipolar line to detect correlation therebetween.

3. A picture generating apparatus as set forth in claim 1, which comprises variable-density (gradation) picture generating means for generating variable-density picture when the object to be imaged is imaged from the virtual position on the basis of respective picture data generated by the respective image pick-up means,

wherein the variable-density picture generating means generates variable-density picture by using luminance information of picture data generated by the respective image pick-up means.

4. A picture generating apparatus as set forth in claim 1, which comprises a reference camera disposed at the virtual position, wherein the reference camera generates variable-density picture of the object to be imaged, and wherein the distance picture generating means generates distance picture indicating distance between the reference camera and the object to be imaged on the basis of picture data generated by the two image pick-up means or more.

5. A picture generating apparatus as set forth in claim 4, which comprises:

light emitting means for irradiating pattern light of a predetermined area onto the object to be imaged; and

filter means for shielding pattern light of the predetermined area incident to the reference camera,

wherein the reference camera generates variable-density picture of the object to be imaged, and

wherein the distance picture generating means generates distance picture indicating distance between the reference camera and the object to be imaged on the basis of picture data that the two image pick-up means or more generate by using light reflected from the object to be imaged onto which the pattern light is irradiated.

6. A picture generating apparatus as set forth in claim 4, wherein the reference camera is used for generating distance data indicating the relationship between pictures imaged by the respective image pick-up means and distance between the virtual position and the object to be imaged.

7. A picture generating apparatus as set forth in claim 1, wherein the correlation detecting means compares, with each other, picture data on each epipolar line corresponding to distance between the virtual position and the object to be imaged to detect correlations every respective distances with respect to respective pixel blocks consisting of at least one pixel constituting distance picture, and

wherein the distance picture generating means allows distance in picture data having highest correlation of correlations every respective distances detected by the correlation detecting means to be distance with respect to the object to be imaged of pixel block of the distance picture.

8. A picture generating apparatus as set forth in claim 7, which comprises variable-density picture generating means for generating picture data when the object to be imaged is imaged from the virtual position by using plural picture data imaged by the respective image pick-up means on epipolar line corresponding to distance having highest correlation.

9. A picture generating apparatus as set forth in claim 1, wherein the correlation detecting means carries out, with respect to all distances, processing to compare, with each other, picture data indicating a predetermined distance of plural picture data on each of the epipolar lines corresponding to distance between the virtual position and the object to be imaged to detect correlation in regard to the predetermined distance with respect to the entirety of the distance picture, and

wherein the distance picture generating means allows distance in each of picture data having highest correlation every respective distances every respective pixel blocks consisting of at least one pixel constituting respective distance pictures to be distance with respect to the object to be imaged in regard to each of pixel blocks.

10. A picture generating apparatus as set forth in claim 9, which comprises variable-density picture generating means for generating picture data when the object to be imaged is imaged from the virtual position by using plural picture data imaged by the respective image pick-up means on the epipolar line corresponding to distance having highest correlation.

11. A picture generating method, comprising:

picking up an image of an object to be imaged by at least two image pick-up means respectively disposed at different positions to generate picture data;

comparing, with each other, on an epipolar line which is an intersection line of an image pick-up plane of the image pick-up means and a plane determined by a line of sight connecting a virtual position and the object to be imaged, and a line connecting an optical center of the virtual position and an optical center of a detection, respective picture data generated by the respective image pick-up means to detect correlation therebetween; and

generating distance picture indicating distance between virtual position and the object to be imaged on the basis of the detected correlation therebetween.

12. A picture generating method as set forth in claim 11, wherein picture data of small areas consisting of plural pixel data located on the epipolar line are compared with each other to detect correlation therebetween.

13. A picture generating method as set forth in claim 11, wherein luminance patterns of respective picture data generated by the respective (solid-state) image pick-up means (devices) are used to generate variable-density (gradation) picture.

14. A picture generating method as set forth in claim 11, which comprises:

comparing, with each other, respective picture data imaged by the (solid-state) two image pick-up means (devices) or more to detect correlation of the entirety of picture; and

generating distance picture indicating distance between the virtual position and the object to be imaged on the basis of the correlation of the entirety of the picture.

15. A picture generating method as set forth in claim 11, which comprises:

comparing, with each other, picture data on each epipolar line corresponding to distance between the virtual position and the object to be imaged;

detecting correlations every respective distances with respect to respective pixel blocks consisting of at least one pixel constituting the distance picture; and

allowing distance in each picture data having highest correlation of correlations every respective distances to be distance between the virtual position and the object to be imaged.

16. A picture generating method as set forth in claim 15, wherein plural picture data imaged by the respective (solid-state) image pick-up means (devices) on epipolar line corresponding to distance in each of the picture data having highest correction are used to generate picture data when the object to be imaged is imaged from the virtual position.

17. A picture generating method as set forth in claim 11, wherein a procedure is taken to carry out processing with respect to all distances to compare, with each other, picture data indicating a predetermined distance of plural picture data on each epipolar line corresponding to distance between the virtual position and the object to be imaged to detect, in connection with the entirety of the distance picture, correlations with respect to the predetermined distance, and

wherein a further procedure is taken to allow distance in each of the picture data having highest correlation of correlations every respective distances, every respective pixel blocks consisting of at least one pixel constituting respective distance pictures, to be distance with respect to the object to be imaged in regard to each of pixel blocks.

18. A picture generating method as set forth in claim 17, wherein plural picture data imaged by (solid-state) image pick-up means (devices) on epipolar line corresponding to distance in each of the picture data having highest correlation are used to generate picture data when the object to be imaged is imaged from the virtual position.