DISTANCE MEASURING APPARATUS, DISTANCE MEASURING METHOD, DISTANCE MEASURING PROGRAM, DISTANCE MEASURING SYSTEM, AND IMAGE PICKUP APPARATUS

• US 20120229628A1
• Filed: 10/26/2010
• Published: 09/13/2012
• Est. Priority Date: 11/13/2009
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Ishiyama, Eiji, Masuda, Tomonori
• Granted Patent: US 8,654,195 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A distance measuring apparatus according to the present invention includes an image input unit configured to input a plurality of viewpoint images obtained by capturing a subject from different viewpoints, an obtaining unit configured to obtain, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space, an image correcting unit configured to correct, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data, a parallax calculation unit configured to calculate, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images, a comparing unit configured to compare the corrected viewpoint images over the plurality of distances of interest to specify a piece of optimum calibration data from the plurality of pieces of calibration data, a distance calculation unit configured to calculate a subject distance based on the parallax, and an output unit configured to output, as a distance measurement result, the subject distance calculated from the parallax between the corrected viewpoint images with the piece of optimum calibration data.

Claims

1-72. -72. (canceled)

73. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining unit configured to obtain, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting unit configured to correct, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation unit configured to calculate, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a comparing unit configured to compare the corrected viewpoint images over the plurality of distances of interest to specify a piece of optimum calibration data from the plurality of pieces of calibration data;

a distance calculation unit configured to calculate a subject distance based on the parallax; and

an output unit configured to output, as a distance measurement result, the subject distance calculated from the parallax between the corrected viewpoint images with the piece of optimum calibration data.

74. The distance measuring apparatus according to claim 73, further comprising

a correlation degree calculation unit configured to calculate, for each of the distances of interest, a correlation degree of at least one of luminance and color between the plurality of corrected viewpoint images, wherein

the comparing unit compares the correlation degrees over the plurality of distances of interest to specify a piece of calibration data with the correlation degree being maximum as the piece of optimum calibration data.

75. The distance measuring apparatus according to claim 73, further comprising

a vertical deviation amount calculation unit configured to calculate, for each of the distances of interest, a vertical deviation amount between corresponding pixels between the plurality of corrected viewpoint images, wherein

the comparing unit compares the vertical deviation amounts over the plurality of distances of interest to specify a piece of calibration data with the vertical deviation amount being minimum as the piece of optimum calibration data.

76. The distance measuring apparatus according to claim 73, further comprising

a parallax-calculated pixel count calculating unit configured to calculate, for each of the distances of interest, a parallax-calculated pixel count that allows parallax calculation between the plurality of corrected viewpoint images, wherein

the comparing unit compares the parallax-calculated pixel counts over the plurality of distances of interest to specify a piece of calibration data with the parallax-calculated pixel count being maximum as the piece of optimum calibration data.

77. The distance measuring apparatus according to claim 73, further comprising

a local-noise pixel count calculation unit configured to calculate, for each of the distances of interest, a local-noise pixel count that is locally present in a parallax distribution of the plurality of corrected viewpoint images, wherein

the comparing unit compares the local-noise pixel counts over the plurality of distances of interest to specify a piece of calibration data with the local-noise pixel count being minimum as the piece of optimum calibration data.

82. The distance measuring apparatus according to claim 73, further comprising:

a distance-of-interest determining unit configured to determine a beginning distance of interest among the plurality of distances of interest and to determine the calculated subject distance as a next distance of interest to switch the distance of interest; and

a correlation degree calculation unit configured to calculate, for each of the distances of interest, a correlation degree of at least one of luminance and color between the plurality of corrected viewpoint images, wherein

the comparing unit compares the correlation degrees for each switching of the distance of interest and, when the correlation degree calculated after switching of the distance of interest is smaller than the correlation degree calculated before switching of the distance of interest, adopts the subject distance calculated before switching of the distance of interest as the distance measurement result.

83. The distance measuring apparatus according to claim 73, further comprising:

a distance-of-interest determining unit configured to determine a beginning distance of interest among the plurality of distances of interest and to determine the calculated subject distance as a next distance of interest to switch the distance of interest; and

a vertical deviation amount calculation unit configured to calculate, for each of the distances of interest, a vertical deviation amount between corresponding pixels between the plurality of corrected viewpoint images, wherein

the comparing unit compares the vertical deviation amounts for each switching of the distance of interest and, when the vertical deviation amount calculated after switching of the distance of interest is larger than the vertical deviation amount calculated before switching of the distance of interest, adopts the subject distance calculated before switching of the distance of interest as the distance measurement result.

84. The distance measuring apparatus according to claim 73, further comprising:

a distance-of-interest determining unit configured to determine a beginning distance of interest among the plurality of distances of interest and to determine the calculated subject distance as a next distance of interest to switch the distance of interest; and

a parallax-calculated pixel count calculating unit configured to calculate, for each of the distances of interest, a parallax-calculated pixel count that allows parallax calculation between the plurality of corrected viewpoint images, wherein

the comparing unit compares the parallax-calculated pixel counts for each switching of the distance of interest and, when the parallax-calculated pixel count calculated after switching of the distance of interest is smaller than the parallax-calculated pixel count calculated before switching of the distance of interest, adopts the subject distance calculated before switching of the distance of interest as the distance measurement result.

85. The distance measuring apparatus according to claim 73, further comprising:

a distance-of-interest determining unit configured to determine a beginning distance of interest among the plurality of distances of interest and to determine the calculated subject distance as a next distance of interest to switch the distance of interest; and

a local-noise pixel count calculation unit configured to calculate, for each of the distances of interest, a local-noise pixel count that is locally present in a parallax distribution of the plurality of corrected viewpoint images, wherein

the comparing unit compares the local-noise pixel counts for each switching of the distance of interest and, when the local-noise pixel count calculated after switching of the distance of interest is larger than the local-noise pixel count calculated before switching of the distance of interest, adopts the subject distance calculated before switching of the distance of interest as the distance measurement result.

86. The distance measuring apparatus according to claim 73, further comprising

a representative parallax determining unit configured to create a histogram indicating a relation between a parallax and a number of occurrences between the viewpoint images and to determine a representative parallax based on the number of occurrences in the histogram, wherein

the distance calculation unit calculates the subject distance based on the representative parallax.

87. The distance measuring apparatus according to claim 86, wherein

the representative parallax determining unit detects a most frequent value from the histogram, and determines the most frequent value as the representative parallax.

88. The distance measuring apparatus according to claim 73, further comprising

a representative distance determining unit configured to create a histogram indicating a relation between a distance and a number of occurrences of each of pixels of the viewpoint images and to determine a representative distance based on the number of occurrences in the histogram, wherein

the distance calculation unit calculates a distance of each of the pixels, and

the output unit outputs the representative distance as the distance measurement result.

89. The distance measuring apparatus according to claim 88, wherein

the most frequent value is detected in the histogram, and the most frequent value is determined as the representative distance.

90. The distance measuring apparatus according to claim 86, wherein

whether the number of occurrences is high or low is discriminated in the histogram, and a component of a pixel locally present is removed from the histogram.

91. The distance measuring apparatus according to claim 86, further comprising

a region-of-interest determining unit configured to determine a region of interest for which the number of occurrences is to be calculated, wherein

the number of occurrences is calculated restrictively to the region of interest to create the histogram.

92. The distance measuring apparatus according to claim 91, wherein

the region-of-interest determining unit determines a calculation region for use in exposure control or focusing control at the time of shooting the viewpoint images as the region of interest.

93. The distance measuring apparatus according to claim 91, wherein

the region-of-interest determining unit divides the viewpoint images into a plurality of divisional regions in an array, performs a frequency analysis for each of the divisional regions, and determines a divisional region having a high-frequency component as the region of interest.

94. The distance measuring apparatus according to claim 91, wherein

the region-of-interest determining unit determines a specific target extraction region where a specific target is extracted from the viewpoint images as the region of interest.

95. The distance measuring apparatus according to claim 86, further comprising

a weight determining unit determining a weight with respect to the number of occurrences based on a position of each of the pixels in the viewpoint images, wherein

the histogram is created by weighting based on the weight.

78. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining unit configured to obtain, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting unit configured to correct, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation unit configured to calculate, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a distance calculation unit configured to calculate, for each of the distances of interest, a subject distance based on the parallax;

a comparing unit configured to compare differences each between the subject distance and the distance of interest over the plurality of distances of interest to specify a piece of optimum calibration data from the plurality of pieces of calibration data; and

an output unit configured to output, as a distance measurement result, the subject distance calculated from the parallax between the corrected viewpoint images with the piece of optimum calibration data.

79. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining unit configured to obtain, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

a first parallax calculation unit configured to calculate, for each of the distances of interest, a first parallax based on the distance of interest;

an image correcting unit configured to correct, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a second parallax calculation unit configured to calculate, for each of the distances of interest, a second parallax between the plurality of corrected viewpoint images;

a comparing unit configured to compare differences each between the first parallax and the second parallax over the plurality of distances of interest to specify a piece of optimum calibration data from the plurality of pieces of calibration data;

a distance calculation unit configured to calculate a subject distance based on the second parallax of the corrected viewpoint image with the piece of optimum calibration data; and

an output unit configured to output the calculated subject distance as a distance measurement result.

80. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining unit configured to obtain, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting unit configured to correct, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation unit configured to calculate, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a distance calculation unit configured to calculate, for each of the distances of interest, a subject distance based on the parallax;

a distance difference calculation unit configured to calculate, for each of the distances of interest, a difference between the subject distance and the distance of interest;

a distance-of-interest determining unit configured to determine a beginning distance of interest among the plurality of distances of interest and to determine the calculated subject distance as a next distance of interest to switch the distance of interest;

a comparing unit configured to compare the differences before and after switching of the distance of interest to select a distance measurement result from the plurality of calculated subject distances; and

an output unit configured to output the distance measurement result.

81. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining unit configured to obtain, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting unit configured to correct, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation unit configured to calculate, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a distance calculation unit configured to calculate, for each of the distances of interest, a subject distance based on the parallax;

a distance-of-interest determining unit configured to determine a beginning distance of interest among the plurality of distances of interest and to determine the calculated subject distance as a next distance of interest to switch the distance of interest;

a distance difference calculation unit configured to calculate, for each of the distances of interest, a difference between the subject distance and the distance of interest;

a comparing unit configured to compare the difference and a threshold; and

an output unit configured to output the calculated subject distance as a distance measurement result when the difference is smaller than the threshold.

96. A distance measuring method comprising:

an image input step of inputting a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining step of obtaining, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting step of correcting, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation step of calculating, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a comparing step of comparing the corrected viewpoint images over the plurality of distances of interest, thereby specifying a piece of optimum calibration data from the plurality of pieces of calibration data;

a distance calculation step of calculating a subject distance based on the parallax; and

an output step of outputting, as a distance measurement result, the subject distance calculated from the parallax between the corrected viewpoint images with the piece of optimum calibration data.

97. The distance measuring method according to claim 96, further comprising

a correlation degree calculation step of calculating, for each of the distances of interest, a correlation degree of at least one of luminance and color between the plurality of corrected viewpoint images, wherein

the correlation degrees are compared over the plurality of distances of interest to specify a piece of calibration data with the correlation degree being maximum as the piece of optimum calibration data.

98. The distance measuring method according to claim 96, further comprising

a vertical deviation amount calculation step of calculating, for each of the distances of interest, a vertical deviation amount between corresponding pixels between the plurality of corrected viewpoint images, wherein

the vertical deviation amounts are compared over the plurality of distances of interest to specify a piece of calibration data with the vertical deviation amount being minimum as the piece of optimum calibration data.

99. The distance measuring method according to claim 96, further comprising

a parallax-calculated pixel count calculating step of calculating, for each of the distances of interest, a parallax-calculated pixel count that allows parallax calculation between the plurality of corrected viewpoint images, wherein

the parallax-calculated pixel counts are compared over the plurality of distances of interest to specify a piece of calibration data with the parallax-calculated pixel count being maximum as the piece of optimum calibration data.

100. The distance measuring method according to claim 96, further comprising

a local-noise pixel count calculation step of calculating, for each of the distances of interest, a local-noise pixel count that is locally present in a parallax distribution of the plurality of corrected viewpoint images, wherein

the local-noise pixel counts are compared over the plurality of distances of interest to specify a piece of calibration data with the local-noise pixel count being minimum as the piece of optimum calibration data.

103. The distance measuring method according to claim 96, wherein

a first distance measurement is performed while the distance of interest is being changed with a second change width larger than a first change width to obtain a first subject distance, a second distance measurement is performed while the distance of interest being changed with the first change width near the first subject distance to obtain a second subject distance, and the second subject distance is outputted as the distance measurement result.

104. The distance measuring method according to claim 103, wherein

a judgment criterion for the piece of optimum calibration data differs between the first distance measurement and the second distance measurement.

107. The distance measuring method according to claim 96, further comprising:

a step of determining a beginning distance of interest among a plurality of distances of interest;

a step of calculating, for each of the distances of interest, a correlation degree of at least one of luminance and color between the plurality of corrected viewpoint images; and

a step of determining the calculated subject distance as a next distance of interest, thereby switching the distance of interest, wherein

the correlation degrees are compared for each switching of the distance of interest and, when the correlation degree calculated after switching of the distance of interest is smaller than the correlation degree calculated before switching of the distance of interest, the subject distance calculated before switching of the distance of interest is adopted as the distance measurement result.

108. The distance measuring method according to claim 96, further comprising:

a step of determining a beginning distance of interest among a plurality of distances of interest;

a step of calculating, for each of the distances of interest, a vertical deviation amount between corresponding pixels between the plurality of corrected viewpoint images; and

a step of determining the calculated subject distance as a next distance of interest, thereby switching the distance of interest, wherein

the vertical deviation amounts are compared for each switching of the distance of interest and, when the vertical deviation amount calculated after switching of the distance of interest is larger than the vertical deviation amount calculated before switching of the distance of interest, the subject distance calculated before switching of the distance of interest is adopted as the distance measurement result.

109. The distance measuring method according to claim 96, further comprising:

a step of determining a beginning distance of interest among a plurality of distances of interest;

a step of calculating, for each of the distances of interest, a parallax-calculated pixel count that allows parallax calculation between the plurality of corrected viewpoint images; and

a step of determining the calculated subject distance as a next distance of interest, thereby switching the distance of interest, wherein

the parallax-calculated pixel counts are compared for each switching of the distance of interest and, when the parallax-calculated pixel count calculated after switching of the distance of interest is larger than the parallax-calculated pixel count calculated before switching of the distance of interest, the subject distance calculated before switching of the distance of interest is adopted as the distance measurement result.

110. The distance measuring method according to claim 96, further comprising:

a step of determining a beginning distance of interest among a plurality of distances of interest;

a step of calculating, for each of the distances of interest, a local-noise pixel count that is locally present in a parallax distribution of the plurality of corrected viewpoint images; and

a step of determining the calculated subject distance as a next distance of interest, thereby switching the distance of interest, wherein

the local-noise pixel counts are compared for each switching of the distance of interest and, when the local-noise pixel count calculated after switching of the distance of interest is larger than the local-noise pixel count calculated before switching of the distance of interest, the subject distance calculated before switching of the distance of interest is adopted as the distance measurement result.

112. The distance measuring method according to claim 96, wherein

a histogram indicating a relation between a parallax and a number of occurrences between the viewpoint images is created to determine a representative parallax based on the number of occurrences in the histogram, and the subject distance is calculated based on the representative parallax.

113. The distance measuring method according to claim 112, wherein

a most frequent value is detected in the histogram, and the most frequent value is determined as the representative parallax.

114. The distance measuring method according to claim 96, wherein

a histogram indicating a relation between a distance and a number of occurrences of each pixel of the viewpoint images is created, a representative distance is determined based on the number of occurrences in the histogram, and the representative distance is outputted as the distance measurement result.

115. The distance measuring method according to claim 114, wherein

a most frequent value is detected in the histogram, and the most frequent value is determined as the representative distance.

116. The distance measuring method according to claim 112, wherein

whether the number of occurrences is high or low is discriminated in the histogram, and a component of a pixel locally present is removed from the histogram.

117. The distance measuring method according to claim 112 wherein

a region of interest for which the number of occurrences is to be calculated is determined, and the number of occurrences is calculated restrictively to the region of interest to create the histogram.

118. The distance measuring method according to claim 117, wherein

a calculation region for use in exposure control or focusing control at the time of shooting the viewpoint images is determined as the region of interest.

119. The distance measuring method according to claim 117, wherein

the viewpoint images are divided into a plurality of divisional regions in an array, a frequency analysis for each of the divisional regions, and a divisional region having a high-frequency component as the region of interest.

120. The distance measuring method according to claim 117 wherein

a specific target extraction region where a specific target is extracted from the viewpoint images is determined as the region of interest.

121. The distance measuring method according to claim 112, wherein

a weight with respect to the number of occurrences is determined based on a position of each of the pixels in the viewpoint images, and the histogram is created by weighting based on the weight.

101. A distance measuring method comprising:

an image input step of inputting a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining step of obtaining, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting step of correcting, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation step of calculating, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a distance calculation step of calculating, for each of the distances of interest, a subject distance based on the parallax;

a comparing step of comparing differences each between the subject distance and the distance of interest over the plurality of distances of interest, thereby specifying a piece of optimum calibration data from the plurality of pieces of calibration data; and

an output step of outputting, as a distance measurement result, the subject distance calculated from the parallax between the corrected viewpoint images with the piece of optimum calibration data.

102. A distance measuring method comprising:

an image input step of inputting a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining step of obtaining, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

a first parallax calculation step of calculating, for each of the distances of interest, a first parallax based on the distance of interest;

an image correcting step of correcting, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a second parallax calculation step of calculating, for each of the distances of interest, a second parallax between the plurality of corrected viewpoint images;

a comparing step of comparing differences each between the first parallax and the second parallax over the plurality of distances of interest, thereby specifying a piece of optimum calibration data from the plurality of pieces of calibration data;

a distance calculation step of calculating a subject distance based on the second parallax of the corrected viewpoint image with the piece of optimum calibration data; and

an output step of outputting the calculated subject distance as a distance measurement result.

105. A distance measuring method comprising:

an image input step of inputting a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a calibration data obtaining step of obtaining, regarding each of a plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

a step of determining a beginning distance of interest among the plurality of distances of interest;

an image correcting step of correcting, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation step of calculating, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a distance calculation step of calculating, for each of the distances of interest, a subject distance based on the parallax;

a step of determining the calculated subject distance as a next distance of interest, thereby switching the distance of interest;

a comparing step of calculating, for each of the distances of interest, a difference between the subject distance and the distance of interest and comparing the differences before and after switching of the distance of interest, thereby selecting a distance measurement result from the plurality of calculated subject distances; and

an output step of outputting the distance measurement result.

111. The distance measuring method according to claim 105, wherein

a judgment criterion for the piece of optimum calibration data differs between a first distance measurement for determining the beginning distance of interest and obtaining an initial subject distance and a second distance measurement for determining the subject distance obtained in the first distance measurement as a next distance of interest and obtaining the subject distance as the distance measurement result.

106. A distance measuring method comprising:

an image input step of inputting a plurality of viewpoint images obtained by capturing a subject from different viewpoints by a plurality of image pickup devices;

a step of determining a beginning distance of interest among a plurality of distances of interest;

a calibration data obtaining step of obtaining, regarding each of the plurality of distances of interest, a piece of calibration data corresponding to each of the plurality of distances of interest from a plurality of pieces of calibration data for matching two-dimensional coordinates of the viewpoint images and three-dimensional coordinates in a real space;

an image correcting step of correcting, for each of the distances of interest, the plurality of viewpoint images based on each of the pieces of calibration data;

a parallax calculation step of calculating, for each of the distances of interest, a parallax between the plurality of corrected viewpoint images;

a distance calculation step of calculating, for each of the distances of interest, a subject distance based on the parallax;

a step of determining the calculated subject distance as a next distance of interest, thereby switching the distance of interest;

a comparing step of calculating, for each of the distances of interest, a difference between the subject distance and the distance of interest and comparing the difference and a threshold; and

an output step of outputting the calculated subject distance as a distance measurement result when the difference is smaller than the threshold.

122. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a same subject from different viewpoints by an image pickup apparatus capable of focus adjustment;

a calibration parameter storage unit configured to store calibration parameters corresponding to focal positions in the image pickup apparatus;

a distance estimating unit configured to obtain a piece of shooting mode information included in the viewpoint images or a piece of collateral information of the viewpoint images, and to estimate a subject distance from the image pickup apparatus to the subject, based on a relation between a piece of previously stored shooting mode information and a shooting distance;

a focal position selecting unit configured to select a focal position based on the subject distance estimated by the distance estimating unit;

an initial calibration data setting unit configured to set a piece of initial calibration data for use in initial distance measurement calculation for the subject from the calibration parameters stored in the calibration parameter storage unit based on the focal position selected by the focal position selecting unit; and

a distance measurement calculation unit configured to calculate a subject distance based on the piece of initial calibration data set by the initial calibration data setting unit and a parallax between the plurality of viewpoint images.

123. The distance measuring apparatus according to claim 122, further comprising:

a judging unit configured to evaluate the distance calculated by the distance measurement calculation unit and to judge success or failure of distance measurement calculation; and

a calibration parameter resetting unit configured to reset a calibration parameter applicable to the subject distance from the calibration parameters stored in the calibration parameter storage unit according to the judgment result of the judging unit.

124. The distance measuring apparatus according to claim 123, wherein

the distance estimating unit estimates the subject distance based on a piece of collateral information of the viewpoint images,

the initial calibration data setting unit sets the piece of initial calibration data for use in the initial distance measurement calculation for the subject based on the subject distance estimated by the distance estimating unit and the calibration parameters stored in the calibration parameter storage unit, and

the judging unit creates a piece of reliability information indicating reliability of the initial calibration data set by the piece of initial calibration data setting unit based on the result of the distance measurement calculation by the distance measurement calculation unit, and judges success or failure of the distance measurement calculation based on the reliability information.

125. The distance measuring apparatus according to claim 124, wherein

the judging unit selects a condition for judging success or failure of the distance measurement calculation according to the reliability information.

126. The distance measuring apparatus according to claim 123, wherein

the calibration parameter resetting unit extracts a pair of pieces of calibration data corresponding to top two focal positions adjacent to the focal position when the judging unit judges that the distance measuring calculation fails, interpolates the extracted pair of pieces of calibration data and prepares and resets a calibration parameter corresponding to a calibration parameter corresponding to the focal position.

127. The distance measuring apparatus according to claim 122, wherein

the distance estimating unit estimates the subject distance based on a piece of information about a shot scene obtained by analyzing the viewpoint images or a piece of shot-scene information included in the piece of collateral information.

128. The distance measuring apparatus according to claim 122, wherein

the distance estimating unit estimates the subject distance based on a piece of size information of the subject obtained by analyzing the viewpoint images or a piece of subject size information included in the piece of collateral information.

129. The distance measuring apparatus according to claim 128, wherein

the distance estimating unit estimates the subject distance based on a comparison between a size of a human subject included in the viewpoint images and a standard size of the human subject.

130. The distance measuring apparatus according to claim 122, wherein

the distance estimating unit estimates the subject distance based on a parallax in a focus evaluation value calculation region of the viewpoint images.

131. The distance measuring apparatus according to claim 122, wherein

the distance estimating unit estimates the subject distance based on a convergence distance of the image pickup apparatus.

132. The distance measuring apparatus according to claim 131, further comprising

a convergence distance storage unit configured to store a convergence distance for each model of the image pickup apparatus, wherein

the distance estimating unit estimates the subject distance based on a convergence distance corresponding to a model of the image pickup apparatus capturing the viewpoint images among the convergence distances stored in the convergence distance storage unit.

133. The distance measuring apparatus according to claim 131, further comprising

an external parameter obtaining unit configured to obtain an external parameter of the image pickup apparatus, wherein

the distance estimating unit calculates the convergence distance of the image pickup apparatus from the external parameter of the image pickup apparatus obtained by the external parameter obtaining unit, and estimates the calculated convergence distance as the subject distance.

135. The distance measuring apparatus according to claim 122, further comprising

a distance measurement calculation parameter input unit configured to receive an input of a parameter regarding distance measurement calculation, wherein

the distance estimating unit estimates the subject distance from the parameter inputted from the distance measurement calculation parameter input unit.

136. The distance measuring apparatus according to claim 122, wherein

the calibration parameter storage unit stores calibration parameters corresponding to respective focal points set in a discrete manner.

137. The distance measuring apparatus according to claim 122, wherein

the calibration parameter storage unit stores functions defining calibration parameters corresponding to respective focal points.

138. An image pickup apparatus comprising:

a focal position control unit configured to control a focal position of each of focus lenses of different image pickup optical systems; and

an image pickup unit capable of outputting a plurality of viewpoint images obtained by image pickup devices photoelectrically converting subject images formed via the image pickup optical systems to an image input unit included in the distance measuring apparatus according to claims 122.

140. A distance measuring system comprising:

the distance measuring apparatus according to claim 122

a focal position control unit configured to control a focal position of each of focus lenses of different image pickup optical systems; and

an image pickup unit capable of outputting a plurality of viewpoint images obtained by image pickup devices photoelectrically converting subject images formed via the image pickup optical systems to an image input unit of the distance measuring apparatus.

134. A distance measuring apparatus comprising:

an image input unit configured to input a plurality of viewpoint images obtained by capturing a same subject from different viewpoints by an image pickup apparatus capable of focus adjustment;

a calibration parameter storage unit configured to store calibration parameters corresponding to focal positions in the image pickup apparatus;

a distance estimating unit configured to estimate a subject distance from the image pickup apparatus to the subject, based on the viewpoint images or a piece of collateral information of the viewpoint images;

a focal position selecting unit configured to select a focal position based on the subject distance estimated by the distance estimating unit;

an initial calibration data setting unit configured to set a piece of initial calibration data for use in initial distance measurement calculation for the subject from the calibration parameters stored in the calibration parameter storage unit based on the focal position selected by the focal position selecting unit;

a distance measurement calculation unit configured to calculate a subject distance based on the piece of initial calibration data set by the initial calibration data setting unit and a parallax between the plurality of viewpoint images; and

a feature storage unit configured to store features of the viewpoint images, wherein

the distance estimating unit compares a feature of a previous viewpoint image stored in the feature storage unit and a feature of a current viewpoint image, and estimates a subject distance of the previous viewpoint image as the subject distance of the current viewpoint image only when the feature of the previous view point image stored in the feature storage unit and the feature of a current view point image are identical to each other.

139. A distance measuring method comprising the steps, to be executed by an information processing apparatus, of:

inputting a plurality of viewpoint images obtained by capturing a same subject from different viewpoints by an image pickup apparatus capable of focus adjustment;

storing calibration parameters corresponding to focal positions in the image pickup apparatus;

obtaining a piece of shooting mode information included in the viewpoint images or a piece of collateral information of the viewpoint images, and estimating a subject distance from the image pickup apparatus to the subject, based on a relation between a piece of previously stored shooting mode information and a shooting distance

selecting a focal position based on the estimated subject distance;

setting a piece of initial calibration data for use in initial distance measurement calculation for the subject from the stored calibration parameters based on the selected focal position; and

calculating a subject distance based on the piece of set initial calibration data and a parallax between the plurality of viewpoint images.

141. A distance measuring method comprising the steps, to be executed by an information processing apparatus, of:

inputting a plurality of viewpoint images obtained by capturing a same subject from different viewpoints by an image pickup apparatus capable of focus adjustment;

storing calibration parameters corresponding to focal positions in the image pickup apparatus;

storing features of the viewpoint images;

estimating a subject distance from the image pickup apparatus to the subject, based on the viewpoint images or a piece of collateral information of the viewpoint images, wherein a feature of a previous viewpoint image stored in the feature storage unit and a feature of a current viewpoint image are compared, and a subject distance of the previous viewpoint image is estimated as the subject distance of the current viewpoint image only when the feature of the previous view point image stored in the feature storage unit and the feature of a current view point image are identical to each other;

selecting a focal position based on the estimated subject distance;

setting a piece of initial calibration data for use in initial distance measurement calculation for the subject from the stored calibration parameters based on the selected focal position; and

calculating a subject distance based on the piece of set initial calibration data and a parallax between the plurality of viewpoint images.