Picture coincidence detecting apparatus and method

• US 5,754,692 A
• Filed: 04/20/1995
• Issued: 05/19/1998
• Est. Priority Date: 07/31/1993
• Status: Expired due to Term

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Kondo, Tetsujiro, Horishi, Takashi
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): KELLEY, CHRISTOPHER S
• Application Number:
  US08/411,771
• Time in Patent Office: 1,125 Days
• Field of Search: 382/216, 382/227, 382/278, 382/209
• US Class Current: 382/216
• CPC Class Codes: G06T 7/238 using non-full search, e.g. three-step search

Abstract

Lower accuracy picture coincidence detecting means gives positional relationship between picture data of higher resolution picture and lower resolution picture as corresponding pixel position predicted with marker position as reference, i.e., initial position. Block matching processing means sets a search range corresponding to a block of an arbitrary range set within the lower resolution picture in accordance with the initial position to globally search, within the search range, block on the higher resolution picture corresponding to the block of the arbitrary range to find out a candidate of coincidence area to further interpolate between respective pixels of lower resolution picture and higher resolution picture to thereby evaluate, with high accuracy, similarity between the set block and block within the search range to detect a picture in which coincidence is obtained. For this reason, even if corresponding pictures are pictures such that both pixel positions differ from each other in dependency upon pictures of different formats, e.g., different picture angles, it is possible to detect coincidence between pictures or pixels.

Claims

1. A picture coincidence detecting apparatus adapted for detecting coincidence relationship between respective pixels of a first picture that a first image pick-up element comprised of a first number of pixels outputs and a second picture that a second image pick-up element comprised of a second number of pixels outputs,

the apparatus comprising:

lower accuracy picture coincidence detecting means for carrying out lower accuracy pixel matching on the basis of reference positions of the first and second pictures; and

block matching processing means for selectively carrying out block matching processing in pixel or subpixel units to set, on the second picture, a search range corresponding to a block on the first picture in dependency upon a detection result of the lower accuracy picture coincidence detecting means to search a block on the second picture, which matches with the block on the first picture, wherein when the first picture is assumed to be a lower resolution picture and the second picture is assumed to be a higher resolution picture, the block matching processing means sets, on the higher resolution picture. a search range corresponding to the block on the lower resolution picture in dependency upon a detection result of the lower accuracy picture coincidence detecting means to globally search, within the search range, a block on the higher resolution picture, which matches with the block on the lower resolution picture to find a candidate to further interpolate between respective pixels of the lower resolution picture and the higher resolution picture to thereby evaluate similarity between the set block and the block within the search range to detect a picture in which coincidence is established.

6. A picture coincidence detecting apparatus as set forth in claim 1, wherein the block matching processing means generates, by interpolation, data between respective pixels of the first and second pictures thereafter to carry out the block matching processing.

7. A picture coincidence detecting apparatus as set forth in claim 1, wherein the first and second image pick-up elements image the same object.

8. A picture coincidence detecting apparatus as set forth in claim 7, wherein a marker indicating a pixel position of reference is attached to the object.

2. A picture coincidence detecting apparatus adapted for detecting coincidence relationship between respective pixels of a first picture that a first image pick-up element comprised of a first number of pixels outputs and a second picture that a second image pick-up element comprised of a second number of pixels outputs,

the apparatus comprising:

lower accuracy picture coincidence detecting means for carrying out lower accuracy pixel matching on the basis of reference positions of the first and second pictures; and

block matching processing means for globally searching, within a search range set on the second picture, the first and second pictures in dependency upon a detection result of the lower accuracy picture coincidence detecting means to select respective points of remark to set corresponding blocks including these points of remark in the first and second pictures to make a correction such that mean values or standard deviations of these luminance signals are in correspondence with each other thereafter to selectively implement block matching processing in pixel or subpixel units to the corresponding blocks, wherein the block matching processing means repeats, n number of times, while hierarchically reducing n number of times the corresponding block, detection and correction of a difference between mean levels of luminance signals or the detection and correction of a ratio between standard deviations of luminance signals, and block matching processing.

9. A picture coincidence detecting apparatus as set forth in claim 2, wherein the block matching processing means generates, by interpolation, data between respective pixels of the first and second pictures thereafter to carry out the block matching processing.

10. A picture coincidence detecting apparatus as set forth in claim 2, wherein the first and second image pick-up elements image the same object.

11. A picture coincidence detecting apparatus as set forth in claim 10, wherein a marker indicating a pixel position of reference is attached to the object.

3. A picture coincidence detecting method of detecting coincidence between respective still pictures imaged by cameras having different resolutions, said method comprising the steps of:

setting a block of a predetermined size within a lower resolution picture,

setting a search range corresponding to the block on a higher resolution picture,

globally searching the search range to find a candidate, and

block coincidence detecting by interpolating between respective pixels of the higher resolution picture and the lower resolution picture to selectively evaluate in pixel or subpixel units, with high accuracy, similarity between the set block and a block within the search range to detect a picture in which coincidence is established, said block coincidence detecting step including the step of allowing mean values or standard deviations of luminance signal levels of a higher resolution picture and a lower resolution picture to coincide with each other over the entire picture prior to carrying out the picture coincidence detecting processing to allow the mean values or standard deviations of luminance signal levels to coincide with each other in local corresponding block units for a second time in carrying out coincidence detection of block unit.

12. A picture coincidence detecting method as set forth in claim 3, wherein, in the block coincidence detecting step, as a method of evaluating similarity between blocks of higher resolution picture and lower resolution picture, correlative coefficients between blocks are used to carry out evaluation.

13. A picture coincidence detecting method as set forth in claim 3, wherein, in the block coincidence detecting step, as a method of evaluating similarity between blocks of higher resolution picture and lower resolution picture, absolute value sum of differences between blocks is used to carry out evaluation.

14. A picture coincidence detecting method as set forth in claim 3, wherein, in the block coincidence detecting step, as a method of evaluating similarity between blocks of higher resolution picture and lower resolution picture, square sum of differences between blocks is used to carry out evaluation.

4. A picture coincidence detecting method of detecting coincidence between respective still pictures imaged by cameras having different resolutions, the method comprising the steps of:

setting a block of a predetermined size within a lower resolution picture;

setting a search range corresponding to the block on the higher resolution picture;

globally searching the search range to find a candidate; and

block coincidence detecting by setting a corresponding block including a candidate in pictures of lower resolution and higher resolution to make a correction such that mean levels or standard deviations of luminance signals are in correspondence with each other thereafter to selectively evaluate in pixel or subpixel units, with high accuracy, similarity between the corresponding blocks to detect a picture in which coincidence is established wherein the block coincidence detecting step repeats, n number of times, detection and correction of a difference between mean values of luminance signals or detection and correction of a ratio between standard deviations of luminance signals, and block matching processing the corresponding block.

15. A picture coincidence detecting method as set forth in claim 4, wherein the block coincidence detecting step mathematically judges evaluation values in matching every rank to correct a matching candidate pixel.

16. A picture coincidence detecting method as set forth in claim 4, wherein the block coincidence detecting step adjusts block size and search area in matching so that an evaluation value of higher rank corresponding to an evaluation object pixel position of lower rank exists when evaluation value of higher rank is applied to evaluation value of lower rank.

5. A picture coincidence detecting method of detecting coincidence between respective still pictures imaged by cameras having different resolutions, the method comprising the steps of:

setting a block of a predetermined size within a lower resolution picture;

setting a search range corresponding to the block on a higher resolution picture;

globally searching the search range to find a candidate; and

block coincidence detecting by respectively setting corresponding blocks including the candidate to determine a difference between mean values or a ratio between standard deviations of luminance signals of a block within the lower resolution picture and a block within the higher resolution picture while allowing the corresponding blocks to be greater in a stepwise manner to successively selectively implement matching processing in pixel or subpixel units from larger blocks to smaller blocks of hierarchical plural blocks by using the difference or the ratio when the determined difference or ratio converges to detect a picture in which coincidences established wherein the block coincidence detecting step generates, by interpolation data between respective pixels of the first and second pictures to respectively set corresponding blocks including the candidate.