MULTI-FRAME IMAGE CALIBRATOR

• US 20150124059A1
• Filed: 06/08/2012
• Published: 05/07/2015
• Est. Priority Date: 06/08/2012
• Status: Abandoned Application

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Gotchev, Atanas, Hannuksela, Miska, Georgiev, Mihail
• Application Number:
  US14/405,782
• Publication Number:
  US 20150124059A1
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 2013/0092 Image segmentation from stereoscopic image signals

Abstract

An apparatus comprising: an image analyser configured to analyse at least two images to determine at least one matched feature; a camera definer configured to determine at least two difference parameters between the at least two images; and a rectification determiner configured to determine values for the at least two difference parameters in an error search using an error criterion based on the at least one matched feature in the at least two images and an estimated difference parameter value, wherein the value for each difference parameter is determined serially.

Claims

1-27. -27. (canceled)

28. A method comprising:

analysing at least two images to determine at least one matched feature;

determining at least two difference parameters between the at least two images; and

determining values for the at least two difference parameters in an error search using an error criterion based on the at least one matched feature in the at least two images and an estimated difference parameter value, wherein the value for each difference parameter is determined serially.

29. The method as claimed in claim 28, wherein determining values for the at least two difference parameters in an error search comprises determining values for the at least two parameters to minimise the error search.

30. The method as claimed in claim 28, wherein analysing at least two images to determine at least one matched feature comprises:

determining at least one feature from a first image of the at least two images;

determining at least one feature from a second image of the at least two images; and

matching at least one feature from the first image and at least one feature from the second image to determine the at least one matched feature.

31. The method as claimed in claim 30, wherein analysing at least two images to determine at least one matched feature further comprises filtering the at least one matched feature.

32. The method as claimed in claim 31, wherein filtering the at least one matched feature comprises at least one of:

removing matched features occurring within a threshold distance of the image boundary;

removing repeated matched features;

removing distant matched features;

removing intersecting matched features;

removing non-consistent matched features; and

selecting a sub-set of the matches according to a determined matching criteria.

33. The method as claimed in claim 28, wherein determining at least two difference parameters between at least two images comprises:

determining from the at least two images a reference image;

defining for an image other than the reference image at least two difference parameters, wherein the at least two difference parameters are stereo setup misalignments.

34. The method as claimed in claim 28, wherein determining at least two difference parameters between at least two images comprises:

defining a range of values within which the difference parameter value can be determined in the error search; and

defining an initial value for the difference parameter value determination in the error search.

35. The method as claimed in claim 28, wherein determining values for the difference parameters in the error search comprises:

selecting a difference parameter, wherein the difference parameter has an associated defined initial value and value range;

generating a camera rectification dependent on the initial value of the difference parameter;

generating a value of the error criterion dependent on the camera rectification and at least one matched feature;

repeating selecting a further difference parameter value, generating a camera rectification and generating a value of the error criterion until a smallest value of the error criterion is found for the difference parameter; and

repeating selecting a further difference parameter until all of the at least two difference parameters have determined values for the difference parameters which minimise the error search.

36. The method as claimed in claim 28, further comprising:

generating a first image of the at least two images with a first camera; and

generating a second image of the at least two images with a second camera.

37. The method as claimed in claim 28, further comprising:

generating a first image of the at least two images with a first camera at a first position; and

generating a second image of the at least two images with the first camera at a second position displaced from the first position.

38. The method as claimed in claim 28, wherein the error criterion comprises at least one of:

a Sampson distance metric;

a symmetric epipolar distance metric;

a vertical feature shift metric;

a left-to-right consistency metric;

a mutual area metric; and

a projective distortion metric.

39. The method as claimed in claim 28, wherein the difference parameter comprises at least one of:

a rotation shift;

a Rotation Shift Pitch;

a Rotation Shift Roll;

a Rotation Shift Yaw;

a translational shift;

a translational shift on the Vertical (Y) Axis;

a translation shift on the Depth (Z) Axis;

a horizontal focal length difference;

a vertical focal length difference;

an optical distortion in the optical system;

a difference in zoom factor;

a non-rigid affine distortion;

a Horizontal Axis (X) Shear;

a Vertical Axis (Y) Shear; and

a Depth (Z) Axis Shear.

40. An apparatus comprising at least one processor and at least one memory including computer code for one or more programs, the at least one memory and the computer code configured to with the at least one processor cause the apparatus to at least to:

analyse at least two images to determine at least one matched feature;

determine at least two difference parameters between the at least two images; and

determine values for the at least two difference parameters in an error search using an error criterion based on the at least one matched feature in the at least two images and an estimated difference parameter value, wherein the value for each difference parameter is determined serially.

41. The apparatus as claimed in claim 40, wherein the apparatus caused to determine values for the at least two difference parameters in an error search is further caused to determine values for the at least two parameters to minimise the error search.

42. The apparatus as claimed in claim 40, wherein the apparatus caused to analyse at least two images to determine at least one matched feature is further caused to:

determine at least one feature from a first image of the at least two images;

determine at least one feature from a second image of the at least two images; and

match at least one feature from the first image and at least one feature from the second image to determine the at least one matched feature.

43. The apparatus as claimed in claim 42, wherein the apparatus caused to analyse at least two images to determine at least one matched feature is further caused to filter the at least one matched feature.

44. The apparatus as claimed in claim 43, wherein the apparatus caused to filter the at least one matched feature is further caused to at least one of:

remove matched features occurring within a threshold distance of the image boundary;

remove repeated matched features;

remove distant matched features;

remove intersecting matched features;

remove non-consistent matched features; and

select a sub-set of the matches according to a determined matching criteria.

45. The apparatus as claimed in claim 40, wherein the apparatus caused to determine at least two difference parameters between at least two images is further caused to:

determine from the at least two images a reference image; and

define for an image other than the reference image at least two difference parameters, wherein the at least two difference parameters are stereo setup misalignments.

46. The apparatus as claimed in claim 40, wherein the apparatus caused to determine at least two difference parameters between at least two images is further caused to:

define a range of values within which the difference parameter value can be determined in the error search; and

define an initial value for the difference parameter value determination in the error search.

47. The apparatus as claimed in claim 40, wherein the apparatus caused to determine values for the difference parameters in the error search is further caused to:

select a difference parameter, wherein the difference parameter has an associated defined initial value and value range;

generate a camera rectification dependent on the initial value of the difference parameter;

generate a value of the error criterion dependent on the camera rectification and at least one matched feature;

repeat selecting a further difference parameter value, generating a camera rectification and generating a value of the error criterion until a smallest value of the error criterion is found for the difference parameter; and

repeat selecting a further difference parameter until all of the at least two difference parameters have determined values for the difference parameters which minimise the error search.

48. The apparatus as claimed in claim 40, wherein the apparatus is further caused to:

generate a first image of the at least two images with a first camera; and

generate a second image of the at least two images with a second camera.

49. The apparatus as claimed in claim 40, wherein the apparatus is further caused to:

generate a first image of the at least two images with a first camera at a first position; and

generate a second image of the at least two images with the first camera at a second position displaced from the first position.

50. The apparatus as claimed in claim 40, wherein the error criterion comprises at least one of:

a Sampson distance metric;

a symmetric epipolar distance metric;

a vertical feature shift metric;

a left-to-right consistency metric;

a mutual area metric; and

a projective distortion metric.

51. The apparatus as claimed in claim 40, wherein the difference parameter comprises at least one of:

a rotation shift;

a Rotation Shift Pitch;

a Rotation Shift Roll;

a Rotation Shift Yaw;

a translational shift;

a translational shift on the Vertical (Y) Axis;

a translation shift on the Depth (Z) Axis;

a horizontal focal length difference;

a vertical focal length difference;

an optical distortion in the optical system;

a difference in zoom factor;

a non-rigid affine distortion;

a Horizontal Axis (X) Shear;

a Vertical Axis (Y) Shear; and

a Depth (Z) Axis Shear.

52. A computer program product comprising a non-transitory computer readable medium having program code portions stored thereon, the program code portions configured upon execution to:

analyse at least two images to determine at least one matched feature;

determine at least two difference parameters between the at least two images; and

determine values for the at least two difference parameters in an error search using an error criterion based on the at least one matched feature in the at least two images and an estimated difference parameter value, wherein the value for each difference parameter is determined serially.