System and method for processing stereoscopic vehicle information

• US 9,449,378 B2
• Filed: 07/20/2012
• Issued: 09/20/2016
• Est. Priority Date: 05/22/2008
• Status: Active Grant

• Current Assignee: Matrix Electronic Measuring Properties, LLC
• Original Assignee: Matrix Electronic Measuring Properties, LLC
• Inventors: Srack, Thomas L., Anzalone, Mark A., Stevens, George B., Srack, Robert W., Srack, Janet M., Bailey, Robert D.
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US13/554,939
• Publication Number:
  US 20130188018A1
• Time in Patent Office: 1,523 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

A stereoscopic measurement system determines relative location of a point on an object based on a stereo image pair of the object. The system comprises an image capture device for capturing a stereo image pair of the object, the image pair comprising a first image and a second image of the object. The system comprises a processing system configurable to designate a first point and a second point on the first image, designate the first point and the second point on the second image, define stereo points based on the designated points, and to calculate a distance between the stereo points.

Claims

1. A system for determining the wheel alignment for a vehicle, the system comprising:

an object, wherein the object is a part of the vehicle,

an image capture device for capturing a first image of the object and a second image of the object;

a stereo image pair comprising the first and second images;

a processing system comprising a memory, wherein the processing system is configured to;

store the stereo image pair in the memory;

designate a first measurement point in the first image;

designate a second measurement point in the first image;

designate the first measurement point in the second image;

designate the second measurement point in the second image;

define a first stereo point of the stereo image pair based on the first measurement point;

define a second stereo point of the stereo image pair based on the second measurement point; and

calculate the distance between the first stereo point and the second stereo point;

a test portion of the object;

a reference portion of the object, the reference portion corresponding to the test portion;

a second stereo image pair comprising a first reference image of the object and a second reference image of the object; and

wherein the stereo image pair comprises images of the test portion of the object, and the second stereo image pair comprises images of the reference portion of the object.

2. The system of claim 1 wherein the processing system is further configured to receive user inputs to designate the first measurement point and the second measurement point in the first image.

3. The system of claim 2 wherein the processing system is further configured to receive user inputs to designate the first measurement point and the second measurement point in the second image.

4. The system of claim 1 wherein the processing system is further configured to receive:

a first user input designating the first measurement point in the first image;

then,

a second user input designating the first measurement point in the second image;

then,

a third user input designating the second measurement point in the first image;

then,

a fourth user input designating the second measurement point in the second image.

5. The system of claim 1 wherein the first and second measurement points are located on the test portion, the system further comprising:

a reference portion of the object, the reference portion corresponding to the test portion;

wherein the processing system is further configured to;

designate the first measurement point and the second measurement point on the reference portion;

calculate the distance between the first measurement point on the test portion and the first measurement point on the reference portion; and

calculate the distance between the second measurement point on the test portion and the second measurement point on the reference portion.

6. The system of claim 5 wherein the test portion is on an opposite side of the vehicle from the reference portion.

7. The system of claim 1 wherein a first side of the object comprises the reference portion and a second side of the object comprises the test portion.

8. The system of claim 1 wherein the processing system is further configured to define a reference plane, the plane separating the reference portion from the test portion.

9. The system of claim 8 wherein the reference plane is a central reference plane.

10. The system of claim 8 further comprising:

a damaged portion of the object located on the test portion;

a first reference point on the reference portion of the object;

a second reference point on the test portion of the object, wherein the second reference point is not located on the damaged portion, and the second reference point corresponds to the first reference point; and

wherein the distance between any given point on the reference plane and the first reference point is equal to the distance between the given point on the reference plane and the second reference point.

26. The system of claim 1 wherein the object is selected from a group consisting of:

1) a wheel and suspension subsystem of the vehicle,

     2) an engine cradle of the vehicle,

     3) a lower control arm of the vehicle,

     4) a unibody structure of the vehicle,

     5) lower rails of the vehicle,

     6) rear toe of the vehicle,

     7) rear thrust angle of the vehicle, and

     8) rear cradle of the vehicle.

27. The system of claim 1 wherein the first measurement point is selected from a group consisting of the vehicle'"'"'s rocker panel, spindle, rotor, knuckle, and strut.

28. The system of claim 7 wherein the processing system is further configured to:

designate a first reference measurement point in the first reference image, the first reference measurement point corresponding to the first measurement point;

designate a second reference measurement point in the first reference image, the second reference measurement point corresponding to the second measurement point;

designate the first reference measurement point in the second reference image;

designate the second reference measurement point in the second reference image;

define a first reference stereo point corresponding to the first reference measurement point;

define a second reference stereo point corresponding to the second reference measurement point;

define a reference plane separating the first side of the object from the second side of the object; and

calculate symmetry deviations between the first and second stereo points and the first and second reference stereo points as a function of the defined reference plane.

29. The system of claim 1 wherein the processing system is further configured to create a customized report comprising the calculated distance between the first stereo point and the second stereo point.

30. The system of claim 1 wherein the processing system is further configured to:

compare the stereo points with known reference points for the object; and

determine the measurements between the stereo points and the known reference points.

31. The system of claim 29 wherein the processing system is further configured to:

define a reference stereo point on the object;

calculate a second distance between the first stereo point and the reference stereo point;

calculate a third distance between the second stereo point and the reference stereo point;

store reference stereo point data corresponding to the reference stereo point on the object; and

wherein the customized report further comprises the second calculated distance and the third calculated distance.

32. The system of claim 31 wherein the processing system is further configured to:

define a Cartesian reference coordinate space based on the first stereo point, the second stereo point, and the reference point;

define an origin for the reference coordinate space; and

wherein the customized report further comprises the first stereo point, the second stereo point, and the reference stereo point, wherein each calculated distance is reported relative to the reference coordinate space.

33. The system of claim 32 wherein the processing system is further configured to define the reference coordinate space independently of the object.

34. The system of claim 33 wherein the processing system is further configured to define the origin of the reference coordinate space independently of the object.

35. The system of claim 32 wherein:

at least one of the points is located on a vehicle; and

the defined reference coordinate space is selected from the group consisting of;

     1) center line,

     2) opposite side,

     3) under hood,

     4) inner body,

     5) under body, and

     6) rear body.

36. The system of claim 33 wherein the processing system is further configured to designate an origin not located on the object.

37. The system of claim 32 further comprising a damaged portion of the object, wherein the processing system is further configured to define an origin of the reference coordinate space wherein the origin is proximal to the damaged portion.

38. The system of claim 37 further comprising a first side of the object and a second side of the object, the first side comprising an undamaged portion and the second side comprising the damaged portion, wherein:

the second side is opposite to the first side;

the first stereo point is on the first side;

the second stereo point is on the second side, and is not on the damaged portion;

the first stereo point corresponds to the second stereo point;

the reference stereo point is on the first side;

the processing system is further configured to;

define a third stereo point on the damaged portion, wherein the third stereo point corresponds to the reference point; and

determine a deviation of the third stereo point.

39. The system of claim 1 wherein the processing system is further configured to generate a list of stereo image pairs for display, and to generate the first and second images of a selected one of the list of stereo image pairs for display.

40. The system of claim 1 wherein the processing system is further configured to generate a first selection assist line in the second image based the first measurement point, and to generate a second selection assist line in the second image based on the second measurement point.

41. The system of claim 40 wherein the processing system is further configured to identify a range of points in the second image based on the first measurement point to generate the first selection assist line, and to identify another range of points in the second image based on the second measurement point to generate the second selection assist line.

42. The system of claim 40 wherein the processing system is further configured to designate the first measurement point along the first selection assist line in the second image.

43. The system of claim 42 wherein the processing system is further configured to designate the second measurement point along the second selection assist line in the second image.

44. The system of claim 1 wherein the image capture device comprises a first camera to capture the first image and a second camera to capture the second image.

45. The system of claim 44 wherein a calibration pattern is captured together with each of the first image and the second image.

46. The system of claim 44 wherein the processing system is further configured to store download history data for the stereo image pair in the memory, the download history data comprising metadata and intrinsic calibration data for each of the first and second cameras and a time and date that the first and second images of the stereo image pair were received from the image capture device.

47. The system of claim 44 further comprising a monitor to display the first and the second images of the object, wherein the image capture device further comprises a switch to control the first camera and second camera to capture the first and second images of the object simultaneously.

50. The system of claim 1 wherein the image capture device transmits the first and second images via a communication link selected from a group consisting of a wired connection and a wireless communication link.

11. A system for determining the wheel alignment for a vehicle, the system comprising:

an object, wherein the object is a part of the vehicle,

an image capture device for capturing a first image of the object and a second image of the object;

a stereo image pair comprising the first and second images;

a processing system comprising a memory, wherein the processing system is configured to;

store the stereo image pair in the memory;

designate a first measurement point in the first image;

designate a second measurement point in the first image;

designate the first measurement point in the second image;

designate the second measurement point in the second image;

define a first stereo point of the stereo image pair based on the first measurement point;

define a second stereo point of the stereo image pair based on the second measurement point; and

calculate the distance between the first stereo point and the second stereo point;

wherein the object is damaged, the system further comprising;

an object, wherein the object is a part of the vehicle,

an image capture device for capturing a first image of the object and a second image of the object;

a stereo image pair comprising the first and second images;

a processing system comprising a memory, wherein the processing system is configured to;

store the stereo image pair in the memory;

designate a first measurement point in the first image;

designate a second measurement point in the first image;

designate the first measurement point in the second image;

designate the second measurement point in the second image;

define a first stereo point of the stereo image pair based on the first measurement point; and

define a second stereo point of the stereo image pair based on the second measurement point; and

calculate the distance between the first stereo point and the second stereo point;

a second object, wherein the second object corresponds to the object, and is undamaged;

a first image of the second object;

a second image of the second object; and

a second stereo image pair comprising the first and second images of the second object.

12. The system of claim 11 wherein the second object is part of the vehicle.

13. The system of claim 11 wherein the second object is part of another vehicle.

14. The system of claim 11 wherein the processing system is further configured to:

designate a third measurement point in the first image of the object;

designate the third measurement point in the second image of the object;

define a third stereo point of the stereo image pair based on the third measurement point;

designate the first measurement point, the second measurement point, and the third measurement point in the first image of the second object;

designate the first measurement point, the second measurement point, and the third measurement point in the second image of the second object; and

define three stereo points of the second stereo image pair based on the three designated measurement points of the second object.

15. The system of claim 14 wherein the processing system is further configured to:

define reference planes based on the three defined stereo points of the stereo image pair; and

define reference planes based on the three defined stereo points of the second stereo image pair.

16. The system of claim 15 wherein:

the three defined stereo points of the stereo image pair are reference stereo points; and

the three defined stereo points of the second stereo image pair are reference stereo points.

17. The system of claim 14 wherein the processing system is further configured to:

define a reference plane of the stereo image pair based on the three defined stereo points of the stereo image pair;

define a reference plane of the second stereo image pair based on the three defined stereo points of the second stereo image pair;

designate a fourth measurement point in the first image of the object;

designate the fourth measurement point in the second image of the object;

designate the fourth measurement point in the first image of the second object;

designate the fourth measurement point in the second image of the second object; and

wherein the fourth measurement point of the object is a test point and the fourth measurement point of the second object is a reference point.

18. The system of claim 17 wherein the processing system is further configured to determine a deviation between the test point and the reference point based on the reference planes.

19. The system of claim 17 wherein the reference planes are substantially parallel to the vehicle'"'"'s direction of travel, and are at substantially right angles with the ground.

20. The system of claim 17 wherein the reference planes are substantially parallel to the vehicle'"'"'s direction of travel, and are substantially parallel to the ground.

21. The system of claim 17 wherein the reference planes are at substantially right angles with the vehicle'"'"'s direction of travel, and are at substantially right angles to the ground.

22. The system of claim 17 wherein the processing system is further configured to:

define a second reference plane of the stereo image pair based on the three defined stereo points of the stereo image pair;

define a second reference plane of the second stereo image pair based on the three defined stereo points of the second stereo image pair;

define a third reference plane of the stereo image pair based on the three defined stereo points of the stereo image pair;

define a third reference plane of the second stereo image pair based on the three defined stereo points of the second stereo image pair;

wherein the three reference planes of the stereo image pair are orthogonal to each other; and

wherein the three reference planes of the second stereo image pair are orthogonal to each other.

23. The system of claim 11 wherein the processing system is further configured to:

designate the first measurement point in the first image of the second object;

designate the second measurement point in the first image of the second object;

designate the first measurement point in the second image of the second object;

designate the second measurement point in the second image of the second object;

define a third stereo point of the second stereo pair based on the designated first measurement point;

define a fourth stereo point of the second stereo pair based on the designated second measurement point; and

calculate the distance between the third stereo point and the fourth stereo point.

24. The system of claim 11 wherein the second object is on a side of the vehicle opposite to the object.

25. The system of claim 11 wherein the second object is considered to be a reflection of the object.

48. A system for determining the wheel alignment for a vehicle, the system comprising:

an object, wherein the object is a part of the vehicle,

an image capture device for capturing a first image of the object and a second image of the object;

a stereo image pair comprising the first and second images;

a processing system comprising a memory, wherein the processing system is configured to;

store the stereo image pair in the memory;

designate a first measurement point in the first image;

designate a second measurement point in the first image;

designate the first measurement point in the second image;

designate the second measurement point in the second image;

define a first stereo point of the stereo image pair based on the first measurement point;

define a second stereo point of the stereo image pair based on the second measurement point; and

calculate the distance between the first stereo point and the second stereo point;

wherein the image capture device comprises a first camera to capture the first image and a second camera to capture the second image;

wherein the processing system is further configured to;

process the stereo image pair stored in the memory to determine if the first and second images of the object are images of a calibration pattern; and

determine stereo calibration data for the image capture device when the first and second images of the object are of the calibration pattern, the stereo calibration data comprising location information for the first camera relative to the second camera in a Cartesian coordinate space of the image capture device, and wherein the processing system is configured to store the stereo calibration data in the memory.

49. The system of claim 48 wherein the processing system is further configured to:

associate the stereo calibration data with the stereo image pair based on the download history data for the stereo image pair when the first and second images of the stereo image pair are not of the calibration pattern.

51. A system for determining a deviation from a desired location of an object, the system comprising:

an image capture device to capture simultaneously a first image of the object, and a second image of the object, and a calibration pattern;

a processing system comprising a memory, wherein the processing system is configured to;

store in the memory a stereo image pair comprising the first and second images;

designate a plurality of measurement points in the first image;

designate the plurality of measurement points in the second image; and

define a plurality of stereo points based on the plurality of measurement points;

wherein the object further comprises a damaged portion;

wherein the image capture device further captures a second pair of first and second images of the object, the second pair of images comprising images of a corresponding undamaged portion of the object; and

wherein the undamaged portion of the object is considered to be a reflection across a plane separating the undamaged portion from the damaged portion of the object.

52. The system of claim 51 further comprising at least one reference stereo point on the object, wherein the processing system is further configured to store reference stereo point data corresponding to the at least one reference stereo point.

53. The system of claim 52 wherein the processing system is further configured to identify the deviation based on the plurality of defined stereo points and the reference stereo point data.

54. The system of claim 51 wherein the object is a part of a vehicle.

55. The system of claim 54 wherein the object is a component of a subsystem of the vehicle, the subsystem selected from a group consisting of the vehicle'"'"'s 1) suspension subsystem, 2) steering subsystem, and 3) wheel alignment subsystem.

56. The system of claim 54 wherein the object is selected from a group consisting of:

1) a wheel and suspension subsystem of the vehicle,

     2) an engine cradle of the vehicle,

     3) a lower control arm of the vehicle,

     4) a unibody structure of the vehicle,

     5) lower rails of the vehicle,

     6) rear toe of the vehicle,

     7) rear thrust angle of the vehicle, and

     8) rear cradle of the vehicle.

57. The system of claim 54 wherein the plurality of measurement points is located on objects selected from the group consisting of the vehicle'"'"'s rocker panel, spindle, rotor, knuckle, and strut.

60. The system of claim 51 wherein the processing system is further configured to:

identify a range of points in the second image based on at least one of the plurality of measurement points to generate a first selection assist line in the first image; and

identify another range of points in the second image based on at least another of the plurality of measurement points to generate a second selection assist line in the second image.

61. The system of claim 51 wherein the processing system is further configured to:

calculate a distance between any two defined stereo points; and

create a customized report comprising the calculated distance.

62. The system of claim 61 wherein the processing system is further configured to:

store reference stereo point data corresponding to a reference stereo point on the object, wherein the customized report further comprises at least one calculated distance between at least one of the stereo points and the reference stereo point.

58. A system for determining a deviation from a desired location of an object, the system comprising:

an image capture device to capture simultaneously a first image of the object, and a second image of the object, and a calibration pattern;

a processing system comprising a memory, wherein the processing system is configured to;

store in the memory a stereo image pair comprising the first and second images;

designate a plurality of measurement points in the first image;

designate the plurality of measurement points in the second image; and

define a plurality of stereo points based on the plurality of measurement points;

wherein the object further comprises a damaged portion; and

wherein the image capture device further captures a second stereo pair, the second stereo pair comprising a first reference image and a second reference image of a corresponding undamaged portion of a second object, the second object corresponding to the object.

59. The system of claim 58 wherein the second object is considered to be a reflection across a plane separating the second object from the object.

63. A method for determining the relative position of a point on an object, the method comprising:

capturing a first image of the object, and a second image of the object;

storing first and second images as a stereo image pair in a memory of a processing system;

designating a first measurement point in the first image;

designating a second measurement point in the first image;

designating the first measurement point in the second image;

designating the second measurement point in the second image;

defining a first stereo point that corresponds to the first measurement point;

defining a second stereo point that corresponds to the second measurement;

calculating the distance between the first stereo point and the second stereo point;

wherein the first and second images comprise images of a damaged portion of the object;

capturing a second pair of first and second images of the object at the image capture device, wherein the second pair of images comprise images of an undamaged portion of the object;

defining a central plane separating the undamaged portion from the damaged portion of the object into a first side and a second side of the object, wherein the undamaged portion is located in the first side, and the damaged portion is located in the second side;

designating a first reference point on the first side of the object;

designating a second reference point on the second half of the object, wherein the second reference point is not located on the damaged portion, and the second reference point corresponds to the first reference point; and

wherein the distance between any point on the defined plane and the first reference point is equal to the distance between the point on the defined plane and the second reference point.

64. The method of claim 63 further comprising:

comparing the stereo points with reference stereo points of the object; and

determining the measurements between the stereo points and the reference stereo points.

65. A method for determining the relative position of a point on an object, the method comprising:

capturing a first image of the object, and a second image of the object;

storing first and second images as a stereo image pair in a memory of a processing system;

designating a first measurement point in the first image;

designating a second measurement point in the first image;

designating the first measurement point in the second image;

designating the second measurement point in the second image;

defining a first stereo point that corresponds to the first measurement point;

defining a second stereo point that corresponds to the second measurement;

calculating the distance between the first stereo point and the second stereo point;

wherein the first and second images comprise images of a damaged portion of the object; and

capturing a reference pair of images, the reference pair comprising a first reference image and second reference image of a second object at the image capture device, wherein the reference pair of images comprise an undamaged portion of the second object, the undamaged portion corresponding to the damaged portion of the object.

66. The method of claim 65 wherein the second object is considered to be a reflection across a plane separating the second object from the object.