Method and apparatus for lifting objects

• US 6,466,843 B1
• Filed: 10/16/2001
• Issued: 10/15/2002
• Est. Priority Date: 10/16/2001
• Status: Expired due to Fees

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Madiwale, Appasaheb Ningappa, De Bedout, Juan Manuel, Harding, Kevin George, Bonanni, Pierino Gianni
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): Marc, McDieunel, Marc, McDieunel
• Application Number:
  US09/682,763
• Time in Patent Office: 364 Days
• Field of Search: 700/165, 700/145, 700/97, 700/245, 700/206, 700/247, 700/260, 700/103, 700/251, 700/252, 700/255, 700/259, 318/568.11, 901/8, 72/162, 72/167, 72/173, 724/22, 724/24, 724/46, 724/72, 724/811, 738/620.43, 701/301
• US Class Current: 700/245
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 2219/40292 Manipulator is positioned by a crane to cover a large workpiece, extended range
• CPC Class Codes: G05B 2219/40609 Camera to monitor end effector as well as object to be handled

Abstract

In one embodiment of the present invention, a system for lifting objects comprises a Cartesian manipulator, a machine vision system for acquiring video images of the objects, and a control processor for processing the video images and providing position command signals to the Cartesian manipulator.

Claims

1. A system for lifting objects comprising:

a robotic manipulator;

a distance sensor adapted to acquire object height measurements of said objects;

a light plane projector adapted to illuminate said objects;

a video camera adapted to acquire video images of said objects; and

a control processor adapted to process said video images and said object height measurements and provide control signals to said robotic manipulator by implementing a polygonal object location method to provide position command signals to said Cartesian manipulator,

said polygonal object location method comprising;

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

2. The system of claim 1 wherein:

said video camera comprises an optical bandpass filter having a wavelength pass band; and

said structured light projector comprises a monochromatic light source having a wavelength within said wavelength pass band.

3. The system of claim 2, wherein said wavelength pass band begins at about 520 nanometers and ends at about 540 nanometers.

4. The system of claim 2 wherein said monochromatic light source comprises a laser.

5. A system for lifting objects comprising:

a gantry crane;

a distance sensor disposed on a trolley of said gantry crane;

a light plane projector disposed on said trolley of said gantry crane;

a video camera disposed on a bridge of said gantry crane; and

a control processor adapted to process video images from said video camera and object height measurements from said distance sensor and provide control signals to said gantry crane by implementing a polygonal object location method to provide position command signals to said Cartesian manipulator,

said polygonal object location method comprising;

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

6. The system of claim 5 wherein:

said video camera comprises an optical bandpass filter having a wavelength pass band; and

said structured light projector comprises a monochromatic light source having a wavelength within said wavelength pass band.

7. The system of claim 6 wherein said wavelength pass band begins at about 520 nanometers and ends at about 540 nanometers.

8. The system of claim 6 wherein said monochromatic light source comprises a laser.

9. A method of lifting objects comprising:

projecting a light plane to illuminate said objects;

acquiring video images of said objects;

providing object height measurements of said objects; and

processing said video images and said object height measurements using a polygonal object location algorithm to provide position command signals to a robotic manipulator,

said polygonal object location method comprising;

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

10. The method of claim 9 wherein:

providing said video images further comprises optical bandpass filtering using an optical filter having a wavelength pass band; and

projecting said light plane comprises producing a monochromatic light having a wavelength within said wavelength pass band.

11. The method of claim 10 wherein said wavelength pass band begins at about 520 nanometers and ends at about 540 nanometers.

12. The method of claim 10 wherein producing said monochromatic light comprises using a laser.

13. A method of lifting objects comprising:

projecting a light plane to illuminate said objects using a light plane projector;

acquiring video images of said objects using a video camera;

providing object height measurements of said objects; and

processing said video images and said object height measurements using a polygonal object location algorithm to provide position command signals to a gantry crane,

said light plane projector being disposed on a trolley of said gantry crane,

said video camera being disposed on a bridge of said gantry crane,

said polygonal object location method comprising;

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

14. The method of claim 13 wherein:

providing said video images further comprises optical bandpass filtering using an optical filter having a wavelength pass band; and

projecting said light plane comprises producing a monochromatic light having a wavelength within said wavelength pass band.

15. The method of claim 14 wherein said wavelength pass band begins at about 520 nanometers and ends at about 540 nanometers.

16. The method of claim 14 wherein producing a monochromatic light comprises using a laser.

17. A method of locating a polygonal object comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical. point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

18. The method of claim 17 wherein:

acquiring said video image of said polygonal object comprises projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line; and

locating said new edge point further comprises detecting a line break in said structured light line.

19. The method of claim 17 wherein detecting said line break comprises:

estimating a critical angle;

estimating a critical offset using said critical angle; and

finding a line segment endpoint using said critical angle and said critical offset.

20. The method of claim 19 wherein estimating said critical angle comprises:

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns; and

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to said critical angle.

21. The method of claim 20 wherein computing said plurality of angular performance values comprises finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

22. The method of claim 20 wherein computing each of said plurality of angular performance values comprises:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

23. The method of claim 19 wherein estimating said critical offset comprises:

computing, from said video image, a plurality of critical angle projection values at said critical angle;

finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding ,to at least one relative maximum offset; and

selecting said critical offset among said at least one relative maximum offset.

24. The method of claim 23 wherein selecting said critical offset comprises:

providing a reference ray having a reference offset; and

finding said critical offset, said critical offset being closest to said reference offset among said at least one relative maximum offset.

25. The method of claim 19 wherein finding said line segment endpoint comprises:

masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray; and

selecting said line segment endpoint among said line segment offsets.

26. The method of claim 25 further comprising calculating a confidence value from said line segment offsets and wherein computing said plurality of edge lines from said historical point set comprises calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

27. The method of claim 17 wherein:

said new edge point comprises a confidence value; and

computing said plurality of edge lines from said historical point set comprises calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

28. The method of claim 17 wherein computing said set of edge lines further comprises refining to exclude outliers, said refining comprising:

creating a previous point set from said historical point set;

identifying a candidate outlier from a previous edge line and said previous point set;

excluding said candidate outlier from said previous point set to yield a new point set;

calculating a new edge line from said new point set;

comparing said new edge line to said previous edge line to yield an edge line difference; and

replacing said previous point set with said new point set whenever said edge line difference exceeds a prescribed error tolerance.

29. The method of claim 28 wherein identifying said candidate outlier comprises finding an element of said previous point set farthest from said previous edge line.

30. The method of claim 28 wherein comparing said new edge line to said previous edge line comprises:

evaluating said previous edge line at a vector of prescribed abscissas to yield a vector of previous ordinates;

evaluating said new edge line at said vector of prescribed abscissas to yield a vector of new ordinates;

subtracting said vector of previous ordinates from said vector of new ordinates to yield an error vector having a plurality of error vector elements; and

calculating an error norm of said error vector to yield said edge line difference.

31. The method of claim 30 wherein calculating said error norm of said error vector comprises finding a square root of a sum of squares of all said error vector elements.

32. The method of claim 30 wherein calculating said error norm of said error vector comprises finding a maximum value among absolute values of all said error vector elements.

33. A projection-based method of locating a polygonal object comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line;

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and

each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns;

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle;

computing, from said video image, a plurality of critical angle projection values at said critical angle;

finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset;

selecting a critical offset among said at least one relative maximum offset;

masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

computing a plurality of orthogonal projection values -in a direction orthogonal to said critical angle;

detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray;

selecting a line segment endpoint among said line segment offsets;

locating a new edge point from said critical angle, said critical offset, and said line segment endpoint;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

34. The method of claim 33 wherein computing said plurality of angular performance values comprises finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

35. The method of claim 33 wherein computing each of said plurality of angular performance values comprises:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

36. The method of claim 33 wherein selecting said critical offset comprises finding, among said at least one relative maximum offset, a closest offset to a reference offset of said reference ray.

37. The method of claim 33 further comprising calculating a confidence value from said line segment offsets and wherein computing said plurality of edge lines from said historical point set comprises calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

38. The method of claim 33 wherein:

said new edge point comprises a confidence value; and

computing said plurality of edge lines from said historical point set comprises calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

39. The method of claim 33 wherein computing said set of edge lines further comprises refining to exclude outliers, said refining comprising:

creating a previous point set from said historical point set;

identifying a candidate outlier from a previous edge line and said previous point set;

excluding said candidate outlier from said previous point set to yield a new point set;

calculating a new edge line from said new point set;

comparing said new edge line to said previous edge line to yield an edge line difference; and

replacing said previous point set with said new point set whenever said edge line difference exceeds a prescribed error tolerance.

40. The method of claim 39 wherein identifying said candidate outlier comprises:

finding an element of said previous point set farthest from said previous edge line.

41. The method of claim 39 wherein comparing said new edge line to said previous edge line comprises:

evaluating said previous edge line at a vector of prescribed abscissas to yield a

vector of previous ordinates;

evaluating said new edge line at said vector of prescribed abscissas to yield a vector of new ordinates;

subtracting said vector of previous ordinates from said vector of new ordinates to yield an error vector having a plurality of error vector elements; and

calculating an error norm of said error vector to yield said edge line difference.

42. The method of claim 41 wherein calculating said error norm of said error vector comprises finding a square root of a sum of squares of all said error vector elements.

43. The method of claim 41 wherein calculating said error norm of said error vector comprises finding a maximum value among absolute values of all said error vector elements.

44. An apparatus for locating a polygonal object comprising:

a machine vision system for acquiring a video image of said polygonal object, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

an edge point locator for locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

an edge point accumulator for incorporating said new edge point into a historical point set;

an edge line computer for computing a plurality of edge lines from said historical point set;

a polygon center computer for computing a polygon center from said edge lines; and

a manipulator command generator for commanding said robotic manipulator to move an end effector toward said polygon center by moving '"'"'said component carrying link.

45. The apparatus of claim 44 wherein:

said machine vision system comprises a light plane projector for projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line; and

said edge point locator comprises a line break detector for detecting a line break in said structured light line.

46. The apparatus of claim 44 wherein said line break detector comprises:

a critical angle estimator for estimating a critical angle;

a critical offset estimator for estimating a critical offset using said critical angle; and

an end point finder for finding said new edge point using said critical angle and said critical offset.

47. The apparatus of claim 46 wherein said critical angle estimator comprises:

a projection array computer for computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

an angular performance evaluator for computing a plurality of angular performance values from said projection array columns; and

a maximum finder for finding a maximum performance value among said angular performance values, said maximum performance value corresponding to said critical angle.

48. The apparatus of claim 47 wherein said angular performance evaluator is adapted to compute said plurality of angular performance values by a method comprising finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

49. The apparatus of claim 47 wherein said angular performance evaluator is adapted to compute each of said plurality of angular performance values by a method comprising:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

50. The apparatus of claim 46 wherein said critical offset estimator comprises:

a projection value computer for computing, from said video image, a plurality of critical angle projection values at said critical angle;

a maximum finder for finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset; and

a maximum selector for selecting said critical offset among said at least one relative maximum offset.

51. The apparatus of claim 50 wherein said maximum selector is adapted to select said critical offset by a method comprising:

providing a reference ray having a reference offset; and

finding said critical offset, said critical offset being closest to said reference offset among said at least one relative maximum offset.

52. The apparatus of claim 46 wherein said end point finder comprises:

an image masker for masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

an orthogonal projection computer for computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

a threshold crossing detector for detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray; and

an end point selector for selecting said line segment endpoint among said line segment offsets.

53. The apparatus of claim 52 further comprising a confidence value calculator for calculating a confidence value from said line segment offsets, and wherein said edge line computer comprises a weighted regression calculator for calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

54. The apparatus of claim 44 wherein:

said edge point locator comprises a confidence value calculator for calculating a plurality of confidence values; and

said edge line computer comprises a weighted regression calculator for calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

55. The apparatus of claim 44 wherein said edge line computer further comprises:

a candidate outlier identifier for identifying a candidate outlier from a previous edge line and a previous point set;

a weighted regression calculator for calculating a new edge line from a new point set;

an edge line comparator for comparing said new edge line to said previous edge line to yield an edge line difference; and

a point set manager for creating said previous point set from said historical point set, excluding said candidate outlier from said previous point set to yield said new point set, and, whenever said edge line difference exceeds a prescribed error tolerance, replacing said previous point set with said new point set.

56. The apparatus of claim 55 wherein said candidate outlier identifier is adapted to identify said candidate outlier by a method comprising finding an element of said previous point set farthest from said previous edge line.

57. The apparatus of claim 55 wherein said edge line comparator comprises:

a previous line evaluator for evaluating said previous edge line at a vector of prescribed abscissas to yield a vector of previous ordinates;

a new line evaluator for evaluating said new edge line at said vector of prescribed abscissas to yield a vector of new ordinates;

a vector differencer for subtracting said vector of previous ordinates from said vector of new ordinates to yield an error vector having a plurality of error vector elements; and

a norm calculator for calculating an error norm of said error vector to yield said edge line difference.

58. The apparatus of claim 57 wherein said norm calculator is adapted to calculate said error norm of said error vector by a method comprising finding a square root of a sum of squares of all said error vector elements.

59. The apparatus of claim 57 wherein said norm calculator is adapted to calculate said error norm of said error vector by a method comprising finding a maximum value among absolute values of all said error vector elements.

60. An apparatus for locating a polygonal object comprising:

a machine vision system for acquiring a video image of said polygonal object, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

a light plane projector for projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line;

a projection array computer for computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

an angular performance evaluator for computing a plurality of angular performance values from said projection array columns;

a maximum finder for finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle;

a projection value computer for computing, from said video image, a plurality of critical angle projection values at said critical angle;

a maximum finder for finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset;

a maximum selector for selecting a critical offset among said at least one relative maximum offset;

an image masker for masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

an orthogonal projection computer for computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

a threshold crossing detector for detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray;

an end point selector for selecting a line segment endpoint among said line segment offsets;

an edge point computer for locating a new edge point from said critical angle, said critical offset, and said line segment endpoint;

an edge point accumulator for incorporating said new edge point into a historical point set;

an edge line computer for computing a plurality of edge lines from said historical point set;

a polygon center computer for computing a polygon center from said edge lines; and

a manipulator command generator for commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

61. The apparatus of claim 60 wherein said angular performance evaluator is adapted to compute said plurality of angular performance values by a method comprising finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

62. The apparatus of claim 60 wherein said angular performance evaluator is adapted to compute each of said plurality of angular performance values by a method comprising:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

63. The apparatus of claim 60 wherein said maximum selector is adapted to select said critical offset by a method comprising:

providing a reference ray having a reference offset; and

finding said critical offset, said critical offset being closest to said reference offset among said at least one relative maximum offset.

64. The apparatus of claim 60 further comprising a confidence value calculator for calculating a confidence value from said line segment offsets, and wherein said edge line computer comprises a weighted regression calculator for calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

65. The apparatus of claim 60 wherein:

said edge point locator comprises a confidence value calculator for calculating a plurality of confidence values; and

said edge line computer comprises a weighted regression calculator for calculating a weighted linear regression wherein each new edge point is weighted by a respective confidence value.

66. The apparatus of claim 60 wherein said edge line computer further comprises:

a candidate outlier identifier for identifying a candidate outlier from a previous edge line and a previous point set;

a weighted regression calculator for calculating a new edge line from a new point set;

an edge line comparator for comparing said new edge line to said previous edge line to yield an edge line difference; and

a point set manager for creating said previous point set from said historical point set, excluding said candidate outlier from said previous point set to yield said new point set, and, whenever said edge line difference exceeds a prescribed error tolerance, replacing said previous point set with said new point set.

67. The apparatus of claim 66 wherein said candidate outlier identifier is adapted to identify said candidate outlier by a method comprising finding an element of said previous point set farthest from said previous edge line.

68. The apparatus of claim 66 wherein said edge line comparator comprises:

a previous line evaluator for evaluating said previous edge line at a vector of prescribed abscissas to yield a vector of previous ordinates;

a new line evaluator for evaluating said new edge line at said vector of prescribed abscissas to yield a vector of new ordinates;

a vector differencer for subtracting said vector of previous ordinates from said vector of new ordinates to yield an error vector having a plurality of error vector elements; and

a norm calculator for calculating an error norm of said error vector to yield said edge line difference.

69. The apparatus of claim 68 wherein said norm calculator is adapted to calculate said error norm of said error vector by a method comprising finding a square root of a sum of squares of all said error vector elements.

70. The apparatus of claim 68 wherein said norm calculator is adapted to calculate said error norm of said error vector by a method comprising finding a maximum value among absolute values of all said error vector elements.

71. A method of detecting a tilted object comprising:

providing a reference ray having a reference angle;

estimating a critical angle from a video image;

differencing said critical angle from said reference angle of said reference ray to yield an error angle; and

comparing an absolute value of said error angle to an angle error tolerance.

72. The method of claim 71 wherein estimating said critical angle comprises:

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns; and

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to said critical angle.

73. The method of claim 72 wherein computing said plurality of angular performance values comprises finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

74. The method of claim 72 wherein computing each of said plurality of angular performance values comprises

forming a plurality of column element differences from said column elements of a respective projection array column;

and

calculating a square root of a sum of squares of all of said column element differences.

79. The apparatus of claim 71 wherein said critical angle estimator comprises:

a projection array computer for computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

an angular performance evaluator for computing a plurality of angular performance values from said projection array columns; and

a maximum finder for finding a maximum performance value among said angular performance values, said maximum performance value corresponding to said critical angle.

80. The apparatus of claim 79 wherein said angular performance evaluator is adapted to compute said plurality of angular performance values by a method comprising finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

81. The apparatus of claim 79 wherein said angular performance evaluator is adapted to compute each of said plurality of angular performance values by a method comprising:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

75. A method of detecting a tilted object comprising:

providing a reference ray having a reference angle;

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns; and

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle and thereby estimating said critical angle;

differencing said critical angle from said reference angle of said reference ray to yield an error angle; and

comparing an absolute value of said error angle to an angle error tolerance.

76. The method of claim 75 wherein computing said plurality of angular performance values comprises finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

77. The method of claim 75 wherein computing each of said plurality of angular performance values comprises:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

78. An apparatus for detecting a tilted object comprising:

a critical angle estimator for estimating a critical angle from a video image;

an angle differencer for differencing said critical angle from a reference angle of a reference ray to yield an error angle; and

an angle tolerance comparator for comparing an absolute value of said error angle to an angle error tolerance.

82. An apparatus for detecting a tilted object comprising:

a projection array computer for computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

an angular performance evaluator for computing a plurality of angular performance values from said projection array columns; and

a maximum finder for finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle thereby estimating said critical angle;

an angle differencer for differencing said critical angle from a reference angle of a reference ray to yield an error angle; and

an angle tolerance comparator for comparing an absolute value of said error angle to an angle error tolerance.

83. The apparatus of claim 82 wherein said angular performance evaluator is adapted to compute said plurality of angular performance values by a method comprising finding a plurality of maximum column values among said column elements of respective ones of said projection array columns.

84. The apparatus of claim 82 wherein said angular performance evaluator is adapted to compute each of said plurality of angular performance values by a method comprising:

forming a plurality of column element differences from said column elements of a respective projection array column; and

calculating a square root of a sum of squares of all of said column element differences.

85. A system for lifting objects comprising:

a Cartesian manipulator;

a machine vision system adapted to acquire video images of said objects; and

a control processor adapted to process said video images and provide position command signals to said Cartesian manipulator,

said machine vision system comprising;

a structured light projector for illuminating said objects and

a video camera adapted to acquire said video images of said illuminated objects,

said structured light projector being disposed on a first link of said Cartesian manipulator,

said video camera being disposed on a second link of said Cartesian manipulator,

said first link being different from said second link.

86. The system of claim 85 wherein said control processor is further adapted to implement a polygonal object location method to provide position command signals to said Cartesian manipulator,

said polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

87. The system of claim 85 wherein said Cartesian manipulator comprises a gantry crane.

88. The system of claim 85 wherein said structured light projector comprises a light plane projector.

89. The system of claim 88 further comprising a distance sensor adapted to acquire object height measurements of said objects and wherein said control processor is further adapted to process said object height measurements and implement a projection-based polygonal object location method to provide position command signals to said Cartesian manipulator,

said projection-based polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line;

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns;

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle;

computing, from said video image, a plurality of critical angle projection values at said critical angle;

finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset;

selecting a critical offset among said at least one relative maximum offset;

masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray;

selecting a line segment endpoint among said line segment offsets;

locating a new edge point from said critical angle, said critical offset, and said line segment endpoint;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

90. The system of claim 85 wherein:

said video camera comprises an optical bandpass filter having a wavelength pass band; and

said structured light projector comprises a monochromatic light source having a wavelength within said wavelength pass band.

91. The system of claim 90 wherein said wavelength pass band begins at about 520 nanometers and ends at about 540 nanometers.

92. The system of claim 90 wherein said monochromatic light source comprises a laser.

93. A system for lifting objects comprising:

a Cartesian manipulator;

a machine vision system adapted to acquire video images of said objects; and

a control processor adapted to process said video images and provide position command signals to said Cartesian manipulator,

said machine vision system comprising;

a structured light projector for illuminating said objects and

a video camera adapted to acquire said video images of said illuminated objects,

said video camera comprising an optical bandpass filter having a wavelength pass band,

said structured light projector comprising a monochromatic light source having a wavelength within said wavelength pass band,

said wavelength pass band beginning at about 520 nanometers and ending at about 540 nanometers.

94. The system of claim 93 wherein said control processor is further adapted to implement a polygonal object location method to provide position command signals to said Cartesian manipulator,

said polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

95. The system of claim 93 wherein said Cartesian manipulator comprises a gantry crane.

96. The system of claim 93 wherein said structured light projector comprises a light plane projector.

97. The system of claim 96 further comprising a distance sensor adapted to acquire object height measurements of said objects and wherein said control processor is further adapted to process said object height measurements and implement a projection-based polygonal object location method to provide position command signals to said Cartesian manipulator,

said projection-based polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line;

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns;

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle;

computing, from said video image, a plurality of critical angle projection values at said critical angle;

finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset;

selecting a critical offset among said at least one relative maximum offset;

masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

detecting a plurality of line segment offsets, whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray;

selecting a line segment endpoint among said line segment offsets;

locating a new edge point from said critical angle, said critical offset, and said line segment endpoint;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

98. The system of claim 93 wherein said monochromatic light source comprises a laser.

99. A method of lifting objects comprising:

acquiring video images of said objects using a video camera;

processing said video images so as to provide position command signals to a Cartesian manipulator;

projecting structured light onto said objects using a structured light projector,

said structured light projector being disposed on a first link of said Cartesian manipulator,

said video camera being disposed on a second link of said Cartesian manipulator,

said first link being different from said second link.

100. The method of claim 99 wherein processing said video images comprises implementing a polygonal object location method to provide position command signals to said Cartesian manipulator,

said polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

101. The method of claim 99 wherein said Cartesian manipulator comprises a gantry crane.

102. The method of claim 99 wherein said structured light comprises a light plane.

103. The method of claim 102 further comprising providing object height measurements of said objects and wherein processing said video images comprises implementing a projection-based polygonal object location method to provide position command signals to said Cartesian manipulator,

said projection-based polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line;

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns;

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle;

computing, from said video image, a plurality of critical angle projection values at said critical angle;

finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset;

selecting a critical offset among said at least one relative maximum offset;

masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray;

selecting a line segment endpoint among said line segment offsets;

locating a new edge point from said critical angle, said critical offset, and said line segment endpoint;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

104. The method of claim 99 wherein:

providing said video images further comprises optical bandpass filtering using an optical filter having a wavelength pass band; and

projecting said structured light comprises producing a monochromatic light having a wavelength within said wavelength pass band.

105. The method of claim 104 wherein said wavelength pass band begins at about 520 nanometers and ends at about 540 nanometers.

106. The method of claim 104 wherein producing said monochromatic light comprises using a laser.

107. A method of lifting objects comprising:

acquiring video images of said objects using a video camera;

processing said. video images so as to provide position command signals to a Cartesian manipulator;

projecting structured light onto said objects using a structured light projector, said step of acquiring said video images comprising optical bandpass filtering using an optical filter having a wavelength pass band,

said step of projecting said structured light comprising producing a monochromatic light having a wavelength within said wavelength pass band,

said wavelength pass band beginning at about 520 nanometers and ending at about 540 nanometers.

108. The method of claim 107 wherein processing said video images comprises implementing a polygonal object location method to provide position command signals to said Cartesian manipulator,

said polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying. link of a robotic manipulator;

locating, in said video image, a new edge point corresponding to a respective edge of said polygonal object;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

109. The method of claim 107 wherein said Cartesian manipulator comprises a gantry crane.

110. The method of claim 107 wherein said structured light comprises a light plane.

111. The method of claim 110 further comprising providing object height measurements of said objects and wherein processing said video images comprises implementing a projection-based polygonal object location method to provide position command signals to said Cartesian manipulator,

said projection-based polygonal object location method comprising:

acquiring a video image of said polygonal object using a machine vision system, at least one component of said machine vision system being disposed on a component carrying link of a robotic manipulator;

projecting a light plane to intersect a horizontal face of said polygonal object whereby said video image comprises at least one structured light line;

computing, from said video image, a projection array having a plurality of projection array rows and a plurality of projection array columns, said projection array columns respectively corresponding to a plurality of projection angles and each comprising a plurality of column elements;

computing a plurality of angular performance values from said projection array columns;

finding a maximum performance value among said angular performance values, said maximum performance value corresponding to a critical angle;

computing, from said video image, a plurality of critical angle projection values at said critical angle;

finding, among said critical angle projection values, at least one relative maximum projection value respectively corresponding to at least one relative maximum offset;

selecting a critical offset among said at least one relative maximum offset;

masking said video image in a neighborhood of a line defined by said critical angle and said critical offset to yield a masked image;

computing a plurality of orthogonal projection values in a direction orthogonal to said critical angle;

detecting a plurality of line segment offsets whereat said orthogonal projection values cross respective ones of a plurality of end point threshold values in a direction defined by a reference ray;

selecting a line segment endpoint among said line segment offsets;

locating a new edge point from said critical angle, said critical offset, and said line segment endpoint;

incorporating said new edge point into a historical point set;

computing a plurality of edge lines from said historical point set;

computing a polygon center from said edge lines; and

commanding said robotic manipulator to move an end effector toward said polygon center by moving said component carrying link.

112. The method of claim 107 wherein producing said monochromatic light comprises using a laser.