SYSTEM AND METHOD OF VISUAL TRACKING

• US 20070073439A1
• Filed: 09/22/2006
• Published: 03/29/2007
• Est. Priority Date: 09/23/2005
• Status: Abandoned Application

• Current Assignee: Braintech, Inc.
• Original Assignee: Braintech, Inc.
• Inventors: Clark, Geoffrey, Habibi, Babak
• Application Number:
  US11/534,578
• Publication Number:
  US 20070073439A1
• Time in Patent Office: Days
• US Class Current: 700/213
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/4182 manipulators and conveyor only
• CPC Class Codes: G05B 2219/37189 Camera with image processing emulates encoder output
• CPC Class Codes: G05B 2219/40546 Motion of object
• CPC Class Codes: G05B 2219/40554 Object recognition to track object on conveyor
• CPC Class Codes: G05B 2219/40617 Agile eye, control position of camera, active vision, pan-tilt camera, follow object
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A machine-vision system, method and article is useful in the field of robotics. One embodiment produces signals that emulate the output of an encoder, based on captured images of an object, which may be in motion. One embodiment provides digital data directly to a robot controller without the use of an intermediary transceiver such as an encoder interface card. One embodiment predicts or determines the occurrence of an occlusion and moves at least one of a camera and/or the object accordingly.

Claims

1. A method operating a machine vision system to control at least one robot, the method comprising:

successively capturing images of an object;

determining a linear velocity of the object from the captured images; and

producing an encoder emulation output signal based on the determined linear velocity, the encoder emulation signal emulative of an output signal from an encoder.

2. The method of claim 1 wherein successively capturing images of an object includes successively capturing images of the object while the object is in motion.

3. The method of claim 1 wherein successively capturing images of an object includes successively capturing images of the object while the object is in motion along a conveyor system.

4. The method of claim 1 wherein determining a linear velocity of the object from the captured images includes locating at least one feature of the object in at least two of the captured images, determining a change of position of the feature between the at least two of the captured images, and determining a time between the capture of the at least two captured images.

5. The method of claim 1 wherein producing an encoder emulation output signal based on the determined linear velocity includes producing at least one encoder emulative waveform.

6. The method of claim 5 wherein producing at least one encoder emulative waveform includes producing a single pulse train output waveform.

7. The method of claim 5 wherein producing at least one encoder emulative waveform includes producing a quadrature output waveform comprising a first pulse train and a second pulse train.

8. The method of claim 5 wherein producing at least one encoder emulative waveform includes producing at least one of a square-wave pulse train or a sine-wave wave form.

9. The method of claim 1 wherein producing at least one encoder emulative waveform includes producing a pulse train emulative of an incremental output waveform from an incremental encoder.

10. The method of claim 1 wherein producing at least one encoder emulative waveform includes producing an analog waveform.

11. The method of claim 1 wherein producing an encoder emulation output signal based on the determined linear velocity includes producing a set of binary words emulative of an absolute output waveform of an absolute encoder.

12. The method of claim 1, further comprising:

providing the encoder emulation signal to an intermediary transducer communicatively positioned between the machine vision system and a robot controller.

13. The method of claim 1, further comprising:

providing the encoder emulation signal to an encoder interface card of a robot controller.

14. The method of claim 1, further comprising:

automatically determining a position of the object with respect to the camera based at least in part on the captured images a change in position of the object between at least two of the images; and

moving the camera relative to the object based at least in part on the determined position of the object with respect to the camera.

15. The method of claim 14 wherein moving the camera relative to the object based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera.

16. The method of claim 14 wherein moving the camera relative to the object based at least in part on the determined position of the object with respect to the camera includes changing a movement of the object to at least partially avoid an occlusion of a view of the object by the camera.

17. The method of claim 16, further comprising:

automatically determining at least one of a velocity or an acceleration of the object with respect to a reference frame;

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera; and

determining at least one of a new position or a new orientation for the camera relative to the object that at least partially avoids the occlusion.

18. The method of claim 14, further comprising:

determining whether at least one feature of the object in at least one of the images is occluded; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid the occlusion in a view of the object by the camera; and

determining at least one of a new position or a new orientation for the camera relative to the object that at least partially avoids the occlusion.

19. The method of claim 1, further comprising:

determining at least one other velocity of the object from the captured images; and

producing at least one other encoder emulation output signal based on the determined other velocity, the at least one other encoder emulation signal emulative of an output signal from an encoder.

20. The method of claim 1 wherein determining at least one other velocity of the object from the captured images includes determining at least one of an angular velocity or another linear velocity.

21. A machine vision system to control at least one robot, the machine vision system comprising:

a camera operable to successively capture images of an object in motion;

means for determining a linear velocity of the object from the captured images; and

means for producing an encoder emulation output signal based on the determined linear velocity, the encoder emulation signal emulative of an output signal from an encoder.

22. The machine vision system of claim 21 wherein the means for determining a linear velocity of the object from the captured images includes means for locating at least one feature of the object in at least two of the captured images, determining a change of position of the feature between the at least two of the captured images, and determining a time between the capture of the at least two captured images.

23. The machine vision system of claim 21 wherein the means for producing an encoder emulation output signal based on the determined linear velocity produces at least one encoder emulative waveform selected from the group consisting of a single pulse train output waveform and a quadrature output waveform comprising a first pulse train and a second pulse train.

24. The machine vision system of claim 21 wherein means for producing at least one encoder emulative waveform produces a pulse train emulative of an incremental output waveform from an incremental encoder.

25. The machine vision system of claim 21 wherein means for producing an encoder emulation output signal based on the determined linear velocity produces a set of binary words emulative of an absolute output waveform of an absolute encoder.

26. The machine vision system of claim 21 wherein the machine vision system is communicatively coupled to provide the encoder emulation signal to an intermediary transducer communicatively positioned between the machine vision system and a robot controller.

27. The machine vision system of claim 21, further comprising:

at least one actuator physically coupled to move the camera relative to the object based at least in part on at least one of a position, a speed or a velocity of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

28. The machine vision system of claim 21, further comprising:

at least one actuator physically coupled to adjust a movement of the object relative to the camera based at least in part on at least one of a position, a speed or a velocity of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

29. The machine vision system of claim 21, further comprising:

means for automatically determining at least one of a velocity or an acceleration of the object with respect to a reference frame; and

means for predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera.

30. The machine vision system of claim 21, further comprising:

means for determining at least one other velocity of the object from the captured images; and

means for producing at least one other encoder emulation output signal based on the determined other velocity, the at least one other encoder emulation signal emulative of an output signal from an encoder.

31. The machine vision system of claim 30 wherein means for determining at least one other velocity of the object from the captured images includes software means for determining at least one of an angular velocity or another linear velocity from the images.

32. A computer-readable medium storing instructions for causing a machine vision system to control at least one robot, by:

determining at least one velocity of an object along or about at least a first axis from a plurality of successively captured images of the object; and

producing at least one encoder emulation output signal based on the determined at least one velocity, the encoder emulation signal emulative of an output signal from an encoder.

33. The computer-readable medium of claim 32 wherein producing at least one encoder emulation output signal based on the determined at least one velocity, the encoder emulation signal emulative of an output signal from an encoder includes producing at least one encoder emulative waveform selected from the group consisting of a single pulse train output waveform and a quadrature output waveform comprising a first pulse train and a second pulse train.

34. The computer-readable medium of claim 32 wherein producing at least one encoder emulation output signal based on the determined at least one velocity, the encoder emulation signal emulative of an output signal from an encoder includes producing a set of binary words emulative of an absolute output waveform of an absolute encoder.

35. The computer-readable medium of claim 32 wherein the instructions cause the machine-vision system to further control the at least one robot, by:

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera.

36. The computer-readable medium of claim 32 wherein the instructions cause the machine-vision system to additionally control movement of the object, by:

adjust a movement of the object relative to the camera based at least in part on at least one of a position, a speed or a velocity of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

37. The computer-readable medium of claim 32 wherein the instructions cause the machine-vision system to additionally control the camera, by:

moving the camera relative to the object based at least in part on at least one of a position, a speed or a velocity of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

38. The computer-readable medium of claim 32 wherein determining at least one velocity of an object along or about at least a first axis from a plurality of successively captured images of the object includes determining a velocity of the object along or about two different axes from the captured images;

and wherein producing at least one other encoder emulation output signal based on the at least one determined velocity includes producing at least two distinct encoder emulation output signals, each of the encoder emulation output signals indicative of the determined velocity about or along a respective one of the axes.

39. A method operating a machine vision system to control at least one robot, the method comprising:

successively capturing images of an object;

determining a first linear velocity of the object from the captured images;

producing a digital output signal based on the determined first linear velocity, the digital output signal indicative of a position and at least one of a velocity and an acceleration; and

providing the digital output signal to a robot controller without the use of an intermediary transducer.

40. The method of claim 39 wherein successively capturing images of an object includes capturing successive images of the object while the object is in motion.

41. The method of claim 39 wherein successively capturing images of an object includes capturing successive images of the object while the object is in motion along a conveyor system.

42. The method of claim 39 wherein determining a first linear velocity of the object from the captured images includes locating at least one feature of the object in at least two of the captured images, determining a change of position of the feature between the at least two of the captured images, and determining a time between the capture of the at least two captured images.

43. The method of claim 39 wherein providing the digital output signal to a robot controller without the use of an intermediary transducer includes providing the digital output signal to the robot controller without the use of an encoder interface card.

44. The method of claim 39, further comprising:

automatically determining a position of the object with respect to the camera based at least in part on the captured images a change in position of the object between at least two of the images; and

moving the camera relative to the object based at least in part on the determined position of the object with respect to the camera.

45. The method of claim 44 wherein moving the camera relative to the object based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera.

46. The method of claim 44 wherein moving the camera relative to the object based at least in part on the determined position of the object with respect to the camera includes changing a speed of the object to at least partially avoid an occlusion of a view of the object by the camera.

47. The method of claim 46, further comprising:

automatically determining at least one of a velocity or an acceleration of the object with respect to a reference frame;

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera; and

determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

48. The method of claim 44, further comprising:

determining whether at least one feature object in at least one of the images is occluded; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid the occlusion in a view of the object by the camera; and

determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

49. The method of claim 39, further comprising:

determining at least a second linear velocity of the object from the captured images, and wherein producing the digital output signal is further based on the determined second linear velocity.

50. The method of claim 39, further comprising:

determining at least one angular velocity of the object from the captured images, and wherein producing the digital output signal is further based on the at least one determined angular velocity.

51. A machine vision system to control at least one robot, the machine vision system comprising:

a camera operable to successively capture images of an object in motion;

means for determining at least a velocity of the object along or about at least one axis from the captured images;

means for producing a digital output signal based on the determined velocity, the digital output signal indicative of a position and at least one of a velocity and an acceleration, wherein the machine vision system is communicatively coupled to provide the digital output signal to a robot controller without the use of an intermediary transducer.

52. The machine vision system of claim 51 wherein means for determining at least a velocity of the object along or about at least one axis from the captured images includes means for determining a first linear velocity along a first axis and means for determining a second linear velocity along a second axis.

53. The machine vision system of claim 51 wherein means for determining at least a velocity of the object along or about at least one axis from the captured images includes means for determining a first angular velocity about a first axis and means for determining a second angular velocity about a second axis.

54. The machine vision system of claim 51 wherein means for determining at least a velocity of the object along or about at least one axis from the captured images includes means for determining a first linear velocity about a first axis and means for determining a first angular velocity about the first axis.

55. The machine vision system of claim 51, further comprising:

means for moving the camera relative to the object based at least in part on at least one of a position, a speed or an acceleration of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

56. The machine vision system of claim 51, further comprising:

means for adjusting a movement of the object based at least in part on at least one of a position, a speed or an acceleration of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

57. The machine vision system of claim 51, further comprising:

means for predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object.

58. A computer-readable medium storing instructions to operate a machine vision system to control at least one robot, by:

determining at least a first velocity of an object in motion from a plurality of successively captured images of the object;

producing a digital output signal based on at least the determined first velocity, the digital output signal indicative of at least one of a velocity or an acceleration of the object; and

providing the digital output signal to a robot controller without the use of an intermediary transducer.

59. The computer-readable medium of claim 58 wherein determining at least a first velocity of an object includes a first linear velocity of the object along a first axis, and determining a second linear velocity along a second axis.

60. The computer-readable medium of claim 58 wherein determining at least a first velocity of an object includes determining a first angular velocity about a first axis and determining a second angular velocity about a second axis.

61. The computer-readable medium of claim 58 wherein determining at least a first velocity of an object includes determining a first linear velocity about a first axis and determining a first angular velocity about the first axis.

62. The computer-readable medium of claim 58 wherein the instructions cause the machine vision system to control the at least one robot, by

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object.

63. A method operating a machine vision system to control at least one robot, the method comprising:

successively capturing images of an object with a camera that moves independently from at least an end effector portion of the robot;

automatically determining at least a position of the object with respect to the camera based at least in part on the captured images a change in position of the object between at least two of the images; and

moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera.

64. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes moving the camera to track the object as the object moves.

65. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes moving the camera to track the object as the object moves along a conveyor.

66. The method of claim 63 wherein moving at least one of the camera or object based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera.

67. The method of claim 63 wherein moving at least one of the camera or object based at least in part on the determined position of the object with respect to the camera includes adjusting a movement of the object to at least partially avoid an occlusion of a view of the object by the camera.

68. The method of claim 63, further comprising:

automatically determining at least one of a velocity or an acceleration of the object with respect to a reference frame.

69. The method of claim 63, further comprising:

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object; and

wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid an occlusion of a view of the object by the camera.

70. The method of claim 69, further comprising:

determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

71. The method of claim 63, further comprising:

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object; and

wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes adjusting a movement of the object to at least partially avoid an occlusion of a view of the object by the camera.

72. The method of claim 71, further comprising:

determining at least one of at least one of a new position, a new speed, a new acceleration, or a new orientation for the object that at least partially avoids the occlusion.

73. The method of claim 63, further comprising:

determining whether at least one feature of the object in at least one of the images is occluded; and

wherein moving the camera based at least in part on the determined position of the object with respect to the camera includes moving the camera to at least partially avoid the occlusion in a view of the object by the camera.

74. The method of claim 73, further comprising:

determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

75. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes translating the camera.

76. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes change a speed at which the camera is translating.

77. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes pivoting the camera about at least one axis.

78. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes translating the object.

79. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes changing a speed at which the object is translating.

80. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes pivoting the object about at least one axis.

81. The method of claim 63 wherein moving at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera includes changing a speed at which the object is rotating.

82. A machine vision system to control at least one robot, the machine vision system comprising:

a camera operable to successively capture images of an object in motion, the camera mounted;

means for automatically determining at least a position of the object with respect to the camera based at least in part on the captured images a change in position of the object between at least two of the images; and

at least one actuator coupled to move at least one of the camera or the object; and

means for controlling the at least one actuator based at least in part on the determined position of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

83. The machine vision system of claim 82, further comprising:

means for predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object.

84. The machine vision system of claim 83, further comprising:

means for determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

85. The machine vision system of claim 84 wherein the actuator is physically coupled to move the camera.

86. The machine vision system of claim 83, further comprising:

means for determining at least one of a new position or a new orientation for the object that at least partially avoids the occlusion.

87. The machine vision system of claim 86 wherein the actuator is physically coupled to move the object.

88. The machine vision system of claim 82, further comprising:

means for detecting an occlusion of at least one feature of the object in at least one of the images of the object.

89. The machine vision system of claim 88, further comprising:

means for determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

90. The machine vision system of claim 89 wherein the actuator is physically coupled to move at least one of translate or rotate the camera.

91. The machine vision system of claim 82, further comprising:

means for determining at least one of a new position or a new orientation for the object that at least partially avoids the occlusion.

92. The machine vision system of claim 91 wherein the actuator is physically coupled to at least one of translate, rotate or adjust a speed of the object.

93. A computer-readable medium storing instructions that cause a machine vision system to control at least one robot, by:

automatically determining at least a position of an object with respect to a camera that moves independently from at least an end effector portion of the robot, based at least in part on a plurality of successively captured images a change in position of the object between at least two of the images; and

causing at least one actuator to move at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera.

94. The computer-readable medium of claim 93 wherein causing at least one actuator to move at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera includes translating the camera along at least one axis.

95. The computer-readable medium of claim 93 wherein causing at least one actuator to move at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera includes rotating the camera about at least one axis.

96. The computer-readable medium of claim 93 wherein causing at least one actuator to move at least one of the camera or the object based at least in part on the determined position of the object with respect to the camera to at least partially avoid an occlusion of a view of the object by the camera includes adjusting a movement of the object.

97. The computer-readable medium of claim 93 wherein adjusting a movement of the object includes adjusting at least one of a linear velocity or rotational velocity of the object.

98. The computer-readable medium of claim 93 wherein the instructions cause the machine vision system to control the at least one robot, further by:

predicting an occlusion event based on at least one of a position, a velocity or an acceleration of the object.

99. The computer-readable medium of claim 93 wherein the instructions cause the machine vision system to control the at least one robot, further by:

determining whether at least one feature of the object in at least one of the images is occluded.

100. The computer-readable medium of claim 93 wherein the instructions cause the machine vision system to control the at least one robot, further by:

determining at least one of a new position or a new orientation for the camera that at least partially avoids the occlusion.

101. The computer-readable medium of claim 93 wherein the instructions cause the machine vision system to control the at least one robot, further by:

determining at least one of a new position, a new orientation, or a new speed for the object which at least partially avoids the occlusion.