Apparatus and methods for automated observation of three-dimensional objects

• US 5,249,045 A
• Filed: 01/28/1992
• Issued: 09/28/1993
• Est. Priority Date: 12/24/1954
• Status: Expired due to Fees

• Current Assignee: Jerome H. Lemelson
• Original Assignee: Jerome H. Lemelson
• Inventors: Lemelson, Jerome H.
• Primary Examiner(s): Powell, Mark R.
• Assistant Examiner(s): Metjahic, Safet
• Application Number:
  US07/826,617
• Time in Patent Office: 609 Days
• Field of Search: 358/93, 358/101, 358/107, 358/106, 358/108, 358/903, 356/237
• US Class Current: 348/92
• CPC Class Codes: B07C 5/10 measured by light-responsive means sorting according to colour B07C5/342
• CPC Class Codes: G01B 11/022 by means of tv-camera scanning
• CPC Class Codes: G05B 2219/33192 Radio link, wireless
• CPC Class Codes: G05B 2219/36395 Load local computer program from host, data transfer ram to rom, BTR
• CPC Class Codes: G05B 2219/36511 Select by a detector
• CPC Class Codes: G05B 2219/36513 Select out of a plurality of programs, patterns
• CPC Class Codes: G05B 2219/37572 Camera, tv, vision
• CPC Class Codes: G05B 2219/40283 Reservoir with additional material on vehicle with manipulator
• CPC Class Codes: G05B 2219/40284 Toolrack on vehicle with manipulator, toolchanger
• CPC Class Codes: G06T 1/0014 Image feed-back for automatic industrial control, e.g. robot with camera robots B25J19/023
• CPC Class Codes: G07D 7/12 Visible light, infrared or ultraviolet radiation
• CPC Class Codes: G08B 13/19602 Image analysis to detect motion of the intruder, e.g. by frame subtraction
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19634 Electrical details of the system, e.g. component blocks for carrying out specific functions
• CPC Class Codes: G08B 13/19652 Systems using zones in a single scene defined for different treatment, e.g. outer zone gives pre-alarm, inner zone gives alarm
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/782 on tape
• CPC Class Codes: H04N 5/7822 with stationary magnetic heads
• CPC Class Codes: H04N 5/78263 for recording on tracks inclined relative to the direction of movement of the tape
• CPC Class Codes: H04N 5/9201 : involving the multiplexing of an additional signal and the video signal
• CPC Class Codes: H04N 5/9205 : the additional signal being at least another television signal

Abstract

An automatic scanning apparatus and method for detecting the presence of one or more objects in an image field under investigation or inspection. Electro-optical scanning means, such as a television camera, is employed to scan an image field and generate output electrical signals which vary in accordancd with variations in the optical characteristics of the matter and objects in the image field scanned. Such signals are computer processed and analyzed to generate coded electrical signals which define optical characteristics of portion of the image field scanned, such as objects or the images of objects scanned, their shape, color of a combination of color and shape. Electronic means is provided to generate further coded electrical signals which indicate the presence of one or more objects in the image field scanned and may be used to effect intelligent indications thereof, to control one or more devices such as a motor or motors and/or to provide information for computational purposes to be rpocessed and utilized by a computer. In one form, that shape of an object or objects is detected arid coded signals generated are employed to effect a comparison of such shape with information relating to the shapes of known objects to identify the object or objects scanned. In another form, the color or surface characteristics of an object is detected and resulting signals indicative thereof are compared with information derived from a memory to identify either the object or its color or surface characteristics. In a third form both shape and color are detected and compared with recorded information for identification purposes.

Claims

1. An apparatus for automatically analyzing three-dimensional objects comprising:

(a) means for supporting a three-dimensional object proximate an observation system;

(b) means for generating, and for directing at said object, electromagnetic energy signals;

(c) means for detecting resultant electromagnetic energy signals emanating from at least a portion of said object;

(d) means for storing a plurality of command control signals in an electronic storage device, and for selectively addressing, reproducing and using said command control signals to automatically control said generating means and said detection means so that said detection means detects resultant energy signals from a plurality of different portions of said object;

(e) means for computer processing the detected resultant energy signals and for generating and electronically storing image signals representing different portions of said object; and

(f) means for computer analyzing the stored image signals and for generating electrical code signals representing characteristics of said object.

2. An apparatus in accordance with claim 1 wherein said storage means includes an addressable recording device, and wherein said processing means stores said generated image signals in said storage means.

3. An apparatus in accordance with claim 2 further comprising means for addressing and reproducing image signals from said recording device.

4. An apparatus in accordance with claim 3 further comprising means for comparing the addressed and reproduced image signals with previously stored signals.

5. An apparatus in accordance with claim 1 wherein said processing means for generating said image signals includes means for computer processing the detected resultant energy signals emanating from at least two dimensions

6. An apparatus in accordance with claim 1 further comprising means for operating said signal detecting and generating means a plurality of times to generate a plurality of image signals representing the resultant energy signals detected from different portions of the observed object.

7. An apparatus in accordance with claim 1 further comprising:

(a) means for indicating information relating to at least a portion of the object; and

(b) means for applying the image signals to control said indicating means.

8. An apparatus in accordance with claim 1 wherein means for generating electromagnetic energy includes scanning means for generating and directing a radiation beam at the object.

9. An apparatus in accordance with claim 1 wherein said detection means includes means for detecting radiation reflected from said object.

10. An apparatus in accordance with claim 1 wherein said detection means includes means for detecting radiation passed through said object.

11. An apparatus in accordance with claim 1 wherein said detection means includes means for detecting x-ray radiation.

12. An apparatus in accordance with claim 1 further comprising means for relatively moving said object and said detection means to rotate the object about an axis and to present different portions of the object to the detection means.

13. An apparatus in accordance with claim 12 wherein said means for relatively moving said object and said detection means includes means for moving the object in three dimensions to present selected outer surfaces of the object to the detection means.

14. An apparatus in accordance with claim 1 further comprising means for relatively moving said object and said detection means to move said detection means around said observed object.

15. An apparatus in accordance with claim 1 wherein said detection means comprises a plurality of detection devices each of which is positioned to detect a portion of said observed object.

16. An apparatus in accordance with claim 7 further including means for modifying the further electrical code signals to alter the indicated information.

17. An apparatus in accordance with claim 1 further including means for analyzing selected recorded image signals to determine whether a characteristic of the analyzed object is within a specified tolerance.

18. An apparatus in accordance with claim 1 wherein the means for analyzing the image signals includes means for generating feedback signals to control operation of at least the generating means.

19. An apparatus in accordance with claim 18 wherein the feedback control means also controls the operation of the detection means.

20. The apparatus of claim 1 wherein the means for analyzing the image signals includes means for selectively reproducing previously stored data and comparing the reproduced data to said image signals.

21. The apparatus of claim 1 wherein the analyzing means includes means for storing said coded signals in an electronic storage device.

22. The apparatus of claim 1 wherein said coded electrical signals include an error signal.

23. The apparatus of claim 21 further comprising means for storing reference data in an electronic storage device, reproducing the coded signals and selected of the reference data from said electronic storage device, and comparing said coded electrical signals with said reference data to generate further coded signals.

24. The apparatus of claim 23 further comprising:

(a) further means for inspecting the object; and

(b) means for using said further coded signals to control the further means for inspecting the object.

25. The apparatus of claim 1 further comprising (a) means for conducting a preliminary imaging operation on the object to scan and identify a plurality of reflecting marks on the surface of the object, (b) means for generating electrical signals indicating the identity of the object, and (c) means for applying the generated electrical signals to select the command control signals to reproduce from the storage device.

26. A method of analyzing the interior of a three-dimensional object comprising:

(a) causing penetrating radiation to pass through at least a portion of the object and to be modulated by variations in the object'"'"'s structure;

(b) detecting the modulated penetrating radiation and generating electrical signals that vary in accordance with variations in the detected radiation;

(c) computer analyzing said electrical signals and generating electrical code signals defining characteristics of the electrical signals;

(d) recording in a storage device said electrical code signals;

(e) reproducing said electrical code signals from the storage device;

(f) applying the reproduced electrical code signals to generate and display information about the internal characteristics of said object; and

(g) storing a plurality of command control signals in an electronic storage device, and selectively addressing, reproducing and using said command control signals to automatically control the detecting and analyzing operations.

27. A method in accordance with claim 26 wherein the operation of detecting the penetrating radiation includes applying selectively reproduced command control signals to effect controlled relative movement between said object and said sensor.

28. A method in accordance with claim 26 wherein the operation of detecting the penetrating radiation includes applying selectively reproduced command control signals to effect controlled relative movement between said object and a source of the penetrating radiation.

29. A method in accordance with claim 26 further comprising analyzing the electrical signals to determine if a characteristic of the object meets minimum quality requirements.

30. A method in accordance with claim 29 further comprising rejecting the object upon determination that it does not meet predetermined quality requirements.

31. A method in accordance with claim 29 further comprising marking the three-dimensional object upon determination that it does not meet predetermined quality requirements.

32. A method in accordance with claim 28 wherein the operation of effecting controlled relative movement between said object and said source of penetrating radiation includes controlling a moveable arm member upon which the source of said penetrating radiation is mounted.

33. A method in accordance with claim 27 wherein the operation of effecting controlled relative movement between said object and said sensor includes controlling a moveable arm member upon which the sensor is mounted.

34. An apparatus for automatically observing three-dimensional objects comprising:

(a) means for supporting a three-dimensional object at an observation station;

(b) means for detecting electromagnetic energy signals emanating from at least a portion of said object, and for detecting variations in the detected signals, which variations define at least one characteristic of the three-dimensional object;

(c) means for moving said object and said detection means relative to each other;

(d) means for generating image signals representing a characteristic defined by the detected energy signals.

(e) means for storing the image signals in a storage device along with an associated code that identifies the location in the storage device where the associated image signals are stored;

(f) electronic processing means for locating in and obtaining from the storage device selected image signals, for analyzing the selected image signals, and for generating electrical code signals defining a characteristic about said object; and

(g) wherein the storage device includes a plurality of command control signals, and further comprising computing means for selectively addressing, reproducing and using said command control signals to automatically control the means for relatively moving the object and the detection means.

35. The apparatus in accordance with claim 34 wherein the electronic processing means includes means for using the reproduced command control signals to control the means for generating image signals.

36. A method for automatically observing three-dimensional objects comprising:

(a) supporting a three-dimensional object at an observation station, and employing an imaging device to precisely position said object relative to the observation station;

(b) causing the object to modulate electromagnetic energy signals;

(c) detecting electromagnetic energy signals modulated from a first portion of said object;

(d) detecting electromagnetic energy signals modulated from at least one different portion of said object;

(e) computer processing the detected energy signals and generating and electronically storing image signals representing a characteristic of the observed object;

(f) computer analyzing the stored image signals and generating electrical code signals representing further characteristics of the object; and

(g) storing a plurality of command control signals at selectable locations of a signal storage device, and selecting, reproducing, and using some of said stored command control signals to control at least one of the positioning, detecting, computer processing, or computer analyzing operations.

37. A method in accordance with claim 36 further including storing the generated image signals in said signal storage device.

38. A method in accordance with claim 37 further comprising selecting and reproducing image signals from the signal storage device.

39. A method in accordance with claim 38 further comprising comparing the selected and reproduced image signals with previously stored signals.

40. A method in accordance with claim 36 further comprising processing detected electromagnetic energy signals modulated from at least two dimensions of the object.

41. A method in accordance with claim 36 further comprising generating a plurality of image signals representing the energy signals detected from different portions of the observed object.

42. A method in accordance with claim 36 further comprising processing the image signals and employing them to control a device for displaying information defining at least a portion of the object.

43. A method in accordance with claim 36 wherein the object is caused to modulate electromagnetic energy by generating and directing radiation at the object.

44. A method in accordance with claim 43 wherein the operation of detecting the modulated energy signals includes detecting radiation reflected from the object.

45. A method in accordance with claim 43 wherein the operation of detecting the modulated energy signals includes detecting radiation passed through the object.

46. A method in accordance with claim 36 further comprising:

(a) storing the image signals in the storage device along with an associated code that identifies the location in the storage device where the associated image signals are stored; and

(b) employing the associated code to locate in and obtain from the storage device selected image signals.

47. A method in accordance with claim 36 further comprising controllably effecting three-dimensional movement between the object and a radiation scanning device to detect energy signals from different portions of the object.

48. A method in accordance with claim 36 further comprising controlling a moveable arm member upon which an imaging device is mounted to move in three dimensions relative to the object and to detect energy signals from selected outer surfaces of the object.

49. A method in accordance with claim 48 further comprising applying selected recorded command control signals to control the path travelled by the imaging device.

50. A method for automatically generating and using image information relative to work-in-process, comprising:

(a) supporting work-in-process proximate a controllable imaging system;

(b) storing a plurality of selectable command control signals on an electronic storage device for use in automatically controlling the imaging system;

(c) operating the imaging system to locate the work-in-process relative to the imaging system;

(d) selecting and reproducing command control signals from the storage device and applying the reproduced signals to automatically control the imaging system to conduct an imaging operation on the work-in-process;

(e) generating electrical imaging signals during the imaging operation, which imaging signals correspond to characteristics about the work-in-process;

(f) computer processing the imaging signals and generating and storing in an electrical storage device coded signals defining characteristics of the work-in-process;

(g) computer processing the coded signals and generating feedback control signals; and

(h) using at least a portion of the feedback control signals to affect further imaging operations by the imaging system.

51. The method of claim 50 further comprising storing on the electronic storage device command control signals indicative of specified tolerances;

and wherein the operation of computer processing the coded signals includes comparing information defined by the coded signals to stored tolerances defined by the command control signals.

52. The method of claim 50 wherein the operation of using the feedback signals to affect further imaging operations includes using a portion of the feedback signals to control further relative movement between the work in process and the imaging system.

53. The method of claim 50 wherein the imaging operation comprises an inspection operation.

54. The method of claim 50 wherein the imaging system includes a scanning device mounted on a controllable arm member, and wherein selected of the feedback control signals are used to control the arm member to precisely locate the scanning device relative to the work-in-process.

55. The method of claim 54 further comprising controlling the arm member and the scanning device to conduct multi-dimensional imaging operations.

56. The method of claim 50 wherein operation of the imaging system to locate the work-in-process includes identifying variably reflecting marks on the surface of the work-in-process.

57. The method of claim 50 wherein the imaging operation includes generating image signals defining dimensions of portions of the work-in-process.

58. The method of claim 50 further comprising employing the feedback control signals to control a power operated device

59. The method of claim 50 wherein operation of the imaging system to locate the work-in-process includes finding a specific feature of the work-in-process and precisely positioning the imaging device within a defined tolerance relative to the specific feature.

60. The method of claim 50 wherein the operation of controlling the image system includes controlling a source of x-ray radiation, and wherein said image signals define internal characteristics of the work-in-process.

61. The method of claim 50 further comprising using at least a portion of the feedback control signals to step the imaging system through a series of similar imaging operations.

62. The method of claim 55 wherein said arm member is a controllable articulated manipulator arm capable of multi-dimensional movement, and wherein the operation of using the feedback control signals to control further imaging operations includes applying a portion of the feedback signals to affect multi-dimensional control of the manipulator arm.

63. A method for generating, storing and processing image information relating to work-in-process, comprising:

(a) supporting work-in-process proximate a controllable imaging device;

(b) storing a plurality of selectable command control signals on an electronic storage device, which signals are suitable for use in automatically controlling the imaging device and for conducting a plurality of image processing operations;

(c) controlling the imaging device to automatically locate the work-in-process and the imaging device relative to each other;

(d) selecting and reproducing command control signals from the storage device and applying the selected reproduced signals to automatically control the imaging device to conduct an imaging operation on the work-in-process;

(e) generating electrical imaging signals during the imaging operation, which imaging signals correspond to characteristics about the work-in-process;

(f) selecting and reproducing further command control signals from the storage device and applying the further signals to process at least a portion of the imaging signals and to generate and store coded signals defining characteristics of the work-in-process; and

(g) selecting and reproducing still further command control signals from the storage device and applying the further signals to process the coded signals and to generate information signals defining further characteristics of the work-in-process.

64. A method of claim 63 wherein the operation of addressing and reproducing the further command control signals and generating information signals includes generating feedback control signals and using at least a portion of the feedback control signals to affect further imaging operations by the imaging device.

65. The method of claim 63 wherein selected of the command control signals stored in the electronic storage device are indicative of specified tolerances;

and wherein the operation of processing the coded signals includes comparing information defined by the coded signals to stored tolerances defined by selected of the command control signals.

66. The method of claim 64 wherein the operation of using the feedback signals to affect further imaging operations includes using a portion of the feedback signals to induce and control relative movement between the imaging device and the work in process during the imaging operation.

67. The method of claim 63 wherein the imaging operation comprises an inspection operation.

68. The method of claim 66 wherein the imaging device is mounted on a controllable arm member, and wherein the relative movement between the imaging device and the work-in-process is carried out by selectively controlling the arm member.

69. The method of claim 68 further comprising controlling the arm member and the imaging device to conduct multi-dimensional imaging operations on the work-in-process.

70. The method of claim 63 wherein the operation of automatically locating the work-in-process includes employing the imaging device to scan and identify defined characteristics on the surface of the work-in-process.

71. The method of claim 63 wherein the imaging operation includes generating image signals defining dimensions of portions of the work-in-process.

72. The method of claim 64 further comprising employing the feedback control signals to control a power operated device.

73. The method of claim 68 wherein the operation of automatically locating the imaging device and work-in-process includes using the imaging device to find a specific feature of the work-in-process and precisely positioning the imaging device and the work-in-process to within a defined tolerance relative to the specific feature.

74. The method of claim 63 wherein the operation of controlling the imaging device includes controlling a source of x-ray radiation and wherein said image signals define internal characteristics of the work-in-process.

75. The method of claim 64 further comprising using at least a portion of the feedback control signals to step the imaging device through a series of similar imaging operations on the work-in-process.

77. The method of claim 63 wherein the output control signals comprise feedback signals employed to effect further imaging operations on the object.

78. The method of claim 63 wherein the operation of illuminating the object includes scanning the object with a collimated radiation beam.

76. A method for automatically generating, interpreting and using image information related to an object comprising:

(a) conducting a preliminary imaging operation to precisely locate the object relative to an inspection station;

(b) illuminating the aligned object with radiant energy;

(c) detecting radiant energy reflected from the object and modulated in accordance with physical characteristics of the object;

(d) generating characteristic electrical signals which vary in accordance with the detected modulated reflected radiant energy;

(e) storing the generated characteristic electrical signals in an addressable electrical storage device;

(f) storing in an addressable electrical storage device command control and reference data signals applicable for effecting computer analysis and interpretation of stored characteristic electrical signals;

(g) retrieving stored characteristic electrical signals and command control and data signals from the addressable electrical storage device and computer analyzing the retrieved characteristic electrical signals to derive and interpret image information about the object;

(h) storing in an addressable electrical storage device coded electrical signals defining the image information resulting from the computer analyzing operation; and

(i) applying the coded electrical signals to generate further output control signals for use in controlling electrical equipment.