Robotic vision system

• US 4,853,771 A
• Filed: 04/08/1988
• Issued: 08/01/1989
• Est. Priority Date: 07/09/1986
• Status: Expired due to Fees

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: the united states of america as represented by the secretary of the navy
• Inventors: Witriol, Norman M., Brumage, William H., Cowling, David H.
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): Peng, John K.
• Application Number:
  US07/179,327
• Time in Patent Office: 480 Days
• Field of Search: 358/93, 358/101, 358/106, 358/107, 358/110, 358/125, 901/46, 901/47, 901/9, 250/222.1, 250/461.1, 250/462.1, 250/463.1, 250/467.1, 250/483.1, 250/484.1, 235/375, 235/454, 364/513, 364/478, 382/48, 382/61
• US Class Current: 348/94
• CPC Class Codes: B25J 13/089 Determining the position of the robot with reference to its environment
• CPC Class Codes: B25J 19/023 including video camera means
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/4083 Adapting programme, configuration
• CPC Class Codes: G05B 2219/37572 Camera, tv, vision
• CPC Class Codes: G05B 2219/50052 Orienting workpiece relative to tool

Abstract

A Robotic Vision System for real time operation based on a luminous fiduciary code located upon a robot and an object the robot is to operate on. The luminescent code, when irradiated, locate the position of the object relative to the robot thus allowing placement of the fiduciary codes, on the object and the robot, into a predetermined desired position and orientation.

Claims

1. An apparatus for robotic vision to enable an object to be located and moved by a robot, whether the object is situated correctly or not, comprising:

a luminescent fiduciary code which identifies the position and orientation of the object to be moved by the robot, the fiduciary code being marked on the object and on the robot;

luminescent means for causing the fiduciary code to luminesce, said luminescent means being an ultraviolet or infrared light;

sensing means for detecting the position of the fiduciary code wherein the sensing means is a first and a second videocon aimed at the object and positioned to each receive a scene containing the object including, at times, the luminescence of the fiduciary code, said first and second videocons having a first and a second signal, respectively, which signals are representative of luminescent and non-luminescent images seen by both the first and the second videocons;

computing means for determining the position and orientation of any portion of the object in relation to the robot in real time; and

connection means to pass the first and second signals of the first and second videocons, respectively, to the computing means where image subtraction is performed by subtracting a luminescent image from a non-luminescent image or vice versa.

2. An apparatus as described in claim 1 wherein the computing means also converts said first and second signals from analog to digital form and forwards them to a main computer.

3. The apparatus as described in claim 1 wherein said luminescent means irradiates continuously the object marked with the luminescent fiduciary code.

4. The apparatus as described in claim 1 wherein said luminescent means irradiates noncontinuously the object marked with the luminescent fiduciary code, during two time frames separated in time.

5. A method for enabling an object to be located and moved by a robot comprising the steps of:

irradiating continuously the object marked with a luminescent fiduciary code with ultraviolet or infrared light in accordance with said code, which causes the luminescent fiduciary code to luminesce, said luminescent fiduciary code being situated at a predetermined position on the object;

using videocon camera means for sensing the location of the luminescent fiduciary code on the object by receiving the luminescent energy emitted by the irradiated code marked on the object to receive a scene containing the object including, at times, the luminescence of the fiduciary codes, the camera means having a first signal and having a second signal representative of a luminescent image and non-luminescent image seen by the camera means;

computing in real time the position and orientation of the object from the location of the luminescent fiduciary code that has been sensed;

passing the first and second signals of the camera means to a computing means; and

using the computing means for subtracting a luminescent image from a non-luminescent image or vice versa.

6. The method of claim 5 further comprising the step of:

passing the luminescent light through a filter before the luminescent energy from the code is recovered.

7. The method of claim 5 further comprising the step of:

irradiating another luminescent fiduciary code located on the robot while irradiating said fiduciary code on the object so that information acquired from both the fiduciary codes can be used to enable the robot to interact with the object to move the object.

8. A method for enabling an object to be located and moved by a robot comprising the steps of:

irradiating the object marked with a luminescent fiduciary code with ultraviolet or infrared light for a time interval thus causing the luminescent fiduciary code to luminescent during and after the time interval when the object is irradiated, the luminescence subsiding after the irradiation ceases, said fiduciary code being situated at a predetermined position on the object,

taking a first picture of the object while the fiduciary code thereon is luminescing;

storing the first picture in a computer memory;

taking a second picture of the object when the luminescence of the code has subsided;

storing the second picture in a computer memory;

subtracting the second picture from the first picture or subtracting the first picture from the second picture so only locations in the pictures with luminescent codes are retained; and

computing in real time the position and orientation of the object from the pre-positioned luminescent code on the object.

9. The method of claim 8 wherein the irradiating of the object marked with the fiduciary code is discontinued after storing the first picture in the computer memory.

10. The method of claim 8 wherein the irradiating of the object marked with the fiduciary code is discontinued before storing the first picture in the computer memory.

11. The method of claim 8 wherein the subtracting of pictures of the object is accomplished with two videocons acting simultaneously during two distinct time frames.

12. The method of claim 11 wherein the distinct time frames are time separated.

13. The method of claim 8 further comprising the step of:

irradiating another luminescent fiduciary code located on the robot while irradiating said fiduciary code on the object so that information acquired from both fiduciary codes can be used to enable the robot to interact with the object to move the object.

14. The method of claim 8 wherein the irradiating of the object marked with the fiduciary code is discontinued after a finite time interval followed by a time in which luminescence of the fiduciary code subsides

15. The method of claim 14 wherein the irradiating of the object marked with the fiduciary code is discontinued after the first picture is stored in the computer memory.

16. The method of claim 14 wherein the irradiating of the object marked with the fiduciary code is discontinued before the first picture is stored in the computer memory.

17. The method of claim 14 wherein the irradiating of the object is performed noncontinuously to permit subtraction of two pictures of the luminescent code taken at two separate times.

18. A method for enabling an object to be located and moved by a robot comprising the steps of:

irradiating with ultraviolet or infrared light the object and the robot, each of which are marked with a respective fiduciary code of luminescent markings, said fiduciary code on said object and on said robot being situated at a pre-determined position on the object and at a predetermined position on the robot;

sensing the location of the luminescent marks on the object and on the robot by receiving the luminescent energy emitted by the irradiated luminescent markings of both the object and robot;

computing in real time the position and orientation of the object relative to the robot from the location of the luminescent markings that have been sensed; and

placing the luminescent markings on both the object and the robot into a pre-determined desired position and orientation.

19. The method of claim 18 wherein the irradiating of the object and the robot is performed continuously.

20. The method of claim 18 wherein the irradiating of the object and the robot is performed noncontinuously.

21. A method for enabling an object to be located and moved by a robot comprising the steps of:

irradiating for a specific time interval both the object and the robot, each of which are marked with a luminescent fiduciary code, with ultraviolet or infrared light which causes the luminescent code to luminesce so that the luminescence from the code subsides after the irradiating of the object is discontinued, said fiduciary codes being situated at predetermined positions on the object and on the robot,

taking a first picture of the object and the robot while the fiduciary codes are luminescing;

storing the first picture in a computer memory;

taking a second picture of the object and the robot after the luminescence of the fiduciary codes has subsided;

storing the second picture in a computer memory;

subtracting the second picture from the first picture so only locations in the pictures with luminescent codes are retained;

computing in real time the position and orientation of the object relative to the robot from the pre-positioned luminescent codes on the object and robot; and

placing the fiduciary code on the object and the fiduciary code on the robot into a predetermined desired position and orientation.

22. A method for enabling an object to be located and moved by a robot, comprising the steps of:

taking a first picture of the object before a luminescent fiduciary code thereon is irradiated with ultraviolet or infrared luminescent light, said fiduciary code being situated at a predetermined position on the object;

storing the first picture in a computer memory;

irradiating the object marked with the luminescent fiduciary code with ultraviolet or infrared luminescent light for a time interval thus causing the luminescent fiduciary code to luminesce during and after the time interval when the object is irradiated, the luminescence subsiding after the irradiation ceases;

taking a second picture of the object while the fiduciary code thereon is luminescing as a result of irradiating the object with the ultraviolet or infrared luminescent light as aforesaid;

storing the second picture in a computer memory;

subtracting the second picture from the first picture or subtracting the first picture from the second picture so only locations in the pictures with luminescent codes are retained; and

computing in real time the position an orientation of the object from the pre-positioned luminescent code on the object.

23. A method for enabling an object to be located and moved by a robot, comprising the steps of:

taking a first picture of the object and the robot before fiduciary codes on the object and the robot are irradiated with ultraviolet or infrared luminescent light, said fiduciary codes being situated at predetermined positions on the object and on the robot;

storing the first picture in a computer memory;

irradiating for a specific time interval both the object and the robot each marked with the luminescent fiduciary code as aforesaid with ultraviolet or infrared luminescent light, which causes the luminescent codes to luminesce so that the luminescence from the codes subsides after the irradiating of the object is discontinued;

taking a second picture of the object and the robot during the luminescence of the fiduciary codes before the luminescence has subsided;

storing the second picture in a computer memory;

subtracting the second picture from the first picture or subtracting the first picture from the second picture so only locations in the pictures with luminescent codes are retained;

computing in real time the position and orientation of the object relative to the robot from the pre-positioned luminescent codes on the object and robot; and

placing the fiduciary code on the object and the fiduciary code on the robot into a predetermined desired position and orientation.

24. An apparatus for robotic vision to enable an object to be located and moved by a robot, whether the object is situated correctly or not, comprising:

a luminescent fiduciary code which identifies the position and orientation of the object to be moved by the robot, the fiduciary code being marked on the object and on the robot;

luminescent means in the form of ultraviolet or infrared light for causing the fiduciary code to luminesce;

videocon sensing means for detecting the position of the fiduciary code, the sensing means being aimed at the object and positioned to receive a scene containing the object including, at times, the luminescence of the fiduciary codes, said sensing means having a first and a second signal, which signals are representative of luminescent and non-luminescent images seen by the sensing means;

computing means for determining the position and orientation of any portion of the object in relation to the robot in real time; and

connection means to pass the first and second signals of the sensing means to the computing means where image subtraction is performed by subtracting a luminescent image from a non-luminescent image or vice versa.

25. An apparatus as described in claim 24 wherein the videocon sensing means includes a first videocon producing the first signal and a second videocon producing the second signal and the ultraviolet or infrared light irradiates the luminescent fiduciary code.

26. An apparatus as described in claim 25 wherein the videocon sensing means is a first and a second videocon aimed at the object with each videocon positioned to receive a scene containing the object including, at times, the luminescence of the fiduciary codes, said first and second videocons having said first and said second signal, respectively, which signals are representative of luminescent and non-luminescent images seen by both the first and the second videocons.

27. An apparatus as described in claim 25 wherein the computing means also converts said first and second signals from analog to digital form and forwards them to a main computer.