Pre-screening for robotic work

• US 10,160,117 B2
• Filed: 03/28/2016
• Issued: 12/25/2018
• Est. Priority Date: 06/17/2013
• Status: Active Grant

• Current Assignee: International Electronic Machines Corp.
• Original Assignee: International Electronic Machines Corp.
• Inventors: Mian, Zahid F., Gamache, Ronald W.
• Primary Examiner(s): Jen, Ian
• Application Number:
  US15/082,391
• Publication Number:
  US 20160207203A1
• Time in Patent Office: 1,002 Days
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/45083 Manipulators, robot
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A solution for pre-screening an object for further processing is provided. A pre-screening component can acquire image data of the object and process the image data to identify reference target(s) corresponding to the object, which are visible in the image data. Additionally, the pre-screening component can identify, using the reference target(s), the location of one or more components of the object. The pre-screening component can provide pre-screening data for use in further processing the object, which includes data corresponding to the set of reference targets and the location of the at least one component. A reference target can be, for example, an easily identifiable feature of the object and the component can be relevant for performing an operation on the object.

Claims

1. A system comprising:

a pre-screening component including;

a first set of cameras for acquiring image data of a vehicle, wherein the first set of cameras are configured to acquire image data including at least one component of the vehicle relevant to an operation to be performed on the vehicle; and

a computer system including means for pre-screening the vehicle, the means for pre-screening including;

processing the image data to identify a set of reference targets visible in the image data of the vehicle;

identifying, using the set of reference targets, a location of the at least one component of the vehicle relevant to the operation to be performed on the vehicle, wherein at least one reference target in the set of reference targets comprises a feature of the vehicle; and

generating pre-screening data for the vehicle, wherein the pre-screening data includes data corresponding to the set of reference targets identified in the image data of the vehicle and data for locating the at least one component of the vehicle relative to a location of at least one of the set of reference targets; and

a robotic device configured to perform the operation on the vehicle in an autonomous or semi-autonomous manner, wherein the robotic device includes;

a second set of cameras for acquiring image data of the vehicle distinct from the first set of cameras; and

means for locating the at least one component of the vehicle relevant to the operation, the means for locating including;

locating the at least one of the set of reference targets in the image data acquired by the second set of cameras; and

using the pre-screening data and the located at least one of the set of reference targets to locate the at least one component in the image data acquired by the second set of cameras.

2. The system of claim 1, wherein the pre-screening data further includes a vehicle type for the vehicle, and wherein the robotic device further uses the vehicle type to perform the operation.

3. The system of claim 1, wherein the pre-screening data comprises an equipment definition file for the vehicle including data corresponding to the pre-screening component, data corresponding to a date and time the vehicle was imaged, and data corresponding to a vehicle type.

4. The system of claim 1, wherein the vehicle comprises a rail vehicle.

5. The system of claim 4, wherein the operation comprises at least one of:

a brake bleeding operation or a decoupling operation.

6. The system of claim 4, wherein the pre-screening component further selects one of a plurality of robotic devices to perform the operation on the vehicle based on the pre-screening.

7. The system of claim 1, wherein at least one of the pre-screening component or the robotic device is mobile.

8. The system of claim 1, wherein the first set of cameras includes a first camera configured to acquire image data from a perspective similar to a perspective of a third camera in the second set of cameras of the robotic device and a second camera configured to acquire image data from a perspective substantially different from the perspective of the third camera of the robotic device.

9. A system comprising:

a pre-screening component including;

a set of cameras for acquiring image data of an object, wherein the first set of cameras are configured to acquire image data including at least one component of the object; and

a computer system including means for pre-screening the object, the means for pre-screening including;

processing the image data to identify a set of reference targets visible in the image data of the object, wherein at least one reference target in the set of reference targets comprises a feature of the object;

identifying, using the set of reference targets, a location of the at least one component of the object;

generating pre-screening data for the object, wherein the pre-screening data includes data corresponding to the set of reference targets identified in the image data of the object and data for locating the at least one component of the object relative to a location of at least one of the set of reference targets; and

providing the pre-screening data for use in further processing the object.

10. The system of claim 9, wherein the computer system for pre-screening the object further generates an equipment definition file for the object based on the pre-screening, wherein the equipment definition file includes an entry for each of the set of reference targets and an entry for each of the at least one component of the object.

11. The system of claim 10, further comprising means for evaluating the object, the means for evaluating including:

evaluating a plurality of equipment definition files for at least one of;

the object or a plurality of objects of the same type; and

detecting at least one of;

a set of differences or a set of similarities in the plurality of equipment definition files.

12. The system of claim 11, wherein the means for evaluating further includes identifying an anomaly present in the object based on the evaluating and detecting.

13. The system of claim 9, further comprising a robotic device including means for performing an operation on the object in an autonomous or semi-autonomous manner, the means for performing including:

acquiring image data of the object using a set of cameras located on the robotic device;

processing the image data using the pre-screening data to identify at least one of the set of reference targets visible in the image data of the object;

processing the image data using the pre-screening data and the identified at least one of the set of reference targets to identify at least one component of the object in the image data; and

performing the operation on the at least one component of the object.

14. The system of claim 13, wherein the computer system for pre-screening the object further acquires at least one of:

industry data corresponding to the object or prior pre-screening data corresponding to the object, and includes data corresponding to the at least one of;

industry data or prior pre-screening data, in the pre-screening data, and wherein the robotic device further uses the data corresponding to the at least one of;

industry data or prior pre-screening data, to perform the operation.

15. The system of claim 9, wherein the processing the image data to identify a set of reference targets includes, for at least one reference target, masking a set of areas of the image data incapable of including the at least one reference target.

16. A railyard comprising:

a pre-screening component including;

a set of cameras for acquiring image data of each rail vehicle moving past the pre-screening component, wherein the first set of cameras are configured to acquire image data including at least one component of the rail vehicle relevant to an operation to be performed on the rail vehicle; and

a computer system including means for pre-screening the rail vehicle, the means for pre-screening including;

processing the image data to identify a set of reference targets visible in the image data of the rail vehicle, wherein at least one reference target in the set of reference targets comprises a feature of the rail vehicle;

identifying, using the set of reference targets, a location of the at least one component of the rail vehicle relevant to the operation to be performed on the rail vehicle; and

generating pre-screening data for the vehicle, wherein the pre-screening data includes data corresponding to the set of reference targets and data for locating the at least one component relative to a location of at least one of the set of reference targets; and

a set of robotic devices, wherein at least one robotic device in the set of robotic devices includes means for locating the at least one component of the vehicle relevant to the operation, the means for locating including;

locating the at least one of the set of reference targets in the image data acquired by the robotic device; and

using the pre-screening data and the located at least one of the set of reference targets to locate the at least one component in the image data acquired by the robotic device to perform the operation on the vehicle.

17. The railyard of claim 16, wherein the set of reference targets includes a spring box of a truck of a rail vehicle.

18. The railyard of claim 16, wherein the operation comprises bleeding the brakes of the rail vehicle, and wherein the at least one component comprises a brake rod of the rail vehicle.

19. The railyard of claim 16, wherein the computer system for pre-screening the rail vehicle further includes data corresponding to a set of hints for locating the at least one component in the pre-screening data.

20. The railyard of claim 16, wherein the computer system for pre-screening the rail vehicle further includes data corresponding to a type of the rail vehicle in the pre-screening data.