System and method for work tool recognition

• US 10,011,976 B1
• Filed: 01/03/2017
• Issued: 07/03/2018
• Est. Priority Date: 01/03/2017
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Forcash, Joseph, Mianzo, Lawrence Andrew, Rybski, Paul Edmund
• Primary Examiner(s): To, Tuan C
• Application Number:
  US15/396,939
• Publication Number:
  US 20180187398A1
• Time in Patent Office: 546 Days
• CPC Class Codes: E02F 3/96 with arrangements for alternate or simultaneous use of different digging elements E02F3/7604, E02F3/769, E02F3/78 take precedence; quick-acting devices to connect tools to arms or booms E02F3/3609, for arms to tractors or the like E02F3/627
• CPC Class Codes: E02F 9/264 Sensors and their calibration for indicating the position of the work tool
• CPC Class Codes: E02F 9/265 with follow-up actions (e.g. control signals sent to actuate the work tool)
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

A work tool recognition system for a work tool coupled to a machine is provided. The system includes an image capturing assembly configured to capture an image feed of the work tool. A controller is communicably coupled to the image capturing assembly. The controller receives the image feed of the work tool from the image capturing assembly. The controller extracts a plurality of features of the work tool from the image feed. The controller classifies the extracted image and determine a confidence value. The controller estimates a size of the work tool. The controller estimates a position and an orientation of the work tool. The controller determines verification features of the work tool and compares the image feed of the work tool with a predetermined dataset based on the classification. The controller identifies a type of the work tool based on the comparison.

Claims

1. A work tool recognition system for a work tool coupled to a machine, the work tool recognition system comprising:

an image capturing assembly associated with the work tool, the image capturing assembly configured to capture an image feed of the work tool; and

a controller communicably coupled to the image capturing assembly, the controller configured to;

receive the image feed of the work tool from the image capturing assembly;

extract a plurality of features of the work tool from the image feed using image analysis;

classify the extracted image and determine a confidence value associated with the classification;

estimate a size of the work tool based on information associated with the image capturing assembly;

estimate a position and an orientation of the work tool;

determine verification features of the work tool based on the confidence value and the estimated size, position, and orientation;

compare the image feed of the work tool with a predetermined dataset based on the classification; and

identify a type of the work tool based on the comparison.

2. The work tool recognition system of claim 1, wherein the controller is further configured to display a notification of the identified type of the work tool.

3. The work tool recognition system of claim 1, wherein the controller is further configured to select a control algorithm for operating the identified type of the work tool.

4. The work tool recognition system of claim 1, wherein the image capturing assembly is mounted on an operator cab of the machine.

5. The work tool recognition system of claim 1, wherein the controller is configured to receive signals from a position detection sensor for estimating the position and the orientation of the work tool.

6. The work tool recognition system of claim 1, wherein the controller is configured to determine a position of cylinders attached to the work tool by image analysis for estimating the position and the orientation of the work tool.

7. The work tool recognition system of claim 1, wherein the controller is configured to prompt a user to provide an input for assisting in identifying the type of the work tool if the determined confidence value is lesser than a predetermined threshold.

8. The work tool recognition system of claim 1, wherein the system is activated on start-up of the machine.

9. The work tool recognition system of claim 1, wherein the system is activated based on an engagement of a quick coupler associated with the work tool for coupling the work tool with the machine.

10. The work tool recognition system of claim 1, wherein the information associated with the image capturing assembly includes at least one of a position, an orientation, and image capturing attributes of the image capturing assembly.

11. The work tool recognition system of claim 1, wherein the plurality of features include at least one of colors, gradients, textures, edges, and shapes associated with the work tool.

12. The work tool recognition system of claim 1, wherein the verification features include at least one of number of teeth, external markings, and dimensions of the work tool.

13. The work tool recognition system of claim 1, wherein the predetermined dataset includes data related to different types of the work tool.

14. A method for recognizing a work tool coupled to a machine, the method comprising:

receiving, by a controller, an image feed of the work tool from an image capturing assembly;

extracting, by the controller, a plurality of features of the work tool from the image feed using image analysis;

classifying, by the controller, the extracted image and determining a confidence value associated with the classification;

estimating, by the controller, a size of the work tool based on information associated with the image capturing assembly;

estimating, by the controller, a position and an orientation of the work tool;

determining, by the controller, verification features of the work tool based on the confidence value and the estimated size, position, and orientation;

comparing, by the controller, the classification of the work tool with a predetermined dataset; and

identifying, by the controller, a type of the work tool based on the comparison.

15. The method of claim 14 further comprising:

displaying, by the controller, a notification of the identified type of the work tool.

16. The method of claim 14 further comprising:

selecting, by the controller, a control algorithm for operating the identified type of the work tool.

17. The method of claim 14 further comprising:

prompting, by the controller, a user to provide an input for assisting in identifying the type of the work tool if the determined confidence value is lesser than a predetermined threshold.

18. A machine comprising:

a work tool configured to be coupled to the machine; and

a work tool recognition system for the work tool, the work tool recognition system comprising;

an image capturing assembly associated with the work tool, the image capturing assembly configured to capture an image feed of the work tool; and

a controller communicably coupled to the image capturing assembly, the controller configured to;

receive the image feed of the work tool from the image capturing assembly;

extract a plurality of features of the work tool from the image feed using image analysis;

classify the extracted image and determine a confidence value associated with the classification;

estimate a size of the work tool based on information associated with the image capturing assembly;

estimate a position and an orientation of the work tool;

determine verification features of the work tool based on the confidence value and the estimated size, position, and orientation;

compare the image feed of the work tool with a predetermined dataset based on the classification; and

identify a type of the work tool based on the comparison.

19. The machine of claim 18, wherein the controller is further configured to display a notification of the identified type of the work tool.

20. The machine of claim 18, wherein the controller is further configured to select a control algorithm for operating the identified type of the work tool.