VISION SYSTEM AND METHOD OF MONITORING SURROUNDINGS OF MACHINE

• US 20160212352A1
• Filed: 01/21/2015
• Published: 07/21/2016
• Est. Priority Date: 01/21/2015
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Sanchez, Rodrigo, Petrany, Peter
• Granted Patent: US 9,667,875 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A vision system for a machine is disclosed. The vision system includes a first set of cameras mounted on the first section and a second set of cameras mounted on the second section of the machine. A controller is coupled to the first set of cameras and the second set of cameras. The controller includes a first and a second image transformation modules to convert a first and a second images captured by the first and the second set of cameras into a first and a second transformed image based at least on a view type, respectively. An image combination module combines the first transformed image and the second transformed image to a combined image with reference to the articulation joint. The image combination module communicates with a display device to display the combined image. The combined image is indicative of surroundings of the first section and the second section of the machine.

Claims

1. A vision system for a machine having a first section, a second section and an articulation joint pivotally connecting the first section to the second section, the vision system comprising:

a first set of cameras mounted on the first section of the machine;

a second set of cameras mounted on the second section of the machine;

a controller communicably coupled to the first set of cameras and the second set of cameras, the controller comprising;

a first image transformation module configured to convert a first image captured by the first set of cameras into a first transformed image based at least on a view type;

a second image transformation module configured to convert a second image captured by the second set of cameras into a second transformed image based at least on a view type; and

an image combination module configured to receive the first transformed image from the first image transformation module and the second transformed image from the second image transformation module, the image combination module further configured to combine the first transformed image and the second transformed image to obtain a combined image with reference to the articulation joint, wherein the combined image is indicative of surroundings of the first section and the second section of the machine; and

a display device communicably coupled to the controller, the display device configured to display the combined image.

2. The vision system of claim 1, wherein the view type is one of a three dimensional view, a perspective view, a two dimensional view, a top surround view and a side view.

3. The vision system of claim 2, wherein the controller further comprises:

a first image selection module configured to select the view type based on a user input, the first image selection module configured to be in communication with the first image transformation module; and

a second image selection module configured to select the view type based on a user input, the second image selection module configured to be in communication with the second image transformation module.

4. The vision system of claim 2, wherein the image combination module is further configured to combine the first image and the second image with reference to the articulation joint based on the view type.

5. The vision system of claim 1, wherein the first set of cameras comprises:

a front camera mounted adjacent to a front end of the first section; and

a first side camera mounted adjacent to each of the sides of the first section.

6. The vision system of claim 1, wherein the second set of cameras comprises:

a rear camera mounted adjacent to a rear end of the second section; and

a second side camera mounted adjacent to each of the sides of the second section.

7. The vision system of claim 1, wherein the controller further comprises:

a first image correction module configured to correct the first image captured by the first set of cameras based on a calibration parameter associated with the first set of cameras; and

a second image correction module configured to correct the second image captured by the second set of cameras based on a calibration parameter associated with the second set of cameras.

8. The vision system of claim 1, wherein the display device is disposed within an operator cab of the machine.

9. A machine comprising:

a first section comprising a first set of ground engaging members;

a second section comprising a second set of ground engaging members;

an articulation joint configured to pivotally couple the first section to the second section; and

a vision system comprising;

a first set of cameras mounted on the first section;

a second set of cameras mounted on the second section;

a controller communicably coupled to the first set of cameras and the second set of cameras, the controller comprising;

a first image transformation module configured to convert a first image captured by the first set of cameras into a first transformed image based at least on a view type;

a second image transformation module configured to convert a second image captured by the second set of cameras into a second transformed image based at least on a view type; and

an image combination module configured to receive the first transformed image from the first image transformation module and the second transformed image from the second image transformation module, the image combination module further configured to combine the first transformed image and the second transformed image to obtain a combined image with reference to the articulation member, wherein the combined image is indicative of surroundings of the first section and the second section of the machine; and

a display device communicably coupled to the controller, the display device configured to display the combined image.

10. The machine of claim 9, wherein the view type is one of a three dimensional view, a perspective view, a two dimensional view, a top surround view and a side view.

11. The machine of claim 10, wherein the controller further comprises:

a first image selection module configured to select the view type based on a user input, the first image selection module configured to be in communication with the first image transformation module; and

a second image selection module configured to select the view type based on a user input, the second image selection module configured to be in communication with the second image transformation module.

12. The machine of claim 10, wherein the image combination module is further configured to combine the first image and the second image with reference to the articulation joint based on the view type.

13. The machine of claim 9, wherein the first set of cameras comprises:

a front camera mounted adjacent to a front end of the first section; and

a first side camera mounted adjacent to each of the sides of the first section.

14. The machine of claim 9, wherein the second set of cameras comprises:

a rear camera mounted adjacent to a rear end of the second section; and

a second side camera mounted adjacent to each of the sides of the second section.

15. The machine of claim 9, wherein the controller further comprises:

a first image correction module configured to correct the first image captured by the first set of cameras based on a calibration parameter associated with the first set of cameras; and

a second image correction module configured to correct the second image captured by the second set of cameras based on a calibration parameter associated with the second set of cameras.

16. The machine of claim 9, wherein the first section further comprises an operator cab, and wherein the display device is disposed within the operator cab.

17. A method of monitoring surroundings of a machine having a first section, a second section and an articulation joint pivotally connecting the first section to the second section, the method comprising:

capturing, via a first set of cameras, a first image, wherein the first set of cameras is mounted on the first section of the machine;

capturing, via a second set of cameras, a second image, wherein the second set of cameras is mounted on the second section of the machine;

converting the first image captured by the first set of cameras into a first transformed image based at least on a view type;

converting the second image captured by the second set of cameras into a second transformed image based at least on a view type;

combining the first transformed image and the second transformed image to obtain a combined image with reference to the articulation joint, wherein the combined image is indicative of surroundings of the first section and the second section of the machine; and

displaying, via a display device, the combined image.

18. The method of claim 17 further comprising:

correcting the first image captured by the first set of cameras based on a calibration parameter associated with the first set of cameras; and

correcting the second image by the second set of cameras based on a calibration parameter associated with the second set of cameras.

19. The method of claim 17 further comprising selecting the view type from one of a three dimensional view, a perspective view, a two dimensional view, a top surround view and a side view.