Image processing system using unified images

• US 9,167,214 B2
• Filed: 01/18/2013
• Issued: 10/20/2015
• Est. Priority Date: 01/18/2013
• Status: Active Grant

• Current Assignee: Caterpillar Incorporated
• Original Assignee: Caterpillar Incorporated
• Inventors: Kriel, Bradley Scott, Morris, Daniel, Chang, Hong
• Primary Examiner(s): Lee, Michael
• Application Number:
  US13/745,641
• Publication Number:
  US 20140204215A1
• Time in Patent Office: 1,005 Days
• Field of Search: 348/148
• US Class Current: 1/1
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image processing system has a plurality of cameras and a display that are mounted on a machine. The plurality of cameras are configured to generate image data for an environment of the machine. The image processing system also has a processor connected to the plurality of cameras and the display. The processor is configured to access the image data from the plurality of cameras, access parameters associated with the plurality of cameras, generate a unified image by combining the image data from the plurality of cameras based at least in part on the parameters, access state data associated with the machine, select a portion of the unified image based at least in part on the state data, and render the portion of the unified image on the display.

Claims

1. An image processing system comprising:

a plurality of cameras mounted on a machine, the plurality of cameras configured to generate image data for an environment of the machine;

a display mounted on the machine;

a processor connected to the plurality of cameras and the display, the processor configured to;

access the image data from the plurality of cameras;

access parameters associated with the plurality of cameras, the parameters including a plurality of image boundaries expressed as one or more angles from one or more down-vectors of the cameras;

generate a unified image by combining the image data from the plurality of cameras based at least in part on the parameters;

access state data associated with the machine;

select a portion of the unified image based at least in part on the state data; and

render the selected portion of the unified image on the display.

2. The image processing system of claim 1, wherein the state data includes a direction of the machine'"'"'s motion and wherein selecting the portion of the unified image is based at least in part on the direction of the machine'"'"'s motion.

3. The image processing system of claim 1, wherein the state data includes an indication that the machine is not in motion and wherein the selected portion of the unified image includes all of the unified image.

4. The image processing system claim 1, wherein:

the processor is further configured to detect an object in the environment of the machine; and

the selected portion of the unified image includes the detected object.

5. The image processing system of claim 4, wherein the processor is further configured to generate an alert when a collision is likely to occur between the detected object and the machine.

6. The image processing system of claim 1, further comprising:

a radar device mounted to the machine and configured to generate radar data; and

wherein the processor is further configured to;

access the radar data from the radar device; and

,

detect an object in the accessed radar data, and

wherein the selected portion of the unified image includes the detected object.

7. The image processing system of claim 1, wherein the parameters include a position of the plurality of cameras with respect to the machine.

8. The image processing system of claim 1, wherein the unified image data is mapped to a hemi-spherical image map.

9. The image processing system of claim 1, wherein the selected portion of the unified image is rectangular.

10. The image processing system of claim 1, wherein:

the processor is further configured to detect an input describing an operator preference for viewing an area of the environment of the machine, and

the selected portion of the unified image includes the area of the environment.

11. A method for displaying an image comprising:

accessing image data from a plurality of cameras mounted on a machine;

accessing parameters associated with the plurality of cameras, the parameters including a plurality of image boundaries expressed as one or more angles from one or more down-vectors of the cameras;

generating a unified image by combining the image data from the plurality of cameras based on the parameters;

accessing state data associated with the machine;

selecting a portion of the unified image data based at least in part on the state data; and

rendering the selected portion of the unified image on a display mounted on the machine.

12. The method of claim 11, wherein the state data includes a direction of the machine'"'"'s motion and wherein selecting the portion of the unified image is based at least in part on the direction of the machine'"'"'s motion.

13. The method of claim 11, wherein the state data includes an indication that the machine is not in motion and wherein the selected portion of the unified image includes all of the unified image.

14. The method of claim 11 further including detecting an object in an environment of the machine and wherein the selected portion of the unified image includes the detected object.

15. The method of claim 11, further including:

accessing radar data from a radar device mounted on the machine; and

detecting an object an environment of the machine using the radar data,

wherein the selected portion of the unified image includes the detected object.

16. The method of claim 11, wherein the parameters include a position of the plurality of cameras with respect to the machine.

17. The method of claim 11, wherein the unified image data is mapped to a hemi-spherical image map.

18. The method of claim 11 wherein the selected portion of the unified image is rectangular.

19. The method of claim 11, further including:

detecting an input describing an operator preference for viewing an area of an environment of the machine and wherein the selected portion of the unified image includes the area of the environment.

20. A machine comprising:

a cabin,

a display disposed within the cabin;

a frame;

a plurality of cameras connected to the frame, the plurality of cameras configured to generate image data for an environment of the machine;

a processor in communication with the plurality of cameras and the display, the processor configured to;

access the image data from the plurality of cameras;

access parameters for the plurality of cameras, the parameters including a plurality of image boundaries expressed as one or more angles from one or more down-vectors of the cameras;

generate a unified image by combining, based on the parameters, the image data from the plurality of cameras;

detect an object in the environment of the machine;

access state data describing associated with the machine, the state data including a direction of the machine'"'"'s motion;

select a portion of the unified image based at least in part on the state data and a location of the detected object; and

,

render the selected portion of the unified image on the display.