Surroundings monitoring system for a vehicle

• US 9,598,012 B2
• Filed: 03/11/2014
• Issued: 03/21/2017
• Est. Priority Date: 03/11/2014
• Status: Active Grant

• Current Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Nagata, Katsumi, Lota, Charan
• Primary Examiner(s): YENKE, BRIAN P
• Application Number:
  US14/205,296
• Publication Number:
  US 20150258935A1
• Time in Patent Office: 1,106 Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surroundings monitoring system includes a first sensor subsystem operable to provide image data from around the vehicle; a second sensor subsystem operable to provide non-image data from around the vehicle; a control subsystem in communication with the first sensor subsystem and the second sensor subsystem, the control subsystem operable to generate manipulated image data symbology from the image data and arrange the manipulated image data symbology in response to the non-image data to generate a driver perspective view; and a display in communication with the control subsystem to display the driver perspective view. A method of monitoring surroundings for use in a vehicle includes obtaining image data from around the vehicle; obtaining non-image data from around the vehicle; manipulating the image data in response to the non-image data to generate a driver perspective view; and displaying the driver perspective view.

Claims

1. A surroundings monitoring system for use in a vehicle, comprising:

a first sensor subsystem operable to provide image data from around the vehicle;

a second sensor subsystem operable to provide non-image data from around the vehicle;

a control subsystem in communication with the first sensor subsystem and the second sensor subsystem, the control subsystem operable to;

when a view from the first sensor subsystem is obscured with respect to targets around the vehicle, generate representative symbology that schematically portrays the targets,

when a view from the first sensor subsystem is unobscured with respect to targets around the vehicle, generate, from images of the targets in the image data, manipulated image data symbology that portrays only the targets, from a driver perspective, and

based on ranging data for the targets in the non-image data, horizontally spatially arrange the generated representative and manipulated image data symbology around a central vehicle symbol in an overhead view around the vehicle that portrays, via the horizontal spatial arrangement of the generated representative and manipulated image data symbology around the central vehicle symbol, the relative positions of the targets around the vehicle in a horizontal plane; and

a display subsystem in communication with the control subsystem and located within the vehicle operable to display the overhead view around the vehicle.

2. The system as recited in claim 1, wherein the image data includes video.

3. The system as recited in claim 1, wherein the first sensor subsystem includes a multiple of first sensors that respectively provide image data from each of a front, passenger side, rear, and driver side of the vehicle.

4. The system as recited in claim 3, wherein each of the multiple of first sensors includes at least one of a camera, a charge coupled device, a forward looking infrared, and a thermal system.

5. The system as recited in claim 1, wherein the second sensor subsystem includes a multiple of second sensors that respectively provide non-image data from each of a front, passenger side, rear, and driver side of the vehicle.

6. The system as recited in claim 1, wherein each of the multiple of second sensors includes at least one of RADAR, SONAR, laser, and LIDAR.

7. The system as recited in claim 1, wherein the display subsystem is located within a human-machine interface within the vehicle.

8. The system as recited in claim 1, wherein the display subsystem is located on a windshield within the vehicle.

9. The system as recited in claim 1, wherein the control subsystem is operable to horizontally spatially arrange the generated representative and manipulated image data symbology to focus upon a target in response to the non-image data.

10. The system as recited in claim 1, wherein the control subsystem is operable to extract the images of the targets from their surroundings in the image data to generate the manipulated image data symbology.

11. The system as recited in claim 1, wherein the overhead view is a three-hundred and sixty degree overhead view around the vehicle.

12. A method of monitoring surroundings for use in a vehicle, comprising:

obtaining image data from around the vehicle from a first sensor subsystem;

obtaining non-image data from around the vehicle from a second sensor subsystem;

when a view from the first sensor subsystem is obscured with respect to targets around the vehicle, generate representative symbology that schematically portrays the targets, when a view from the first sensor subsystem is unobscured with respect to targets around the vehicle, generating, from images of the targets in the image data, manipulated image data symbology that portrays only the targets, from a driver perspective;

based on ranging data for the targets in the non-image data, horizontally spatially arranging the generated representative and manipulated image data symbology around a central vehicle symbol in an overhead view around the vehicle that portrays, via the horizontal spatial arrangement of the generated representative and manipulated image data symbology around the central vehicle symbol, the relative positions of the targets around the vehicle in a horizontal plane; and

displaying the overhead view around the vehicle.

13. The method as recited in claim 12, further comprising:

extracting the images of the targets from their surroundings in the image data to generate the manipulated image data symbology.

14. The method as recited in claim 12, wherein the overhead view is a three-hundred and sixty degree overhead view around the vehicle.