Ego-vehicles, systems, and methods for monitoring target objects

• US 9,969,267 B2
• Filed: 06/29/2016
• Issued: 05/15/2018
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: McNew, John-Michael
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US15/196,401
• Publication Number:
  US 20180001766A1
• Time in Patent Office: 685 Days
• Field of Search: 701 65
• CPC Class Codes: B60K 2360/167 Vehicle dynamics information
• CPC Class Codes: B60K 2360/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2360/5915 Inter vehicle communication
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G01S 13/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 7/22 Producing cursor lines and indicia by electronic means
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

An ego-vehicle for displaying a behavior of a target object in a spatio-temporal manner may include one or more processors. One or more memory modules are communicatively coupled to the one or more processors. A display is communicatively coupled to the one or more processors. One or more sensors are communicatively coupled to the one or more processors. Machine readable instructions are stored in the one or more memory modules and cause the one or more processors to display on the display an object indicator associated with a position of a target object relative to the ego-vehicle, wherein the object indicator depicts a spatio-temporal patterning indicating the behavior of the target object.

Claims

1. An ego-vehicle for displaying a behavior of a target object in a spatio-temporal manner, the ego-vehicle comprising:

one or more processors;

one or more memory modules communicatively coupled to the one or more processors;

a display communicatively coupled to the one or more processors;

one or more sensors communicatively coupled to the one or more processors; and

machine readable instructions stored in the one or more memory modules that cause the one or more processors to, based on a signal from the one or more sensors, display on the display an object indicator associated with a position of the target object relative to the ego-vehicle, wherein;

the object indicator depicts a spatio-temporal patterning indicating the behavior of the target object; and

the spatio-temporal patterning of the object indicator comprises at least one of a light pattern and waveform, the at least one of the light pattern and waveform having a frequency proportional to a speed of the target object.

2. The ego-vehicle of claim 1, wherein the machine readable instructions cause the one or more processors to adjust the spatio-temporal patterning of the object indicator in both time and space to indicate the behavior of the target object at any given time.

3. The ego-vehicle of claim 1, wherein:

the spatio-temporal patterning of the object indicator further indicates a direction the target object is traveling; and

the machine readable instructions further cause the one or more processors to move the object indicator across the display to correspond with a movement of the target object within a vicinity of the ego-vehicle.

4. The ego-vehicle of claim 1, wherein the spatio-temporal patterning of the object indicator indicates the speed and direction of travel of the target object relative to a speed and direction of travel of the ego-vehicle.

5. The ego-vehicle of claim 1, wherein the spatio-temporal patterning of the object indicator indicates an absolute speed and direction of travel of the target object.

6. The ego-vehicle of claim 5, wherein:

the machine readable instructions further cause the one or more processors to display an ego-vehicle indicator on the display, wherein the ego-vehicle indicator depicts a spatio-temporal patterning indicating at least the absolute speed of the ego-vehicle; and

the object indicator is displayed adjacent to the ego-vehicle indicator at a position on the display corresponding to the position of the target object within a vicinity of the ego-vehicle.

7. The ego-vehicle of claim 6, wherein:

the spatio-temporal patterning of the ego-vehicle indicator comprises at least one of a light pattern and waveform, the at least one of the light pattern and waveform having a frequency proportional to the absolute speed of the ego-vehicle and a direction of flow indicative of a direction of travel of the ego-vehicle; and

the at least one of the light pattern and waveform of the spatio-temporal patterning of the object indicator has a frequency proportional to the absolute speed of the target object and a direction of flow indicative of the direction of travel of the target object.

8. The ego-vehicle of claim 1, wherein the light pattern and waveform of the spatio-temporal patterning of the object indicator has a direction of flow indicative of a direction of travel of the target object relative to a direction of travel of the ego-vehicle.

9. The ego-vehicle of claim 1, wherein the frequency of the spatio-temporal patterning of the object indicator adjusts to correspond to changes in the speed of the target object.

10. The ego-vehicle of claim 1, wherein the machine readable instructions further cause the one or more processors to identify a target object type, wherein the object indicator indicates the target object type.

11. The ego-vehicle of claim 1, wherein the ego-vehicle is an autonomous vehicle.

12. The ego-vehicle of claim 1, wherein the machine readable instructions further cause the one or more processors to:

identify road parameters, wherein the road parameters include at least a lane of a road; and

display the road parameters on the display, wherein the object indicator is displayed in the lane corresponding to the lane the target object is in.

13. The ego-vehicle of claim 1, wherein the display is at least one of a heads-up display, an instrument cluster display, a navigation display, and a mobile device display.

14. A system for displaying a behavior of a target object in a spatio-temporal manner relative to the system, the system comprising:

one or more processors;

one or more memory modules communicatively coupled to the one or more processors;

a display communicatively coupled to the one or more processors;

one or more sensors communicatively coupled to the one or more processors; and

machine readable instructions stored in the one or more memory modules that cause the one or more processors to, based on a signal from the one or more sensors, display on the display an object indicator associated with a position of the target object relative to the system, wherein;

the object indicator depicts a spatio-temporal patterning indicating the behavior of the target object relative to the system; and

the spatio-temporal patterning of the object indicator comprises at least one of a light pattern and waveform, the at least one of the light pattern and waveform having a frequency proportional to a speed of the target object.

15. The system of claim 14, wherein:

the at least one of the light pattern and waveform of the spatio-temporal patterning of the object indicator has a direction of flow indicative of a direction of travel of the target object relative to a direction of travel of the system; and

the frequency of the spatio-temporal patterning adjusts to correspond to the changes in the speed and the direction of travel of the target object relative to the system.

16. The system of claim 14, wherein the one or more sensors include at least one of a camera, LiDAR, RADAR, and vehicle-to-vehicle communication.

17. The system of claim 14, wherein the machine readable instructions further cause the one or more processors to move the object indicator across the display to correspond with a movement of the target object within a vicinity of the system.

18. A method of displaying a behavior of a target object in a spatio-temporal manner relative to an ego-vehicle comprising:

detecting, with one or more sensors, the target object in a vicinity of the ego-vehicle;

monitoring a speed, direction of travel, and position of the target object relative to the ego-vehicle;

displaying, with one or more processors, an object indicator associated with the position of the target object relative to the ego-vehicle on a display, wherein;

the object indicator depicts a spatio-temporal patterning indicating the behavior of the target object; and

the spatio-temporal patterning of the object indicator comprises at least one of a light pattern and waveform, the at least one of the light pattern and waveform having a frequency proportional to a speed of the target object; and

adjusting the spatio-temporal patterning of the object indicator on the display to correspond with the behavior of the target object within the vicinity of the ego-vehicle.

19. The method of claim 18, wherein:

the at least one of the light pattern and waveform of the spatio-temporal patterning of the object indicator has a direction of flow indicative of a direction of travel of the target object relative to a direction of travel of the ego-vehicle.

20. The method of claim 18, further comprising:

identifying road parameters, wherein road parameters include at least a lane of a road; and

displaying virtual road parameters on the display, wherein;

the virtual road parameters include at least a virtual lane corresponding to the lane of the road; and

the object indicator of the target object is displayed in the virtual lane corresponding to the lane the target object is in.