Vehicle operation assistance information management for autonomous vehicle control transfer

• US 9,937,795 B2
• Filed: 02/29/2016
• Issued: 04/10/2018
• Est. Priority Date: 06/24/2015
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan North America Incorporated (Nissan Motor Co., Ltd.)
• Inventors: Konet, Heather, Goudy, Roy, Christensen, Andrew
• Primary Examiner(s): SOSANYA, OBAFEMI OLUDAYO
• Application Number:
  US15/056,744
• Publication Number:
  US 20160375767A1
• Time in Patent Office: 771 Days
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/285 for improving awareness by directing driver's gaze direction or eye points
• CPC Class Codes: B60K 35/81 for controlling displays

Abstract

A vehicle implementing vehicle operation assistance information management for autonomous vehicle control transfer may identify a vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer, and may, on a condition that an immanency for the vehicle operation assistance information item is within a maximum relevant immanence, determine an urgency based on the immanency, and present a representation of the vehicle operation assistance information item, which may include controlling a primary graphical display portion of the vehicle to present a graphical representation of the vehicle operation assistance information item, controlling a secondary graphical display portion of the vehicle to present the graphical representation, or a combination thereof. The vehicle may, control an auditory presentation device of the vehicle to present an auditory representation of the vehicle operation assistance information item.

Claims

1. A vehicle configured to traverse a portion of a vehicle transportation network, the vehicle comprising:

a primary graphical display portion;

a secondary graphical display portion;

an auditory presentation device; and

a processor configured to execute instructions stored on a non-transitory computer readable medium to;

identify a first vehicle operation assistance information item, the first vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer,

on a condition that an immanency for the first vehicle operation assistance information item exceeds a maximum relevant immanence, store information representing the first vehicle operation assistance information item, and

on a condition that the immanency is within the maximum relevant immanence;

determine a first urgency based on the immanency; and

present a representation of the first vehicle operation assistance information item, wherein presenting the representation of the first vehicle operation assistance information item includes;

on a condition that the first urgency indicates a warning urgency or an advisory urgency;

 

controlling the primary graphical display portion to present a graphical representation of the first vehicle operation assistance information item, wherein controlling the primary graphical display portion to present the graphical representation includes controlling the primary graphical display portion to present a first portion of the graphical representation; and

 

controlling the secondary graphical display portion to present the graphical representation, wherein controlling the secondary graphical display portion to present the graphical representation includes controlling the secondary graphical display portion to present a second portion of the graphical representation,

on a condition that the first urgency indicates an advance urgency, controlling the secondary graphical display portion to present the graphical representation of the first vehicle operation assistance information item, and

on a condition that the first urgency indicates a warning urgency or an advance urgency, controlling the auditory presentation device to present an auditory representation of the first vehicle operation assistance information item.

2. The vehicle of claim 1, further comprising:

a first graphical display unit, wherein the first graphical display unit includes the first graphical display portion; and

a second graphical display unit, wherein the second graphical display unit includes the second graphical display portion, and wherein the second graphical display unit is physically independent of the first graphical display unit.

3. The vehicle of claim 1, wherein the primary graphical display portion is spatially located within the vehicle along a first angular offset from a center of a primary field of view of the driver of the vehicle, and wherein the secondary graphical display portion is spatially located within the vehicle along a second angular offset from the center of the primary field of view of the driver of the vehicle, wherein the second angular offset is greater than the first angular offset, such that the primary graphical display portion is visually distinct from the secondary graphical display portion.

4. The vehicle of claim 1, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to determine the first urgency by:

on a condition that the first vehicle operation assistance information item includes an urgency value, identifying the urgency value as the first urgency; and

on a condition that the first vehicle operation assistance information item omits the urgency value;

on a condition that the immanency exceeds an advisory cost, determining that the first urgency is an advance urgency, and

on a condition that the immanency is within the advisory cost;

on a condition that the immanency exceeds a warning cost, determining that the first urgency is the advisory urgency; and

on a condition that the immanency is within the warning cost, determining that the first urgency is the warning urgency.

5. The vehicle of claim 1, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

identify the first vehicle operation assistance information item in response to at least one of;

receiving information indicating a vehicle operation control transfer request that includes a driver initiated vehicle operation control transfer request, or

identifying information indicating a vehicle operation control transfer request that includes a vehicle initiated vehicle operation control transfer request.

6. The vehicle of claim 5, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to identify the information indicating the vehicle operation control transfer request that includes the vehicle initiated vehicle operation control transfer request, in response to at least one of:

receiving, via a wireless electronic communication link, external vehicle operation assistance information, wherein the external vehicle operation assistance information includes at least one of an infrastructure message from an external infrastructure device or an inter-vehicle message from another vehicle;

identifying vehicle transportation network information representing a portion of a vehicle transportation network in response to traversing a part of the portion of the vehicle transportation network;

or

receiving sensor information from a sensor, wherein the vehicle includes the sensor.

7. The vehicle of claim 6, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to determine the immanency by:

on a condition that the first vehicle operation assistance information item includes an immanency value, identifying the immanency value as the immanency; and

on a condition that the first vehicle operation assistance information item omits the immanency value;

on a condition that a probability that a current expected path for the vehicle intersects with a target geospatial location in the vehicle transportation network indicated by the first vehicle operation assistance information item is within a minimum relevance threshold, identifying the immanency such that the immanency exceeds the maximum relevant immanence, and

on a condition that the probability that the current expected path for the vehicle intersects with the target geospatial location exceeds the minimum relevance threshold, identifying, as the immanency, a first temporal cost for the vehicle to traverse the vehicle transportation network from a current geospatial location of the vehicle in the vehicle transportation network to the target geospatial location.

8. The vehicle of claim 7, wherein, on a condition that identifying the first vehicle operation assistance information item includes identifying vehicle transportation network information representing the portion of the vehicle transportation network, the processor is configured to execute instructions stored on a non-transitory computer readable medium to determine the target geospatial location by:

identifying a route for the vehicle to traverse the vehicle transportation network from an origin to a destination, wherein the route includes the current geospatial location of the vehicle and the target geospatial location;

identifying a portion of the route as a driver control portion; and

identifying a geospatial location in the vehicle transportation network of the driver control portion that is proximate to the current geo spatial location of the vehicle along the route as the target geospatial location.

9. The vehicle of claim 5, wherein:

on a condition that the identifying the first vehicle operation assistance information item includes receiving information indicating the vehicle operation control transfer request that includes the driver initiated vehicle operation control transfer request, the vehicle operation control transfer request includes;

a request to transfer vehicle operation control from the driver to an autonomous vehicle control unit of the vehicle, or

a request to transfer vehicle operation control from an autonomous vehicle control unit of the vehicle to the driver; and

on a condition that the identifying the first vehicle operation assistance information item includes identifying information indicating the vehicle operation control transfer request that includes the vehicle initiated vehicle operation control transfer request, the vehicle operation control transfer request includes the request to transfer vehicle operation control from the autonomous vehicle control unit of the vehicle to the driver.

10. The vehicle of claim 1, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to identify a vehicle operation control metric for the expected vehicle operation control transfer in response to identifying the first vehicle operation assistance information item.

11. The vehicle of claim 10, wherein the vehicle operation control metric is a steering control metric, an acceleration control metric, or a situational awareness control metric.

12. The vehicle of claim 10, wherein controlling the primary graphical display portion to present the first portion of the graphical representation includes:

on a condition that the first urgency is the advisory urgency, controlling the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes an advisory pictographic representation of the vehicle operation control metric and a primary representation of the immanency proximate to the advisory pictographic representation of the vehicle operation control metric in the first portion of the graphical representation; and

on a condition that the first urgency is the warning urgency;

on a condition that the vehicle operation control transfer request includes the request to transfer vehicle operation control from the autonomous vehicle control unit of the vehicle to the driver;

on a condition that a driver status for the vehicle operation control metric indicates a disengaged driver status, controlling the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes a warning pictographic representation of the vehicle operation control metric.

13. The vehicle of claim 12, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

on a condition that the immanency is within a minimum relevant immanence;

on a condition that the driver status for the vehicle operation control metric indicates the disengaged driver status;

identify a secondary autonomous vehicle control action responsive to the vehicle operation assistance information item;

control the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes a representation of the secondary autonomous vehicle control action; and

control the vehicle in accordance with the secondary autonomous vehicle control action, and

on a condition that the driver status for the vehicle operation control metric indicates an engaged driver status;

transfer control of the vehicle operation control; and

in response to transferring control of the vehicle operation control, control the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes a transferred pictographic representation of the vehicle operation control metric.

14. The vehicle of claim 10, wherein controlling the secondary graphical display portion to present the graphical representation includes:

on a condition that the first urgency is the advance urgency, controlling the secondary graphical display portion to present the graphical representation such that the graphical representation includes a verbal representation of the expected vehicle operation control transfer;

on a condition that the first urgency is the advisory urgency, controlling the secondary graphical display portion to present the second portion of the graphical representation such that the second portion of the graphical representation includes a verbal representation of the vehicle operation control metric, and such that the second portion of the graphical representation includes a secondary representation of the immanency proximate to the verbal representation; and

on a condition that the first urgency is the warning urgency, controlling the secondary graphical display portion to present the second portion of the graphical representation such that the second portion of the graphical representation includes a verbal representation of the vehicle operation control metric.

15. A vehicle configured to traverse a portion of a vehicle transportation network, the vehicle comprising:

a primary graphical display portion;

a secondary graphical display portion;

a sensor;

a trajectory controller; and

a processor configured to execute instructions stored on a non-transitory computer readable medium to;

identify a vehicle operation assistance information item, the vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer, and

utilize the vehicle operation assistance information item to traverse the portion of the vehicle transportation network, wherein utilizing the vehicle operation assistance information item to traverse the portion of the vehicle transportation network includes;

on a condition that an urgency for the vehicle operation assistance information item indicates a warning urgency;

identifying a vehicle operation control metric for the expected vehicle operation control transfer, wherein the vehicle operation control metric is at least one of a steering control metric, an acceleration control metric, or a situational awareness control metric,

determining a driver status corresponding to the vehicle operation control metric based on information received from the sensor, and

on a condition that an immanency for the expected vehicle operation control transfer is within a minimum relevant immanence, on a condition that the expected vehicle operation control transfer is a vehicle operation control transfer from autonomous vehicle control to driver control, and on a condition that the driver status for the vehicle operation control metric indicates a disengaged driver status;

 

identifying a secondary vehicle control action corresponding to the expected vehicle operation control transfer;

 

controlling the trajectory controller to operate the vehicle to traverse a part of the portion of the vehicle transportation network in accordance with the secondary vehicle control action;

 

controlling the primary graphical display portion to present a representation of the vehicle operation assistance information item; and

 

controlling the secondary graphical display portion to present the representation of the vehicle operation assistance information item.

16. The vehicle of claim 15, further comprising:

a first graphical display unit, wherein the first graphical display unit includes the first graphical display portion; and

a second graphical display unit, wherein the second graphical display unit includes the second graphical display portion, and wherein the second graphical display unit is physically independent of the first graphical display unit.

17. The vehicle of claim 15, wherein the primary graphical display portion is spatially located within the vehicle along a first angular offset from a center of a primary field of view of the driver of the vehicle, and wherein the secondary graphical display portion is spatially located within the vehicle along a second angular offset from the center of the primary field of view of the driver of the vehicle, wherein the second angular offset is greater than the first angular offset, such that the primary graphical display portion is visually distinct from the secondary graphical display portion.

18. The vehicle of claim 15, wherein on the condition that an urgency for the vehicle operation assistance information item indicates the warning urgency, and the condition that the immanency for the expected vehicle operation control transfer is within the minimum relevant immanence, and the condition that the expected vehicle operation control transfer is the vehicle operation control transfer from autonomous vehicle control to driver control, and on a condition that the driver status for the vehicle operation control metric indicates an engaged driver status, the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

transfer vehicle operation control from autonomous vehicle control to driver control;

control the primary graphical display portion to present a representation of the vehicle operation assistance information item; and

control the secondary graphical display portion to present the representation of the vehicle operation assistance information item.

20. The vehicle of claim 15, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

in response to controlling the trajectory controller to operate the vehicle to traverse the part of the portion of the vehicle transportation network in accordance with the secondary vehicle control action, control the primary graphical display portion to present the representation of the vehicle operation assistance information item such that the representation includes an indication of an aborted vehicle operation control transfer; and

in response to transferring vehicle operation control from autonomous vehicle control to driver control, control the primary graphical display portion to present the representation of the vehicle operation assistance information item such that the representation includes an indication of a completed vehicle operation control transfer.

19. A vehicle configured to traverse a portion of a vehicle transportation network, the vehicle comprising:

a primary graphical display portion;

a secondary graphical display portion;

a sensor;

a trajectory controller; and

a processor configured to execute instructions stored on a non-transitory computer readable medium to;

identify a vehicle operation assistance information item, the vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer, wherein the expected vehicle operation control transfer is a vehicle operation control transfer from autonomous vehicle control to driver control,

identify a vehicle operation control metric for the expected vehicle operation control transfer, wherein the vehicle operation control metric is at least one of a steering control metric, an acceleration control metric, or a situational awareness control metric,

determine a driver status corresponding to the vehicle operation control metric based on information received from the sensor,

in response to a determination that the driver status for the vehicle operation control metric indicates a disengaged driver status;

identify a secondary vehicle control action corresponding to the expected vehicle operation control transfer; and

control the trajectory controller to operate the vehicle to traverse a part of the portion of the vehicle transportation network in accordance with the secondary vehicle control action,

in response to a determination that the driver status for the vehicle operation control metric indicates an engaged driver status transfer vehicle operation control from autonomous vehicle control to driver control,

control the primary graphical display portion to present a representation of the vehicle operation assistance information item, and

control the secondary graphical display portion to present the representation of the vehicle operation assistance information item.