Vehicle operation assistance information management for autonomous vehicle control transfer

• US 10,150,370 B2
• Filed: 02/20/2018
• Issued: 12/11/2018
• Est. Priority Date: 06/24/2015
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan North America Incorporated (Nissan Motor Co., Ltd.)
• Inventors: Konet, Heather, Goudy, Roy, Christensen, Andrew
• Primary Examiner(s): Sosanya, Obafemi O
• Application Number:
  US15/900,633
• Publication Number:
  US 20180186236A1
• Time in Patent Office: 294 Days
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/285 for improving awareness by directing driver's gaze direction or eye points
• CPC Class Codes: B60K 35/81 for controlling displays

Abstract

A vehicle implementing vehicle operation assistance information management for autonomous vehicle control transfer may identify a vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer, and, in response to a determination that an immanency for the vehicle operation assistance information item is within a maximum relevant immanence, present a representation of the vehicle operation assistance information item, which may include, in response to a determination that an urgency for the vehicle operation assistance information item indicates a warning urgency or an advisory urgency, controlling the primary graphical display portion and the secondary graphical display portion to present a graphical representation of the vehicle operation assistance information item, such that the primary graphical display portion presents a first portion of the graphical representation and the secondary graphical display portion presents a second portion of the graphical representation.

Claims

1. A vehicle configured to traverse a portion of a vehicle transportation network, the vehicle comprising:

a primary graphical display portion;

a secondary graphical display portion; and

a processor configured to execute instructions stored on a non-transitory computer readable medium to;

identify a vehicle operation assistance information item, the vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer; and

in response to a determination that an immanency for the vehicle operation assistance information item is within a maximum relevant immanence, present a representation of the vehicle operation assistance information item, wherein presenting the representation of the vehicle operation assistance information item includes;

in response to a determination that an urgency for the vehicle operation assistance information item indicates a warning urgency or an advisory urgency, controlling the primary graphical display portion and the secondary graphical display portion to present a graphical representation of the vehicle operation assistance information item, such that the primary graphical display portion presents a first portion of the graphical representation and the secondary graphical display portion presents a second portion of the graphical representation.

2. The vehicle of claim 1, further comprising:

a first graphical display unit, wherein the first graphical display unit includes the first graphical display portion; and

a second graphical display unit, wherein the second graphical display unit includes the second graphical display portion, and wherein the second graphical display unit is physically independent of the first graphical display unit.

3. The vehicle of claim 1, wherein the primary graphical display portion is spatially located within the vehicle along a first angular offset from a center of a primary field of view of a driver of the vehicle, and wherein the secondary graphical display portion is spatially located within the vehicle along a second angular offset from the center of the primary field of view of the driver of the vehicle, wherein the second angular offset is greater than the first angular offset, such that the primary graphical display portion is visually distinct from the secondary graphical display portion.

4. The vehicle of claim 1, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to determine the urgency based on the immanency by:

in response to a determination that the vehicle operation assistance information item includes an urgency value, identifying the urgency value as the urgency; and

in response to a determination that the vehicle operation assistance information item omits the urgency value;

in response to a determination that the immanency exceeds an advisory cost, determining that the urgency is an advance urgency; and

in response to a determination that the immanency is within the advisory cost;

in response to a determination that the immanency exceeds a warning cost, determining that the urgency is the advisory urgency; and

in response to a determination that the immanency is within the warning cost, determining that the urgency is the warning urgency.

5. The vehicle of claim 1, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

identify the vehicle operation assistance information item in response to at least one of;

receiving information indicating a vehicle operation control transfer request that includes a driver initiated vehicle operation control transfer request;

or

identifying information indicating the vehicle operation control transfer request that includes a vehicle initiated vehicle operation control transfer request.

6. The vehicle of claim 5, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to identify the information indicating the vehicle operation control transfer request that includes the vehicle initiated vehicle operation control transfer request, in response to at least one of:

receiving, via a wireless electronic communication link, external vehicle operation assistance information, wherein the external vehicle operation assistance information includes at least one of an infrastructure message from an external infrastructure device or an inter-vehicle message from another vehicle;

identifying vehicle transportation network information representing a portion of a vehicle transportation network in response to traversing a part of the portion of the vehicle transportation network;

or

receiving sensor information from a sensor, wherein the vehicle includes the sensor.

7. The vehicle of claim 6, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to determine the immanency by:

in response to a determination that the vehicle operation assistance information item includes an immanency value, identifying the immanency value as the immanency; and

in response to a determination that the vehicle operation assistance information item omits the immanency value;

in response to a determination that a probability that a current expected path for the vehicle intersects with a target geospatial location in the vehicle transportation network indicated by the vehicle operation assistance information item is within a minimum relevance threshold, identifying the immanency such that the immanency exceeds the maximum relevant immanence; and

in response to a determination that the probability that the current expected path for the vehicle intersects with the target geospatial location exceeds the minimum relevance threshold, identifying, as the immanency, a first temporal cost for the vehicle to traverse the vehicle transportation network from a current geospatial location of the vehicle in the vehicle transportation network to the target geospatial location.

8. The vehicle of claim 7, wherein, on a condition that identifying the vehicle operation assistance information item includes identifying vehicle transportation network information representing the portion of the vehicle transportation network, the processor is configured to execute instructions stored on a non-transitory computer readable medium to determine the target geospatial location by:

identifying a route for the vehicle to traverse the vehicle transportation network from an origin to a destination, wherein the route includes the current geospatial location of the vehicle and the target geospatial location;

identifying a portion of the route as a driver control portion; and

identifying a geospatial location in the vehicle transportation network of the driver control portion that is proximate to the current geo spatial location of the vehicle along the route as the target geospatial location.

9. The vehicle of claim 5, wherein:

on a condition that the identifying the vehicle operation assistance information item includes receiving information indicating the vehicle operation control transfer request that includes the driver initiated vehicle operation control transfer request, the vehicle operation control transfer request includes;

a request to transfer vehicle operation control from the driver to an autonomous vehicle control unit of the vehicle;

or

a request to transfer vehicle operation control from the autonomous vehicle control unit of the vehicle to the driver; and

on a condition that the identifying the vehicle operation assistance information item includes identifying information indicating the vehicle operation control transfer request that includes the vehicle initiated vehicle operation control transfer request, the vehicle operation control transfer request includes the request to transfer vehicle operation control from the autonomous vehicle control unit of the vehicle to the driver.

10. The vehicle of claim 1, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to identify a vehicle operation control metric for the expected vehicle operation control transfer in response to identifying the vehicle operation assistance information item, wherein the vehicle operation control metric is a steering control metric, an acceleration control metric, or a situational awareness control metric.

11. The vehicle of claim 10, wherein controlling the primary graphical display portion and the secondary graphical display portion to present a graphical representation of the vehicle operation assistance information item includes:

in response to a determination that the urgency is the advisory urgency, controlling the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes an advisory pictographic representation of the vehicle operation control metric and a primary representation of the immanency proximate to the advisory pictographic representation of the vehicle operation control metric in the first portion of the graphical representation; and

in response to a determination that the urgency is the warning urgency;

in response to a determination that the vehicle operation control transfer request includes the request to transfer vehicle operation control from the autonomous vehicle control unit of the vehicle to the driver;

in response to a determination that a driver status for the vehicle operation control metric indicates a disengaged driver status, controlling the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes a warning pictographic representation of the vehicle operation control metric.

12. The vehicle of claim 11, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

in response to a determination that the immanency is within a minimum relevant immanence;

in response to the determination that the driver status for the vehicle operation control metric indicates the disengaged driver status;

identify a secondary autonomous vehicle control action responsive to the vehicle operation assistance information item;

control the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes a representation of the secondary autonomous vehicle control action; and

control the vehicle in accordance with the secondary autonomous vehicle control action, and

in response to a determination that the driver status for the vehicle operation control metric indicates an engaged driver status;

transfer control of the vehicle operation control; and

in response to transferring control of the vehicle operation control, control the primary graphical display portion to present the first portion of the graphical representation such that the first portion of the graphical representation includes a transferred pictographic representation of the vehicle operation control metric.

13. The vehicle of claim 10, wherein controlling the primary graphical display portion and the secondary graphical display portion to present a graphical representation of the vehicle operation assistance information item includes:

in response to a determination that the urgency is an advance urgency, controlling the secondary graphical display portion to present the graphical representation of the vehicle operation assistance information item such that the graphical representation includes a verbal representation of the expected vehicle operation control transfer;

in response to a determination that the urgency is the advisory urgency, controlling the secondary graphical display portion to present the second portion of the graphical representation such that the second portion of the graphical representation includes a verbal representation of the vehicle operation control metric, and such that the second portion of the graphical representation includes a secondary representation of the immanency proximate to the verbal representation; and

in response to a determination that the urgency is the warning urgency, controlling the secondary graphical display portion to present the second portion of the graphical representation such that the second portion of the graphical representation includes a verbal representation of the vehicle operation control metric.

14. The vehicle of claim 1, further comprising:

an auditory presentation device; and

wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to, in response to the determination that the urgency indicates the warning urgency or the advisory urgency, control the auditory presentation device to present an auditory representation of the vehicle operation assistance information item.

15. A vehicle configured to traverse a portion of a vehicle transportation network, the vehicle comprising:

a primary graphical display portion;

a secondary graphical display portion;

a trajectory controller; and

a processor configured to execute instructions stored on a non-transitory computer readable medium to;

identify a vehicle operation assistance information item, the vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer; and

in response to a determination that an urgency for the vehicle operation assistance information item indicates a warning urgency, a determination that an immanency for the expected vehicle operation control transfer is within a minimum relevant immanence, a determination that the expected vehicle operation control transfer is a vehicle operation control transfer from autonomous vehicle control to driver control, and a determination that a driver status for the vehicle operation control metric indicates a disengaged driver status;

identify a secondary vehicle control action corresponding to the expected vehicle operation control transfer;

control the trajectory controller to operate the vehicle to traverse a part of the portion of the vehicle transportation network in accordance with the secondary vehicle control action;

control the primary graphical display portion to present a representation of the vehicle operation assistance information item; and

control the secondary graphical display portion to present the representation of the vehicle operation assistance information item.

16. The vehicle of claim 15, further comprising:

a first graphical display unit, wherein the first graphical display unit includes the first graphical display portion; and

a second graphical display unit, wherein the second graphical display unit includes the second graphical display portion, and wherein the second graphical display unit is physically independent of the first graphical display unit.

17. The vehicle of claim 15, wherein the primary graphical display portion is spatially located within the vehicle along a first angular offset from a center of a primary field of view of the driver of the vehicle, and wherein the secondary graphical display portion is spatially located within the vehicle along a second angular offset from the center of the primary field of view of the driver of the vehicle, wherein the second angular offset is greater than the first angular offset, such that the primary graphical display portion is visually distinct from the secondary graphical display portion.

18. The vehicle of claim 15, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

in response to controlling the trajectory controller to operate the vehicle to traverse the part of the portion of the vehicle transportation network in accordance with the secondary vehicle control action, control the primary graphical display portion to present the representation of the vehicle operation assistance information item such that the representation includes an indication of an aborted vehicle operation control transfer.

19. A vehicle configured to traverse a portion of a vehicle transportation network, the vehicle comprising:

a primary graphical display portion;

a secondary graphical display portion; and

a processor configured to execute instructions stored on a non-transitory computer readable medium to;

identify a vehicle operation assistance information item, the vehicle operation assistance information item indicating a location of an expected vehicle operation control transfer, wherein the expected vehicle operation control transfer is a vehicle operation control transfer from autonomous vehicle control to driver control; and

in response to a determination that a driver status for the vehicle operation control metric indicates an engaged driver status;

transfer vehicle operation control from autonomous vehicle control to driver control;

control the primary graphical display portion to present a representation of the vehicle operation assistance information item; and

control the secondary graphical display portion to present the representation of the vehicle operation assistance information item.

20. The vehicle of claim 19, wherein the processor is configured to execute instructions stored on a non-transitory computer readable medium to:

in response to transferring vehicle operation control from autonomous vehicle control to driver control, control the primary graphical display portion to present the representation of the vehicle operation assistance information item such that the representation includes an indication of a completed vehicle operation control transfer.