Autonomous vehicle signaling of third-party detection

• US 10,118,548 B1
• Filed: 06/15/2017
• Issued: 11/06/2018
• Est. Priority Date: 06/15/2017
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Fields, Brian Mark, Christiansen, Duane
• Primary Examiner(s): Swarthout, Brent
• Application Number:
  US15/624,363
• Time in Patent Office: 509 Days
• Field of Search: None
• CPC Class Codes: B60Q 1/26 the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
• CPC Class Codes: B60Q 5/005 automatically actuated
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

Techniques for a fully- or semi-autonomously operated vehicle (AV) to signal to or notify a third-party located within the vehicle'"'"'s environment that the vehicle has detected the third-party'"'"'s presence are provided. The third-party may be a pedestrian or a passenger in another proximate vehicle. Sensors associated with the AV may provide data via which the third-party'"'"'s presence (and optionally, movement and behavior) is detected, and signaling device(s) on-board the vehicle may transmit audio, visual, and/or electromagnetic signals into the environment to indicate that the vehicle is aware of the third-party'"'"'s presence. In some situations, a two-way dialog between the signaling device(s) and the third-party may be established to confirm intended behaviors or actions of the vehicle and/or of the third-party. Increased safety and decreased risk of accident or injury may be provided with AVs in a manner that is familiar to third-parties'"'"' defensive signaling with drivers of non-autonomous vehicles.

Claims

1. A system for signaling, to a third-party external to a vehicle, that the third-party has been detected by the vehicle, the system comprising:

one or more sensors associated with the vehicle;

one or more processors associated with the vehicle and communicatively connected to the one or more sensors;

one or more signaling devices associated with the vehicle and communicatively connected to the one or more processors;

one or more tangible, non-transitory memories storing thereon computer-executable instructions that, when executed by the one or more processors, cause the system to;

monitor, via the one or more sensors, an environment in which the vehicle is located, the environment in which the vehicle is located being a vehicle environment;

detect that a third-party is located within the vehicle environment based upon sensor data obtained from the one or more sensors while monitoring the vehicle environment, the third-party being a person; and

engage in a dialogue with the third-party, including;

generate a notification indicating that a presence of the third-party has been detected by the vehicle, the notification being a third-party detection notification;

transmit, into the vehicle environment via the one or more signaling devices associated with the vehicle, one or more indications of the third-party detection notification, thereby alerting the third-party that the presence of the third-party has been detected by the vehicle;

detect via the one or more sensors associated with the vehicle, a bodily signal made by the third-party in response to the third-party detection notification; and

based upon the detected bodily signal of the third party, at least one of;

communicate an acknowledgement to the third-party via the one or more signaling devices, communicate a response to the third-party via the one or more signaling devices, or modify an operation of the vehicle.

2. The system of claim 1, wherein:

the one or more signaling devices include a wireless communication transceiver that is associated with the vehicle and that is included in the one or more sensors associated with the vehicle;

the sensor data obtained via the wireless communication transceiver is based upon a wireless signal generated by one or more mobile computing devices associated with the third-party and received via the wireless communication transceiver; and

the transmission of the one or more indications of the third-party detection notification into the vehicle environment via the one or more signaling devices associated with the vehicle includes a transmission, into the vehicle environment, of a respective indication of the third-party detection via the wireless communication transceiver to the one or more mobile computing devices associated with the third-party.

3. The system of claim 1, wherein the engagement in the dialogue with the third-party further includes a communication that is transmitted into the vehicle environment via the one or more signaling devices and that indicates at least one of an action to be performed by the vehicle or an action to be performed by the third-party based upon the third-party detection notification.

4. The system of claim 1, wherein at least one of:

(i) the one or more signaling devices include an audio signaling device associated with the vehicle, and the one or more indications of the third-party detection notification comprises an audio signal transmitted via the audio signaling device into the environment in which the vehicle is located;

or

(ii) the one or more signaling devices include a visual signaling device associated with the vehicle and facing the environment in which the vehicle is located, the one or more indications of the third-party detection notification comprises a visual signal generated by the visual signaling device and emitted into the environment in which the vehicle is located, the visual signal including a display of at least one of alphanumeric characters or a graphic, and the visual signal generated by the visual signaling device is excluded from a set of visual signals that are generated by other devices disposed at the vehicle and that are indicative of one or more operating behaviors of the vehicle.

5. The system of claim 1, wherein the third-party includes at least one of:

a pedestrian, a cyclist, a person using a wheelchair or other assistive ambulatory device, a person using a wheeled board, a person using roller skates or roller blades, a person using a scooter, or a person who is being transported by another vehicle.

17. The system of claim 1, wherein the vehicle is at least partially-autonomously operated.

19. The system of claim 2, wherein the engagement in the dialogue with the third-party further includes a reception, via the wireless communication transceiver, of a wireless signal generated by the one or more mobile computing devices associated with the third-party in response to the third-party detection notification in addition to the detection of the bodily signal made by the third-party.

6. A computer-implemented method of signaling, to a third-party external to a vehicle, that the third-party has been detected by the vehicle, comprising:

monitoring, using one or more sensors associated with the vehicle, an environment in which the vehicle is located, the environment in which the vehicle is located being a vehicle environment;

detecting, based upon sensor data obtained from the one or more sensors while monitoring the vehicle environment and by using one or more processors associated with the vehicle and communicatively connected to the one or more sensors, that a third-party is located within the vehicle environment, the third-party being a person; and

engaging in a dialogue with the third-party, including;

generating, by one or more processors, a notification indicating that a presence of the third-party has been detected by the vehicle, the notification being a third-party detection notification;

transmitting, into the vehicle environment via one or more signaling devices associated with the vehicle, one or more indications of the third-party detection notification, thereby alerting the third-party that the presence of the third-party has been detected by the vehicle;

detecting, via the one or more sensors associated with the vehicle, a bodily signal made by the third-party in response to the third-party detection notification; and

based upon the detected bodily signal of the third-party, at least one of;

communicating an acknowledgment to the third-party, communicating a response to the third-party, or modifying an operation of the vehicle.

7. The computer-implemented method of claim 6, wherein detecting that the third-party is located within the vehicle environment based upon the sensor data comprises detecting that the third-party is located within the vehicle environment based upon a wireless signal generated by one or more mobile computing devices associated with the third-party and detected by the one or more sensors.

8. The computer-implemented method of claim 6, wherein transmitting the indication of the third-party detection notification via the one or more signaling devices associated with the vehicle comprises transmitting the indication of the third-party detection notification via a wireless communication transceiver associated with the vehicle to the one or more mobile computing devices associated with the third-party.

9. The computer-implemented method of claim 6, further comprising communicating additional information via the one or more signaling devices to the third-party, the additional information indicative of at least one of a desired action to be performed by the vehicle or a desired action to be performed by the third-party, and the additional information based upon the third-party detection notification.

10. The computer-implemented method of claim 6, wherein transmitting the one or more indications of the third-party detection notification into the environment in which the vehicle is located via the one or more signaling devices associated with the vehicle includes transmitting an audio signal generated by an audio signaling device disposed at the vehicle into the environment in which the vehicle is located.

11. The computer-implemented method of claim 6, wherein transmitting the one or more indications of the third-party detection notification into the environment in which the vehicle is located via the one or more signaling devices associated with the vehicle includes generating a visual signal at a visual signaling device disposed at the vehicle and facing into the environment in which the vehicle is located, the visual signal including a display of at least one of alphanumeric characters or a graphic, and emitting the visual signal into the environment in which the vehicle is located.

12. The computer-implemented method of claim 6, wherein detecting that the third-party is located within the vehicle environment comprises detecting that one of:

a pedestrian, a cyclist, a person using a wheelchair or other assistive ambulatory device, a person using a wheeled board, a person using roller skates or roller blades, a person using a scooter, or a person who is being transported by another vehicle is located within the vehicle environment.

18. The system of claim 6, the vehicle is at least partially-autonomously operated.

13. A tangible, non-transitory computer-readable medium storing instructions for acknowledging third-party detection that, when executed by one or more processors of a computer system, cause the computer system to:

monitor, via one or more sensors associated with the vehicle, an environment in which the vehicle is located, the environment in which the vehicle is located being a vehicle environment;

detect that a third-party is located within the vehicle environment based upon sensor data obtained from the one or more sensors while monitoring the vehicle environment in which the vehicle is located, the third-party being a person; and

engage in a dialogue with the third-party including executing at least a portion of the stored instructions to cause the computer system to;

generate a notification indicating that a presence of the third-party has been detected by the vehicle, the notification being a third-party detection notification;

transmit, into the vehicle environment via one or more signaling devices associated with the vehicle, one or more indications of the third-party detection notification, thereby alerting the third-party that the presence of the third-party has been detected by the vehicle;

detect, via the one or more sensors associated with the vehicle, a bodily signal made by the third-party in response to the third-party detection notification; and

based upon the detected bodily signal of the third-party, at least one of;

communicate an acknowledgment to the third-party, communicate a response to the third-party, or modify an operation of the vehicle.

14. The tangible, non-transitory computer-readable medium of claim 13, wherein at least one of:

(i) the sensor data includes data included in a wireless signal generated by a personal electronic device associated with third-party and received at a wireless communication transceiver disposed at the vehicle;

or

(ii) the wireless communication transceiver is included in the one or more sensors.

15. The tangible, non-transitory computer-readable medium of claim 13, wherein at least one of:

the one or more signaling devices include an audio signaling device disposed at the vehicle, and the one or more indications of the third-party detection notification includes an audio signal transmitted by the audio signaling device into the vehicle environment in which the vehicle is located;

or

the one or more signaling devices include a visual signaling device disposed at the vehicle and facing into the vehicle environment in which the vehicle is located, the one or more indications of the third-party detection notification includes a visual signal generated by the visual signaling device and emitted into the environment in which the vehicle is located, the visual signal including a display of at least one of alphanumeric characters or a graphic, and the visual signal generated by the visual signaling device is excluded from a set of visual signals that are generated by other devices disposed at the vehicle and that are indicative of one or more operating behaviors of the vehicle.

16. The tangible, non-transitory computer-readable medium of claim 13, wherein the third-party includes at least one of:

a pedestrian, a cyclist, a person using a wheelchair or other assistive ambulatory device, a person using a wheeled board, a person using roller skates or roller blades, a person using a scooter, or a person who is being transported by another vehicle.

20. The system of claim 13, wherein the bodily signal made by the third-party includes at least one of:

a head movement of the third-party, an eye movement of the third-party, or a hand signal of the third-party.