Communication between autonomous vehicle and external observers

• US 9,475,422 B2
• Filed: 05/22/2014
• Issued: 10/25/2016
• Est. Priority Date: 05/22/2014
• Status: Active Grant

• Current Assignee: Applied Invention, LLC
• Original Assignee: Applied Invention, LLC
• Inventors: Williams, Kjerstin I., Sarrett, James W., Khanlian, Luke W., Kaehler, Adrian L., Howe, Russel, Hillis, W. Daniel, Tombrello, Thomas A.
• Primary Examiner(s): Beaulieu, Yonel
• Assistant Examiner(s): Ramesh, Krishnan
• Application Number:
  US14/285,357
• Publication Number:
  US 20150336502A1
• Time in Patent Office: 887 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60Q 1/26 the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
• CPC Class Codes: B60Q 1/5037 the display content changing automatically, e.g. depending on traffic situation
• CPC Class Codes: B60Q 1/507 specific to autonomous vehicles
• CPC Class Codes: B60Q 1/508 specific to vehicles driving in fleets or convoys
• CPC Class Codes: B60Q 1/545 for indicating other traffic conditions, e.g. fog, heavy traffic
• CPC Class Codes: B60Q 1/547 for issuing requests to other traffic participants; for confirming to other traffic participants they can proceed, e.g. they can overtake
• CPC Class Codes: B60Q 1/549 for expressing greetings, gratitude or emotions
• CPC Class Codes: B60Q 2400/50 Projected symbol or information, e.g. onto the road or car body
• CPC Class Codes: B60Q 5/005 automatically actuated
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

At least one embodiment of this disclosure includes a method for an autonomous vehicle (e.g., a fully autonomous or semi-autonomous vehicle) to communicate with external observers. The method includes: receiving a task at the autonomous vehicle; collecting data that characterizes a surrounding environment of the autonomous vehicle from a sensor coupled to the autonomous vehicle; determining an intended course of action for the autonomous vehicle to undertake based on the task and the collected data; and conveying a human understandable output via an output device, the human understandable output expressly or implicitly indicating the intended course of action to an external observer.

Claims

1. A method for an autonomous vehicle to communicate with external observers, comprising:

receiving a task at the autonomous vehicle;

collecting data that characterizes a surrounding environment of the autonomous vehicle from a sensor coupled to the autonomous vehicle;

determining an intended course of action for the autonomous vehicle to undertake based on the task and the collected data;

projecting a human understandable output, via a projector that manipulates or produces light, to a ground surface in proximity to the autonomous vehicle; and

wherein the human understandable output indicates the intended course of action of the autonomous vehicle to an external observer; and

wherein the autonomous vehicle confirms recognition of the external observer by presenting, via the projector, an image of the external observer.

2. The method of claim 1, wherein projecting the human understandable output includes indicating an intended trajectory of the autonomous vehicle.

3. The method of claim 2, further comprising detecting a location of a potential obstacle from the collected data;

and wherein the intended trajectory includes a notch that indicates the detected location of the potential obstacle.

4. The method of claim 2, wherein indicating the intended trajectory includes displaying an extent of the intended trajectory based on a predetermined time interval.

5. The method of claim 2, wherein the intended trajectory is indicated using a laser.

6. The method of claim 2, wherein indicating the intended trajectory includes projecting the intended trajectory as a pattern.

7. The method of claim 6, wherein the pattern includes voids such that, when multiple vehicles project onto a same location, a resulting pattern of overlapping projections has fewer or smaller voids.

8. The method of claim 1, wherein projecting the human understandable output includes indicating an avoidance area that the autonomous vehicle intends to avoid.

9. The method of claim 8, wherein determining the intended course of action includes:

detecting an obstacle based on the collected data; and

determining the avoidance area by estimating a predicted motion of the detected obstacle.

10. A vehicle comprising:

a control system with electronic circuitry configured to enable the vehicle to complete a task autonomously or semi-autonomously;

a sensor, coupled to the control system, configured to collect data related to a surrounding environment of the vehicle;

wherein the control system is configured to determine an intended course of action for the vehicle based on the task and the collected data; and

an anthropomorphic device, coupled to the control system, configured to convey a human understandable output regarding the intended course of action to an external observer,

wherein the vehicle confirms recognition of the external observer by presenting, via the anthropomorphic device, an image of the external observer.

11. The vehicle of claim 10, wherein the anthropomorphic device is configured to move in a pattern or a human understandable gesture to convey the human understandable output.

12. The vehicle of claim 11, wherein the anthropomorphic device is a sensor pack mounted over an area between a driver seat and a passenger seat.

13. The vehicle of claim 12, wherein the anthropomorphic device is a camera capable of pan, tilt, and zoom.

14. The vehicle of claim 10, wherein the anthropomorphic device is an actuated hand or a display device configured to animate a cartoon hand.

15. The vehicle of claim 10, wherein the anthropomorphic device is a display device configured to animate a face with eyes.

16. The vehicle of claim 10, wherein the anthropomorphic device is configured to indicate recognition of an obstacle.

17. The vehicle of claim 10, wherein the anthropomorphic device is configured to indicate acknowledgement of a command received from the external observer.

18. The vehicle of claim 10, wherein the anthropomorphic device is configured to indicate an intended direction of travel of the vehicle.

19. A method for an autonomous vehicle to communicate with external observers, comprising:

receiving a task at the autonomous vehicle;

collecting data that characterizes a surrounding environment of the autonomous vehicle from a sensor coupled to the autonomous vehicle;

determining an intended course of action for the autonomous vehicle to undertake based on the task and the collected data; and

conveying a human understandable output, via an output device, wherein the human understandable output indicates the intended course of action of the autonomous vehicle to an external observer,

wherein the autonomous vehicle confirms recognition of the external observer by presenting, via the output device, an image of the external observer.

20. The method of claim 19, further comprising detecting a location of the external observer based on the collected data;

and wherein conveying the human understandable output includes targeting the human understandable output at the detected location of the external observer.

21. The method of claim 19, wherein the output device conveying the human understandable output is a sound producing apparatus.

22. The method of claim 19, wherein the output device conveying the human understandable output displays at least one of a standardized transportation symbol, a human language word, and an image on a display of the autonomous vehicle.

23. The method of claim 19, wherein the collected data that characterizes the surrounding environment includes detecting a command from the external observer to the autonomous vehicle, wherein the image of the external observer indicates acknowledgement of the command.

24. The method of claim 19, further comprising producing an attention grabbing effect via a second output device to notify the external observer of an intent to convey a message, prior to conveying the human understandable output;

wherein the output device is a first output device.

25. The method of claim 24, wherein the first output device is the second output device.

26. The method of claim 24, wherein producing the attention grabbing effect includes displaying a notification on a ground surface in proximity to the autonomous vehicle.

27. The method of claim 24, further comprising detecting a location of the external observer based on the collected data;

and wherein producing the attention grabbing effect includes projecting light towards a direction corresponding to the detected location of the external observer.

28. The method of claim 19, wherein determining the intended course of action includes generating a request to deliver to the external observer;

and wherein conveying the human understandable output includes presenting the request to the external observer.

29. The method of claim 19, wherein conveying the human understandable output includes indicating that the autonomous vehicle is in a convoy.

30. The method of claim 19, wherein determining the intended course of action includes determining that the autonomous vehicle is to enter into a convoy;

and wherein conveying the human understandable output includes indicating that the autonomous vehicle is waiting for or ready to enter into the convoy.

31. The method of claim 19, wherein the human understandable output is a movement state indicator.

32. The method of claim 31, wherein conveying the human understandable output includes displaying the movement state indicator on a ground surface in proximity to the autonomous vehicle.

33. The method of claim 32, wherein the movement state indicator takes a form of a line indicating that the autonomous vehicle will not cross the line.

34. The method of claim 31, wherein the movement state indicator includes a stopped state, a normal speed state, and a reduced speed state.

35. The method of claim 19, wherein conveying the human understandable output includes embedding machine readable information in the human understandable output.

36. The method of claim 35, wherein the machine readable information conveys additional information beyond that conveyed in the human understandable output.

37. The method of claim 35, wherein embedding the machine readable information includes pulse-coding the human understandable output.