Self-driving vehicle systems and methods

• US 9,429,947 B1
• Filed: 04/14/2016
• Issued: 08/30/2016
• Est. Priority Date: 04/14/2016
• Status: Active Grant

• Current Assignee: Drivent LLC
• Original Assignee: Eric John Wengreen, Wesley Edward Schwie
• Inventors: Schwie, Wesley Edward, Wengreen, Eric John
• Primary Examiner(s): Shaawat, Mussa A
• Application Number:
  US15/099,565
• Time in Patent Office: 138 Days
• US Class Current: 1/1
• CPC Class Codes: B60Q 5/00 Arrangement or adaptation of acoustic signal devices
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/06 Automatic manoeuvring for parking
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/12 Target-seeking control
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 20/12 specially adapted for electronic shopping systems
• CPC Class Codes: G06Q 50/30 Transportation; Communications
• CPC Class Codes: G06Q 50/40 Business processes related to the transportation industry shipping G06Q10/083
• CPC Class Codes: G08G 1/202 Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/1807 using prosody or stress

Abstract

Although self-driving vehicles excel under “normal” driving conditions, they sometimes struggle with new situations that often would not be overly difficult for a person. Several embodiments described herein enable a hybrid approach that leverages the exceptional abilities of self-driving vehicles while soliciting human interaction in select situations. The resulting combination of machine intelligence and human intelligence significantly enlarges the potential of self-driving vehicles in a manner that will enable self-driving vehicles to become widespread much faster than would otherwise be the case.

Claims

1. A method of using a self-driving vehicle, the method comprising:

detecting, by the vehicle, an audible voice of a person located outside the vehicle;

sending, in response to detecting the voice, a first wireless communication to a remote computing device;

receiving, by the vehicle, audio data recorded by a microphone of the remote computing device in response to the first wireless communication;

emitting outside the vehicle, by a speaker of the vehicle, a sound based on the audio data;

recording, by a microphone system of the vehicle, a verbal response to the sound from the person located outside the vehicle; and

sending automatically a recording of the verbal response to the remote computing device.

2. The method of claim 1, wherein the sending the first wireless communication is at least partially in response to detecting, by the vehicle, the person located outside of the vehicle.

3. The method of claim 2, wherein the detecting the person comprises detecting by at least one of a video camera, the microphone system, a proximity sensor, an infrared sensor, a radar detector, and a motion sensor of the vehicle.

4. The method of claim 1, wherein the sending the first wireless communication is at least partially in response to detecting, by the vehicle, a knock on a portion of the vehicle.

5. The method of claim 4, wherein the detecting the knock by the vehicle comprises detecting at least one of a knocking sound by the microphone system and a vibration by a vibration sensor, wherein the knocking sound and the vibration are indicative of knocking on the vehicle.

6. The method of claim 1, further comprising determining, by the vehicle, that the audible voice originated from outside the vehicle.

7. The method of claim 1, further comprising determining, by the vehicle, that the audible voice is greater than a threshold configured to help the vehicle differentiate between background voices and voices directed to the vehicle from a location outside of the vehicle.

8. The method of claim 1, further comprising sending the first wireless communication in response to:

determining, by the vehicle, at least one of the audible voice originated outside the vehicle and the person is located outside the vehicle; and

determining, by the vehicle, that the voice has asked a question.

9. The method of claim 1, wherein the microphone system of the vehicle comprises a first microphone and a second microphone spaced apart from the first microphone, the method further comprising:

detecting, by the first and second microphones of the vehicle, the audible voice;

comparing, by the vehicle, a first voice signal detected by the first microphone and a second voice signal detected by the second microphone to evaluate a directionality of the voice; and

determining, by the vehicle, that the directionality is indicative of the voice being directed towards the vehicle.

10. The method of claim 1, wherein the vehicle comprises a first video camera arranged and configured to record a first video of an area outside the vehicle,

the method further comprising:

recording, by the first video camera, the first video of the area outside the vehicle during the verbal response; and

sending, by the vehicle, the first video to the remote computing device.

11. The method of claim 10, wherein the vehicle further comprises a display screen facing outward such that the person located outside the vehicle can see information on the display screen,

the method further comprising receiving a second video recorded by a second video camera of the remote computing device, and showing the second video on the display screen facing outward such that the vehicle is configured to enable the person located outside the vehicle to see the second video.

12. The method of claim 1, further comprising initiating a three-way audio communication between the person located outside the vehicle, a first human representative of the vehicle, and a second human representative of the vehicle, wherein the first human representative and the second human representative are located remotely relative to the vehicle, wherein the remote computing device is a first remote computing device associated with the first human representative, and a second remote computing device is associated with the second human representative, wherein the three-way audio communication comprises receiving, by the vehicle, a first audio data recorded by the microphone of the first remote computing device, and a second audio data recorded by a microphone of the second remote computing device.

13. The method of claim 1,

further comprising detecting, by the vehicle, a collision of the vehicle,

and initiating, in response to the detecting the collision, a two-way audio communication between the person located outside the vehicle and a human representative of the vehicle while the human representative is located remotely relative to the vehicle.

14. The method of claim 1, further comprising detecting, by a sensor of the vehicle, a condition of a road surface, determining that the condition is potentially hazardous to the vehicle, and

receiving, by the vehicle, an instruction based on input from a human located remotely relative to the vehicle, wherein the input is in response to the condition, wherein the instruction comprises information regarding how the vehicle should respond to the condition of the road surface.

15. The method of claim 1, further comprising identifying, by the vehicle, a discrepancy between an actual road and a road map, and receiving, by the vehicle, an instruction regarding how the vehicle should respond to the discrepancy.

16. The method of claim 1, further comprising identifying, by the vehicle, an impasse due to at least one of road conditions and traffic conditions, and receiving, by the vehicle an instruction regarding how the vehicle should respond to the impasse.

17. A method of using a self-driving vehicle, the method comprising:

identifying, by the vehicle, a need for a remote human interaction;

sending, in response to identifying the need, a first wireless communication to a remote computing device;

receiving, by the vehicle, the remote human interaction in response to the first wireless communication, wherein the receiving the remote human interaction comprises receiving a video recorded by a video camera of the remote computing device; and

showing the video on a display screen of the vehicle, wherein the display screen faces outward such that the display screen is configured to enable the person located outside the vehicle to see the video.

18. A self-driving vehicle configured to communicate with a remote computing device located outside the vehicle in response to identifying a need for a remote human interaction, the vehicle comprising:

a microphone system configured to detect an audible voice of a person located outside the vehicle;

a first wireless communication sent from the vehicle in response to the vehicle detecting the voice;

audio data recorded by a microphone of the remote computing device in response to the first wireless communication;

a speaker configured to emit a sound outside the vehicle based on the audio data; and

a recording of a verbal response to the sound, wherein the verbal response is from the person located outside the vehicle and is recorded by the microphone system of the vehicle.

19. The vehicle of claim 18, further comprising a threshold configured to help the vehicle differentiate between background voices and voices directed to the vehicle from a location outside of the vehicle.

20. The vehicle of claim 18, further comprising a question recognition system configured to facilitate determining, by the vehicle, that the voice has asked a question, wherein the first wireless communication is sent from the vehicle at least partially in response to the determining, by the vehicle, that the voice has asked the question.

21. The vehicle of claim 18, wherein the microphone system of the vehicle comprises a first microphone and a second microphone spaced apart from the first microphone, wherein the first and second microphones are configured to facilitate evaluating a directionality of the voice, wherein the first wireless communication is sent from the vehicle at least partially in response to the directionality of the voice.

22. The vehicle of claim 18, further comprising a video camera configured to detect the person located outside of the vehicle.

23. The vehicle of claim 18, further comprising a proximity sensor configured to detect the person located outside of the vehicle.

24. The vehicle of claim 18, further comprising at least one of an infrared detector and a radar detector configured to detect the person located outside of the vehicle.

25. The vehicle of claim 18, wherein the microphone system is configured to detect a knocking sound indicative of knocking on a portion of the vehicle, wherein the first wireless communication is sent from the vehicle at least partially in response to the detecting the knocking sound.

26. The vehicle of claim 18, further comprising a vibration sensor configured to detect a knock on a portion of the vehicle, wherein the first wireless communication is sent from the vehicle at least partially in response to the detecting the knock.

27. A self-driving vehicle configured to communicate with a remote computing device located outside the vehicle, the vehicle comprising:

a first video camera configured to record a first video of an area outside the vehicle;

a first wireless communication sent from the vehicle in response to the vehicle identifying a need for a remote human interaction;

a second wireless communication received by the vehicle in response to sending the first wireless communication, wherein the second wireless communication comprises audio data recorded by a first microphone of the remote computing device;

a speaker configured to emit a first sound, wherein the first sound is emitted outside the vehicle to enable a person located outside the vehicle hear the audio data; and

a third wireless communication and a fourth wireless communication sent from the vehicle to the remote computing device, wherein the third wireless communication comprises a recording of a verbal response to the first sound from the person located outside the vehicle, and the fourth wireless communication comprises the first video of the area outside the vehicle during the verbal response.

28. The vehicle of claim 27, further comprising a second video recorded by the remote computing device, wherein the vehicle comprises a display screen facing outward such that the display screen is configured to show the second video to the person located outside the vehicle.