Sensory stimulation system for an autonomous vehicle

• US 10,766,500 B2
• Filed: 08/01/2018
• Issued: 09/08/2020
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: Uber Technologies, Inc.
• Original Assignee: Uatc, LLC (Uber Technologies, Inc.)
• Inventors: Sweeney, Matthew, Bartel, Emily
• Primary Examiner(s): Paige, Tyler D
• Application Number:
  US16/051,910
• Publication Number:
  US 20180339715A1
• Time in Patent Office: 769 Days
• Field of Search: 701 23
• CPC Class Codes: A61M 2021/0022 by the tactile sense, e.g. vibrations
• CPC Class Codes: A61M 2021/0027 by the hearing sense
• CPC Class Codes: A61M 2021/0044 by the sight sense
• CPC Class Codes: A61M 21/00 Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
• CPC Class Codes: A61M 2205/332 Force measuring means
• CPC Class Codes: B60H 1/00764 the input being a vehicle driving condition, e.g. speed B60H1/00828, B60H1/00864 take precedence
• CPC Class Codes: B60N 2/0244 with logic circuits
• CPC Class Codes: B60N 2/14 rotatable, e.g. to permit easy access B60N2/10 , B60N2/2869 take precedence
• CPC Class Codes: B60N 2/185 characterised by the drive mechanism
• CPC Class Codes: B60N 2002/981 Warning systems, e.g. the seat or seat parts vibrates to warn the passenger when facing a danger
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60W 40/10 related to vehicle motion
• CPC Class Codes: B60W 50/16 Tactile feedback to the driver, e.g. vibration or force feedback to the driver on the steering wheel or the accelerator pedal
• CPC Class Codes: G05D 1/02 Control of position or course in two dimensions

Abstract

A sensory stimulation system within an interior of an autonomous vehicle (AV) can obtain maneuver data representing one or more maneuvers to be performed by the AV. For each respective maneuver of the one or more maneuvers, the system can determine one or more visual outputs to provide a passenger of the AV with a sensory indication of the respective maneuver, and then output the one or more visual outputs via the display system.

Claims

1. A sensory stimulation system within an interior of an autonomous vehicle (AV), the sensory stimulation system comprising:

a display system;

one or more processors; and

one or more memory resources storing instructions that, when executed by the one or more processors, causes the sensory stimulation system to;

obtain maneuver data representing one or more maneuvers to be performed by the AV;

for each respective maneuver of the one or more maneuvers, determine one or more visual outputs to provide a passenger of the AV with a sensory indication of the respective maneuver; and

output the one or more visual outputs via the display system.

2. The sensory stimulation system of claim 1, wherein the one or more maneuvers comprise acceleration, braking, and change of direction actions to be executed by the AV.

3. The sensory stimulation system of claim 1, wherein the executed instructions further cause the sensory stimulation system to:

output a live presentation on the display system, the live presentation presenting the AV operating along a current route.

4. The sensory stimulation system of claim 3, wherein the live presentation comprises LIDAR data from at least one LIDAR sensor of the AV, the LIDAR data providing a sensor view of a surrounding environment of the AV.

5. The sensory stimulation system of claim 3, wherein the live presentation comprises live video data from one or more cameras of the AV.

6. The sensory stimulation system of claim 5, wherein the executed instructions cause the sensory stimulation system to display the live presentation by generating augmented reality features presented on the live video data, the augmented reality features representing actual features in a surrounding environment of the AV.

7. The sensory stimulation system of claim 6, wherein the augmented reality features comprise (i) an augmented reality representation of the AV traveling along a current route, and (ii) augmented reality representations of proximate external entities to the AV.

8. The sensory stimulation system of claim 7, wherein the proximate external entities comprise proximate pedestrians and proximate vehicles to the AV.

9. The sensory stimulation system of claim 3, wherein the live presentation includes a live map indicating route information of a current route traveled by the AV.

10. The sensory stimulation system of claim 1, wherein the executed instructions cause the sensory stimulation system to output the one or more visual outputs via the display system as preemptive visual indications of each of the one or more maneuvers, the preemptive visual indications of each maneuver being displayed prior to the maneuver being executed.

11. An autonomous vehicle (AV) comprising:

a sensor system to generate sensor data indicating a surrounding environment of the AV;

a display system;

acceleration, steering, and braking systems; and

a control system comprising one or more processing resources executing instructions that cause the control system to;

obtain maneuver data representing one or more maneuvers to be performed by the AV;

for each respective maneuver of the one or more maneuvers, determine one or more visual outputs to provide a passenger of the AV with a sensory indication of the respective maneuver; and

output the one or more visual outputs via the display system.

12. The AV of claim 11, wherein the one or more maneuvers comprise acceleration, braking, and change of direction actions to be executed by the AV.

13. The AV of claim 11, wherein the executed instructions further cause the control system to:

output a live presentation on the display system, the live presentation presenting the AV operating along a current route.

14. The AV of claim 13, wherein the sensor system comprises at least one LIDAR sensor, and wherein the live presentation comprises LIDAR data from the at least one LIDAR sensor of the AV, the LIDAR data providing a sensor view of the surrounding environment of the AV.

15. The AV of claim 13, wherein the sensor system comprises one or more cameras, and wherein the live presentation comprises live video data from the one or more cameras.

16. The AV of claim 15, wherein the executed instructions cause the control system to display the live presentation by generating augmented reality features presented on the live video data, the augmented reality features representing actual features in the surrounding environment of the AV.

17. The AV of claim 16, wherein the augmented reality features comprise (i) an augmented reality representation of the AV traveling along a current route, and (ii) augmented reality representations of proximate external entities to the AV.

18. The AV of claim 17, wherein the proximate external entities comprise proximate pedestrians and proximate vehicles to the AV.

19. The AV of claim 11, wherein the executed instructions cause the control system to output the one or more visual outputs via the display system as preemptive visual indications of each of the one or more maneuvers, the preemptive visual indications of each maneuver being displayed prior to executing the maneuver.

20. A non-transitory computer readable medium storing instructions for providing sensory stimulation for one or more riders to indicate motion of an autonomous vehicle (AV), wherein the instructions, when executed by one or more processors, cause the one or more processors to:

obtain maneuver data representing one or more maneuvers to be performed by the AV;

for each respective maneuver of the one or more maneuvers, determine one or more visual outputs to provide a passenger of the AV with a sensory indication of the respective maneuver; and

output the one or more visual outputs via a display system of the AV.