Sensory stimulation for an autonomous vehicle

• US 9,902,403 B2
• Filed: 06/17/2016
• Issued: 02/27/2018
• Est. Priority Date: 03/03/2016
• Status: Active Grant

• Current Assignee: Uber Technologies, Inc.
• Original Assignee: Uber Technologies, Inc.
• Inventors: Donnelly, Richard, Bradley, David McAllister, Sweeney, Matthew, Bartel, Emily
• Primary Examiner(s): PAIGE, TYLER D
• Application Number:
  US15/186,279
• Publication Number:
  US 20170253252A1
• Time in Patent Office: 620 Days
• Field of Search: 701 23
• CPC Class Codes: A61M 2021/0022 by the tactile sense, e.g. vibrations
• CPC Class Codes: A61M 2021/0027 by the hearing sense
• CPC Class Codes: A61M 2021/0044 by the sight sense
• CPC Class Codes: A61M 2021/005 images, e.g. video
• CPC Class Codes: A61M 21/00 Other devices or methods to cause a change in the state of consciousness; Devices for producing or ending sleep by mechanical, optical, or acoustical means, e.g. for hypnosis
• CPC Class Codes: B60N 2/02246 Electric motors therefor
• CPC Class Codes: B60N 2/04 the whole seat being movable
• CPC Class Codes: B60N 3/066 with protection systems against abnormal g-forces
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 30/025 related to comfort of drivers or passengers
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A sensory stimulation system for autonomous vehicle (AV) can monitor AV data indicating actions to be performed by the AV. The sensory stimulation system can further provide a user interface the enables riders of the AV to adjust output parameters of the sensory stimulation system. Based at least in part on the AV data, the sensory stimulation system can generate a set of stimulation commands for one or more of the output systems. Based on adjustments made by the at least one rider on the user interface, the sensory stimulation system can execute the stimulation commands on individual output systems.

Claims

1. A sensory stimulation system for an autonomous vehicle (AV), the sensory stimulation system comprising:

a number of output systems;

one or more processors; and

one or more memory resources storing instructions that, when executed by the one or more processors, causes the sensory stimulation system to;

monitor AV data indicating actions to be performed by the AV;

provide a user interface that enables a rider of the AV to adjust a number of output parameters of the sensory stimulation system;

based at least in part on the AV data, generate a set of stimulation commands for one or more of the output systems; and

based on adjustments made by the rider on the user interface, execute the stimulation commands on the one or more output systems.

2. The sensory stimulation system of claim 1, wherein the actions comprise acceleration, braking, and change of direction actions to be performed by the AV, and wherein the executed stimulation commands on the one or more output systems provide sensory cues to the rider in anticipation of the actions.

3. The sensory stimulation system of claim 1, wherein the user interface enables the rider to make timing adjustments to a timing parameter of the stimulation commands.

4. The sensory stimulation system of claim 3, wherein the timing adjustments cause the sensory stimulation system to execute stimulation commands prior to the actions being performed by the AV, and either earlier or later than a default timing setting.

5. The sensory stimulation system of claim 1, wherein the user interface enables the at least one rider to make intensity adjustments to an intensity parameter of the stimulation commands.

6. The sensory stimulation system of claim 1, wherein the output systems comprise a hand-puck system that the rider can optionally place a hand on to receive anticipatory sensory stimulation for the actions to be performed by the AV.

7. The sensory stimulation system of claim 1, wherein the output systems comprise a foot pedal system driven by at least one motor operated by the sensory stimulation system and mounted to a floor of the AV.

8. The sensory stimulation system of claim 7, wherein the foot pedal system comprises a foot pedal for each foot of the rider.

9. The sensory stimulation system of claim 7, wherein the foot pedal system is mounted to tilt fore and aft to provide preemptive sensory stimulation for acceleration and deceleration forces of the AV.

10. The sensory stimulation system of claim 9, wherein the foot pedal system is mounted on a universal joint to further tilt laterally in order to further provide preemptive sensory stimulation for lateral forces of the AV.

11. The sensory stimulation system of claim 1, wherein the output systems comprise a virtual reality (VR) headset to be worn by the rider, and wherein the executed stimulation commands cause a visual graphic to be displayed on a display of the VR headset that indicates upcoming forces of the AV.

12. An autonomous vehicle (AV) comprising:

a number of output systems;

a sensor system to generate sensor data indicating a situational environment of the AV;

acceleration, steering, and braking systems; and

a control system comprising one or more processors executing instructions that cause the control system to;

process the sensor data to determine a set of actions to be performed by the AV and autonomously control the acceleration, steering, and braking systems;

provide a user interface that enables a rider of the AV to adjust a number of output parameters of stimulation outputs from the output systems;

based at least in part on the determined set of actions, generate a set of stimulation commands for one or more of the output systems; and

based on adjustments made by the rider on the user interface, execute the stimulation commands on the one or more output systems.

13. The AV of claim 12, wherein the actions comprise acceleration, braking, and change of direction actions to be performed by the AV, and wherein the executed stimulation commands on the one or more output systems provide sensory cues to the rider in anticipation of the set of actions.

14. The AV of claim 12, wherein the user interface enables the rider to make timing adjustments to a timing parameter of the executed stimulation commands.

15. The AV of claim 14, wherein the timing adjustments cause the control system to execute stimulation commands prior to the set of actions of the AV, and either earlier or later than a default timing setting.

16. The AV of claim 12, wherein the user interface enables the rider to make intensity adjustments to an intensity parameter of the stimulation commands.

17. The AV of claim 12, wherein the output systems comprise a hand-puck system that the rider can optionally place a hand on to receive anticipatory sensory stimulation for the set of actions to be performed by the AV.

18. The AV of claim 12, wherein the output systems comprise a foot pedal system driven by at least one motor operated by the control system and mounted to a floor of the AV.

19. The AV of claim 18, wherein the pedal system is mounted on at least one universal joint to provide preemptive sensory stimulation for acceleration, deceleration, and lateral forces of the AV.

20. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a sensory stimulation system for an autonomous vehicle (AV), cause the one or more processors to:

monitor AV data indicating actions to be performed by the AV;

provide a user interface the enables a rider of the AV to adjust a number of output parameters of the sensory stimulation system;

based at least in part on the AV data, generate a set of stimulation commands for one or more of the output systems; and

based on adjustments made by the rider on the user interface, execute the stimulation commands on the one or more output systems.