Systems and methods for a haptically-enabled projected user interface

• US 9,965,034 B2
• Filed: 12/30/2013
• Issued: 05/08/2018
• Est. Priority Date: 12/30/2013
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Levesque, Vincent, Modarres, Ali, Weddle, Amaya B., Birnbaum, David M.
• Primary Examiner(s): Tran, Tam
• Application Number:
  US14/143,675
• Publication Number:
  US 20150185841A1
• Time in Patent Office: 1,590 Days
• Field of Search: 715701, 715702, 715726, 715863, 715864
• CPC Class Codes: G06F 1/1639 the display being based on projection
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

One illustrative system disclosed herein includes a sensor configured to detect a user input associated with a projected user interface and transmit a sensor signal associated with the user interaction, and a processor in communication with the sensor, the processor configured to: receive the sensor signal, determine a remote haptic effect based at least in part on the user input, generate a haptic signal based at least in part on the remote haptic effect, and transmit the haptic signal. The illustrative system further includes a haptic output device in communication with the processor, the haptic output device configured to receive the haptic signal and output the remote haptic effect.

Claims

1. A system comprising:

a sensor configured to detect an interaction with a projected user interface that is projected onto a surface and transmit a sensor signal associated with the interaction;

a processor in communication with the sensor;

a memory device on which instructions executable by the processor are stored to cause the processor to;

receive the sensor signal;

determine a characteristic of the surface based at least in part on at least one sensor signal from at least one sensor;

determine a remote haptic effect based at least in part on the interaction and the characteristic of the surface;

generate a haptic signal based at least in part on the remote haptic effect; and

transmit the haptic signal; and

a haptic output device in communication with the processor and remote from the surface, the haptic output device configured to receive the haptic signal and output the remote haptic effect by projecting an emission comprising at least one of a solid, liquid, gas, plasma, or sound pressure wave outwardly from the haptic output device.

2. The system of claim 1, wherein the sensor comprises one or more of a camera, an accelerometer, a gyroscope, a depth sensor, or a touch-sensitive surface.

3. The system of claim 1, further comprising a projector for generating the projected user interface.

4. The system of claim 1, wherein the haptic output device comprises one or more of an ultrasonic transducer, a speaker, or a jet configured to project the emission outwardly from the haptic output device to create the remote haptic effect.

5. The system of claim 1, wherein the processor and memory device are positioned in a hand-held computing device that comprises a smartphone, tablet, e-reader, or pocket organizer.

6. The system of claim 1, wherein the projected user interface comprises a hidden interface component, and the interaction is associated with the hidden interface component.

7. The system of claim 1, wherein the characteristic comprises a material of the surface.

8. The system of claim 1, wherein the memory device further includes instructions executable by the processor to cause the processor to determine the remote haptic effect based at least in part on a distance between the haptic output device and a body part to which the remote haptic effect is to be applied.

9. The system of claim 1, wherein the characteristic comprises a fundamental frequency of the surface.

10. The system of claim 1, wherein the characteristic comprises a topology of the surface.

11. The system of claim 1, wherein the sensor, the processor, and the haptic output device are disposed within a single device.

12. The system of claim 1, wherein the emission comprises the sound pressure wave and the haptic output device is configured to project the sound pressure wave outwardly to create the remote haptic effect.

13. The system of claim 1, wherein the emission comprises the solid or the liquid and the haptic output device is configured to project the solid or the liquid outwardly to create the remote haptic effect.

14. The system of claim 1, wherein the emission comprises the gas and the haptic output device is configured to project the gas outwardly to create the remote haptic effect.

15. The system of claim 1, wherein the haptic output device is a first haptic output device, the remote haptic effect is a first haptic effect, and further comprising a second haptic output device coupled to the surface and configured to output a second haptic effect.

16. The system of claim 15, wherein the memory device further comprises instructions executable by the processor to cause the processor to:

determine the second haptic effect based at least in part on the interaction; and

cause the second haptic output device to output the second haptic effect concurrently with the first haptic output device outputting the first haptic effect.

17. The system of claim 15, wherein the interaction is a first interaction, and wherein the memory device further comprises instructions executable by the processor to cause the processor to:

determine a haptic effect to output based on a second interaction;

select a particular haptic-output device from among a group of haptic output devices comprising the first haptic output device and the second haptic output device, wherein the particular haptic-output device is selected from among the group of haptic output devices based on the haptic effect; and

cause the particular haptic-output device to output the haptic effect.

18. A method comprising:

receiving, by a processor and from a sensor, a sensor signal associated with an interaction with a projected user interface that is projected onto a surface;

determining, by the processor, a characteristic of the surface based at least in part on at least one sensor signal from at least one sensor;

determining, by the processor, a remote haptic effect based at least in part on the interaction and the characteristic of the surface; and

causing, by the processor, a haptic output device that is positioned remotely from the surface to output the remote haptic effect to a body part by projecting an emission comprising at least one of a solid, liquid, gas, plasma, or sound pressure wave to the body part.

19. The method of claim 18, wherein the sensor comprises one or more of a camera, an accelerometer, a gyroscope, a depth sensor, or a touch-sensitive surface.

20. The method of claim 18, wherein the haptic output device comprises one or more of an ultrasonic transducer, a speaker, or a jet configured to project the emission outwardly from the haptic output device toward the body part.

21. A non-transient computer readable medium comprising program code, which when executed by a processor is configured to cause the processor to:

receive a sensor signal associated with an interaction with a projected user interface that is projected onto a surface;

determine a characteristic of the surface based at least in part on at least one sensor signal from at least one sensor;

determine a remote haptic effect based at least in part on the interaction and the characteristic of the surface;

generate a haptic signal based at least in part on the remote haptic effect; and

transmit the haptic signal associated with the remote haptic effect to a haptic output device positioned remotely from the surface, the haptic signal being configured cause the haptic output device to output the remote haptic effect by projecting an emission comprising at least one of a solid, liquid, gas, plasma, or concentrated pressure wave outwardly from the haptic output device.

22. The non-transient computer readable medium of claim 21, wherein the haptic output device comprises one or more of an ultrasonic transducer, a speaker, or a jet configured to project the emission to create the remote haptic effect.