System for haptically representing sensor input

• US 10,234,948 B2
• Filed: 11/17/2017
• Issued: 03/19/2019
• Est. Priority Date: 08/29/2012
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Birnbaum, David, Ullrich, Christopher J., Grant, Danny, Modarres, Ali, Cruz-Hernandez, Juan Manuel
• Primary Examiner(s): Lau, Hoi C
• Application Number:
  US15/816,336
• Publication Number:
  US 20180074592A1
• Time in Patent Office: 487 Days
• Field of Search: 340 412, 34053912, 3404071, 3404072, 3405731, 3405733, 340575, 340475, 345156, 345158, 345173
• CPC Class Codes: A61B 5/0002 Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network A61B5/07 takes precedence; transmission systems for measured values G08C; transmission H04B; transmission of digital information H04L; wireless communication networks H04W
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G09B 21/003 using tactile presentation of the information, e.g. Braille displays

Abstract

A haptic representation system is provided that generates a haptic effect in response to sensor input. The sensor input is mapped to a haptic signal. The haptic signal is sent to an actuator configured to receive the haptic signal. The actuator utilizes the haptic signal to generate the haptic effect.

Claims

1. A method of generating a haptic effect on a handheld device, the method comprising:

receiving input from at least one sensor including a light sensor, the input including visible light information and extra-sensory information;

mapping the visible light information to a video signal;

displaying the video signal on a display screen coupled to the handheld device;

mapping the extra-sensory information to a haptic signal; and

sending the haptic signal to a haptic output device coupled to the handheld device to generate the haptic effect,

wherein the haptic effect deforms at least a portion of the display screen to augment the video signal.

2. The method of claim 1, wherein the extra-sensory information includes electromagnetic field information, infra-red light information, ultraviolet light information, radiation information, subtle temperature change information, barometric pressure information or galvanic skin response information.

3. The method of claim 2, wherein the at least one sensor includes a magnetometer, a radiation sensor, a temperature sensor, a barometric pressure sensor or a galvanic skin response sensor.

4. The method of claim 2, wherein the light sensor sends the extra-sensory information to the handheld device, and the extra-sensory information includes at least one of infra-red light information and ultraviolet light information.

5. The method of claim 1, wherein:

the input from the sensor is continuously received; and

said mapping the extra-sensory information to the haptic signal includes continuously modulating the haptic signal based on a continuous updating of the extra-sensory information.

6. The method of claim 5, wherein:

the input includes a plurality of interaction parameters, each interaction parameter including a value; and

said mapping the extra-sensory information to the haptic signal includes continuously modulating the haptic signal based on a continuous updating of the value of at least one interaction parameter.

7. A handheld device, comprising:

at least one sensor including a light sensor;

a haptic output device;

a display screen;

a memory; and

a processor, coupled to the sensor, the haptic output device, the display screen, and the memory, the processor configured to;

receive input from the at least one sensor, the input including visible light information and extra-sensory information;

map the visible light information to a video signal;

display the video signal on the display screen;

map the extra-sensory information to a haptic signal; and

send the haptic signal to the haptic output device to generate a haptic effect,

wherein the haptic effect deforms at least a portion of the display screen to augment the video signal.

8. The handheld device of claim 7, wherein the extra-sensory information includes electromagnetic field information, infra-red light information, ultraviolet light information, radiation information, subtle temperature change information, barometric pressure information or galvanic skin response information.

9. The handheld device of claim 8, wherein the at least one sensor includes a magnetometer, a radiation sensor, a temperature sensor, a barometric pressure sensor or a galvanic skin response sensor.

10. The handheld device of claim 8, wherein the light sensor sends the extra-sensory information to the handheld device, and the extra-sensory information includes at least one of infra-red light information and ultraviolet light information.

11. The handheld device of claim 7, wherein:

the input from the sensor is continuously received; and

to map the extra-sensory information to the haptic signal includes continuously modulating the haptic signal based on a continuous updating of the extra-sensory information.

12. The handheld device of claim 11, wherein:

the input includes a plurality of interaction parameters, each interaction parameter including a value; and

to map the extra-sensory information to the haptic signal includes continuously modulating the haptic signal based on a continuous updating of the value of at least one interaction parameter.

13. A non-transitory computer-readable medium storing instructions thereon that, when executed by a processor, perform a method of generating a haptic effect on a handheld device, the method comprising:

receiving input from at least one sensor including a light sensor, the input including visible light information and extra-sensory information;

mapping the visible light information to a video signal;

displaying the video signal on a display screen coupled to the handheld device;

mapping the extra-sensory information to a haptic signal; and

sending the haptic signal to a haptic output device coupled to the handheld device to generate the haptic effect,

wherein the haptic effect deforms at least a portion of the display screen to augment the video signal.

14. The non-transitory computer-readable medium of claim 13, wherein the extra-sensory information includes electromagnetic field information, infra-red light information, ultraviolet light information, radiation information, subtle temperature change information, barometric pressure information or galvanic skin response information.

15. The non-transitory computer-readable medium of claim 14, wherein the at least one sensor includes a magnetometer, a radiation sensor, a temperature sensor, a barometric pressure sensor or a galvanic skin response sensor.

16. The non-transitory computer-readable medium of claim 14, wherein the light sensor sends the extra-sensory information to the handheld device, and the extra-sensory information includes at least one of infra-red light information and ultraviolet light information.

17. The non-transitory computer-readable medium of claim 13, wherein:

the input from the sensor is continuously received; and

said mapping the extra-sensory information to the haptic signal includes continuously modulating the haptic signal based on a continuous updating of the extra-sensory information.

18. The non-transitory computer-readable medium of claim 13, wherein:

the input includes a plurality of interaction parameters, each interaction parameter including a value; and

mapping the extra-sensory information to the haptic signal includes continuously modulating the haptic signal based on a continuous updating of the value of at least one interaction parameter.