Systems and methods for providing a haptic effect associated with a pgraphical simulation or virtual tag

• US 9,710,064 B2
• Filed: 10/14/2014
• Issued: 07/18/2017
• Est. Priority Date: 12/19/2008
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Ramsay, Erin B., Birnbaum, David, Bartel, Hendrik, Grant, Danny A., Cruz-Hernandez, Juan Manuel, Heubel, Robert W., Ullrich, Christopher J.
• Primary Examiner(s): YEN, JASON TAHAI
• Application Number:
  US14/514,026
• Publication Number:
  US 20150029125A1
• Time in Patent Office: 1,008 Days
• Field of Search: None
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality

Abstract

A system includes a display configured to display a graphical simulation, a computer configured to run a software program designed to display the graphical simulation on the display, and a stylus constructed and arranged to be manipulated by a user of the system in at least two dimensions to affect the graphical simulation. The stylus includes an actuator configured to output a haptic effect to the user upon receiving a control signal from the computer.

Claims

1. A system for providing a haptic effect, the system comprising:

a computer configured to run a software program designed to display a graphical simulation; and

a stylus configured to affect the graphical simulation based on movement of the stylus in at least two dimensions of a touch screen, the stylus comprising,

a sensor configured to track positional values of the stylus in the at least two dimensions of the touch screen and configured to transmit a sensor signal representative of the movement to the computer, wherein the computer is configured to update the displaying of the graphical simulation run by the software program based on the sensor signal representative of the movement of the stylus,

the stylus further comprising a processor and an actuator configured to output the haptic effect upon receiving a control signal from the processor, and wherein the processor is configured to cause the actuator to output the haptic effect while the stylus is in contact with the touch screen and as long as the movement of the stylus matches a stored gesture for the graphical simulation.

2. The system according to claim 1, wherein the graphical simulation comprises virtual art.

3. The system according to claim 1, wherein the touch screen is part of a display on which the computer is configured to update the graphical simulation based on the movement of the stylus.

4. The system according to claim 3, wherein the stylus comprises a body portion and a tip portion detachably coupled to the body portion, and wherein the sensor is configured to track movement of the tip portion in contact with the touch screen, and the computer is configured to update the graphical simulation on the display run by the software program based on the movement of the tip portion.

5. The system of claim 3, wherein the display is a display screen of the computer.

6. The system according to claim 1, wherein the sensor is a two or three dimensional capable accelerometer or gyroscope.

7. The system according to claim 1, wherein the sensor uses at least one of light, optics, infrared and RFID to track positional values of the stylus and to transmit the sensor signal representative of the movement to the computer.

8. The system of claim 1, wherein the sensor is a body sensor, the stylus further comprising a tip portion and a tip sensor configured to detect contact between the tip portion of the stylus and a surface of the touch screen, wherein the body sensor is configured to track movement of the stylus when the tip portion is in contact with or near the touch screen, and the computer is configured to cause the actuator to output the haptic effect after the tip sensor indicates that the tip portion of the stylus is in contact with the touch screen.

9. The system of claim 1, wherein

the stylus comprises a body portion and a tip portion attached to the body portion,

the sensor is a body sensor located in the body portion,

the actuator is a body actuator located in the body portion,

the tip portion comprises a tip sensor and a tip actuator located in the tip portion,

the tip sensor is configured to sense an amount of force applied to the stylus,

at least one actuator of the body actuator and the tip actuator comprises or is coupled to a motor, and

the stylus is configured to cause a magnitude of a haptic effect output by the body actuator or the tip actuator to depend on an amount of force applied to the stylus.

10. The system of claim 9, wherein the tip sensor is further configured to sense a distance that the tip portion has been moved by the force applied to the stylus, and wherein the stylus is configured to cause a magnitude of the haptic effect to depend on the distance that the tip portion has been moved by the force applied to the stylus.

11. A method comprising:

displaying a graphical simulation;

receiving an input from a stylus related to movement of the stylus in at least two dimensions of a touch screen to affect the graphical simulation;

tracking movement of the stylus in the at least two dimensions of the touch screen with a sensor on the stylus that is configured to track positional values of the stylus in the at least two dimensions of the touch screen;

updating the graphical simulation based on the input received from the sensor of the stylus; and

while the stylus is in contact with the touch screen, outputting a haptic effect via an actuator in the stylus as long as the movement of the stylus matches a stored gesture for the graphical simulation.

12. The method according to claim 11, wherein the graphical simulation is provided on a display, and the display comprises the touch screen.

13. The method according to claim 12, wherein the stylus comprises a body portion and a tip portion detachably coupled to the body portion, and wherein the sensor tracks movement of the tip portion of the stylus in contact with the touch screen, and wherein the method further comprises updating the graphical simulation on the display based on the movement of the tip portion.

14. The method according to claim 11, wherein the sensor is a two or three dimensional capable accelerometer or gyroscope.

15. The method according to claim 11, wherein the sensor uses at least one of light, optics, infrared and RFID to track positional values of the stylus.

16. A system for providing a haptic effect, the system comprising:

a computer configured to run a software program designed to display a graphical simulation on a display; and

a handheld controller that is representative of a writing or painting instrument, the handheld controller comprising a sensor configured to track positional values of the handheld controller along three dimensions in a three dimensional space and transmit a sensor signal representative of the positional values to the computer, and comprising an actuator configured to output the haptic effect upon receiving a control signal from a processor coupled to the handheld controller, wherein the processor is configured to generate the control signal that causes the haptic effect as long as movement of the handheld controller matches a stored gesture for the graphical simulation.

17. The system of claim 16, wherein the sensor is a three dimensional capable accelerometer or gyroscope.

18. The system of claim 16, wherein the handheld controller is a stylus.

19. The system of claim 16, wherein the sensor is a first sensor, and the positional values are first positional values that are based on movement of the handheld controller along the three dimensions in the three dimensional space, and wherein the handheld controller further comprises a second sensor that is a global positioning system (GPS) device configured to provide second positional values that indicate a geographical location of the handheld controller.