User interface device responsive to data tag associated with physical location

• US 10,152,134 B2
• Filed: 06/13/2017
• Issued: 12/11/2018
• Est. Priority Date: 12/19/2008
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Grant, Danny A., Ramsay, Erin B., Cruz-Hernandez, Juan Manuel, Birnbaum, David, Heubel, Robert W., Bartel, Hendrik, Ullrich, Christopher J.
• Primary Examiner(s): Yen, Jason
• Application Number:
  US15/621,227
• Publication Number:
  US 20170277264A1
• Time in Patent Office: 546 Days
• Field of Search: None
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality

Abstract

A system includes a display configured to display a graphical simulation, a computer configured to run a software program designed to display the graphical simulation on the display, and a stylus constructed and arranged to be manipulated by a user of the system in at least two dimensions to affect the graphical simulation. The stylus includes an actuator configured to output a haptic effect to the user upon receiving a control signal from the computer.

Claims

1. A user interface device, comprising:

a global positioning system (GPS) receiver;

a haptic actuator configured to generate a haptic effect at the user interface device;

a display device; and

a processor in communication with the GPS receiver, the haptic actuator, and the display device, and configured;

to determine, with the GPS receiver, a current location of the user interface device,

to determine, based on the current location of the user interface device, whether the user interface device is at a physical location associated with a data tag stored on a remote host computer, wherein the data tag comprises a visual design or a photo,

in response to a determination that the user interface device is at the physical location associated with the data tag stored on the remote host computer;

to determine that the haptic effect is to be generated by the haptic actuator of the user interface device,

to control a frequency of the haptic effect by determining the frequency of the haptic effect and providing a command to the haptic actuator to indicate the frequency that is determined, wherein the haptic actuator is configured to generate the haptic effect on the user interface device at the frequency determined by the processor,

and

to control the display device to display the visual design or the photo of the data tag.

2. The user interface device of claim 1, further comprising:

a camera configured to capture an image of a physical structure or a physical surface, wherein the processor is configured to cause the display device to display the visual design or the photo by overlaying the visual design or the photo on the image of the physical structure or the physical surface.

3. The user interface device of claim 1, wherein the user interface device is configured to communicate the current location of the user interface device to the remote host computer, and is configured to receive from the remote host computer an indication that the current location of the user interface device is the same as the physical location associated with the data tag stored on the remote host computer, and

wherein the determination of whether the user interface device is at the physical location associated with the data tag is based on the indication received from the remote host computer.

4. The user interface device of claim 3, wherein the physical location is one of a plurality of physical locations associated with a plurality of respective data tags stored on the remote host computer.

5. The user interface device of claim 1, wherein the processor is further configured to receive from the remote host computer information describing the physical location associated with the data tag stored on the remote host computer, and wherein the determination of whether the user interface device is at the physical location associated with the data tag is made by comparing, at the user interface device, the current location of the user interface device with the physical location associated with the data tag.

6. The user interface device of claim 1, wherein the user interface device is configured to display the visual design of the data tag, and wherein the visual design comprises one of a logo, a symbol, and a drawing.

7. The user interface device of claim 1, wherein the processor is further configured

to place an additional data tag at the current location by generating an additional visual design based on user input received at the user interface device, and

to communicate the additional visual design and the current location of the user interface device to the remote host computer.

8. The user interface device of claim 7, wherein the processor is configured to further communicate a user identifier to the remote host computer as part of placing the additional data tag.

9. The user interface device of claim 1, wherein the user interface device further comprises a motion sensor, and the processor is further configured to detect, based on a signal from the motion sensor, movement of the user interface device, and to cause the haptic actuator to generate another haptic effect as long as the movement of the user interface device matches a stored gesture.

10. A method performed by a user interface device having a global positioning system (GPS) receiver, a haptic actuator, a display device, and a processor, the method comprising:

determining, by the processor and based on a signal from the GPS receiver, a current location of the user interface device;

determining, by the processor and based on the current location of the user interface device, whether the user interface device is at a physical location associated with a data tag stored on a remote host computer, wherein the data tag comprises a visual design or a photo;

in response to a determination that the user interface device is at the physical location associated with the data tag stored on the remote host computer;

determining, by the processor, that a haptic effect is to be generated by the haptic actuator of the user interface device,

determining, by the processor, a frequency of the haptic effect based on movement of the user interface device,

providing, by the processor, a command to the haptic actuator to indicate the frequency that is determined,

displaying, by the display device, the visual design or the photo of the data tag, and

generating, by the haptic actuator, the haptic effect on the user interface device at the frequency determined by the processor.

11. The method of claim 10, wherein the user interface device further comprises a camera configured to capture an image of a physical structure or a physical surface, and wherein the step of displaying the visual design or the photo comprises causing the display device to overlay the visual design or the photo on the image of the physical structure or the physical surface.

12. The method of claim 10, further comprising:

communicating the current location of the user interface device to the remote host computer; and

receiving from the remote host computer an indication that the current location of the user interface device is the same as the physical location associated with the data tag,

wherein the step of determining whether the user interface is at the physical location associated with the data tag is based on the indication from the remote host computer.

13. The method of claim 12, wherein the physical location is one of a plurality of physical locations associated with a plurality of respective data tags stored on the remote host computer.

14. The method of claim 10, further comprising:

receiving from the remote host computer information describing the physical location associated with the data tag stored on the remote host computer, and wherein the step of determining whether the user interface device is at the physical location associated with the data tag comprises the processor comparing the current location of the user interface device with the physical location of the data tag.

15. The method of claim 10, wherein the visual design of the data tag is displayed, and comprises one of a logo, a symbol, and a drawing.

16. The method of claim 10, further comprising:

placing an additional data tag at the current location by generating an additional visual design based on user input received at the user interface device; and

communicating the additional visual design and the current location of the user interface device to the remote host computer.

17. The method of claim 16, further comprising:

communicating a user identifier to the remote host computer as part of placing the additional data tag.

18. The method of claim 10, wherein the frequency is determined based on whether the movement of the user interface device follows a stored gesture.

19. A method performed by a user interface device having a global positioning system (GPS) receiver, a display device, a processor, and a camera, the method comprising:

determining, by the processor and based on a signal from the GPS receiver, a current location of the user interface device;

determining, by the processor and based on the current location of the user interface device, whether the user interface device is at a physical location associated with a data tag stored on a remote host computer, wherein the data tag comprises a visual design or a photo;

in response to a determination that the user interface device is at the physical location associated with the data tag stored on the remote host computer

determining, by the processor, that a haptic effect is to be generated by a haptic actuator in communication with the processor,

determining, by the processor, a frequency of the haptic effect based on movement of the user interface device,

providing, by the processor, a command to the haptic actuator to indicate the frequency that is determined,

generating, by the haptic actuator, the haptic effect on the user interface device at the frequency determined by the processor, and

displaying, by the display device, the visual design or the photo of the data tag by overlaying the visual design or the photo on an image captured by the camera.