User interface device responsive to data tag associated with physical location

• US 10,591,995 B2
• Filed: 12/07/2018
• Issued: 03/17/2020
• Est. Priority Date: 12/19/2008
• Status: Active Grant

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Grant, Danny A., Ramsay, Erin B., Cruz-Hernandez, Juan Manuel, Birnbaum, David, Heubel, Robert W., Bartel, Hendrik, Ullrich, Christopher J.
• Primary Examiner(s): Yen, Jason T
• Application Number:
  US16/213,757
• Publication Number:
  US 20190107892A1
• Time in Patent Office: 466 Days
• Field of Search: None
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0483 Interaction with page-structured environments, e.g. book metaphor
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality

Abstract

A gaming concept in which each user in a single-player or a multi-player game is enabled to create artwork or graffiti work virtually or on actual surfaces using haptic-enhanced controllers. The controllers can emulate the experience of using a marker, a paint brush, a paint spray-can or the like. The components of various controllers may be modularized for easily interchanging components to extend the art or graffiti creation experience. The real life experience of using ink or paint on a surface is simulated by haptic feedback. When a paint spray-can controller is used, the experience of paint consumption is recreated by various peripheral enhancements including audio and haptic sensations communicated to the user.

Claims

1. A user interface device, comprising:

a haptic actuator configured to generate a haptic effect at the user interface device;

a display device;

a camera configured to capture an image of a physical structure or a physical surface at a current location of the user interface device; and

a processor in communication with the haptic actuator, the camera, and the display device, and configured;

to determine the current location of the user interface device,

to determine, based on the current location of the user interface device, whether-the user interface device is at a physical location associated with a data tag stored on a host computer separate from the user interface device, wherein the data tag comprises a visual design that comprises one of a logo, a symbol, or a drawing,

in response to a determination that the user interface device is at the physical location associated with the data tag stored on the host computer;

to determine that the haptic effect is to be generated by the haptic actuator of the user interface device,

to control the haptic actuator to generate the haptic effect, and

to control the display device to overlay the visual design of the data tag on the image of the physical structure or the physical surface.

2. The user interface device of claim 1, further comprising a global positioning system (GPS) receiver, wherein the processor is configured to determine the current location of the user interface device by using the GPS receiver.

3. The user interface device of claim 2, wherein the processor is configured to control a frequency of the haptic effect by determining the frequency of the haptic effect and providing a command to the haptic actuator to indicate the frequency that is determined, wherein the haptic actuator is configured to generate the haptic effect on the user interface device at the frequency determined by the processor.

4. The user interface device of claim 3, wherein the frequency is determined based on whether the movement of the user interface device follows a stored gesture.

5. The user interface device of claim 2, wherein the user interface device is configured to control a magnitude of the haptic effect by determining the magnitude of the haptic effect and providing a command to the haptic actuator to indicate the magnitude that is determined, wherein the haptic actuator is configured to generate the haptic effect on the user interface device at the magnitude determined by the processor.

6. The user interface device of claim 2, wherein the user interface device is configured to communicate the current location of the user interface device to the host computer, and is configured to receive from the host computer an indication that the current location of the user interface device is the same as the physical location associated with the data tag stored on the host computer.

7. The user interface device of claim 6, wherein the processor is further configured to receive from the remote host computer information describing the physical location associated with the data tag stored on the remote host computer, and wherein the determination of whether the user interface device is at the physical location associated with the data tag is made by comparing, at the user interface device, the current location of the user interface device with the physical location associated with the data tag.

8. The user interface device of claim 1, wherein the processor is further configured

to place an additional data tag at the current location by generating an additional visual design based on user input received at the user interface device, and

to communicate the additional visual design and the current location of the user interface device to the host computer.

9. The user interface device of claim 8, wherein the processor is configured to further communicate a user identifier associated with the user interface device to the host computer as part of placing the additional data tag.

10. The user interface device of claim 1, wherein the user interface device further comprises a motion sensor, and the processor is further configured to detect, based on a signal from the motion sensor, movement of the user interface device, and to cause the haptic actuator to generate another haptic effect as long as the movement of the user interface device matches a stored gesture.

11. A method performed by a processor of a user interface device, the user interface device having a haptic actuator, a display device, and a camera, and the method comprising:

determining a current location of the user interface device,

controlling the camera to capture an image of a physical structure or a physical surface at the current location of the user interface device;

determining, based on the current location of the user interface device, whether-the user interface device is at a physical location associated with a data tag stored on a host computer separate from the user interface device, wherein the data tag comprises a visual design that comprises one of a logo, a symbol, or a drawing,

in response to a determination that the user interface device is at the physical location associated with the data tag stored on the host computer;

determining that the haptic effect is to be generated by the haptic actuator of the user interface device,

controlling the haptic actuator to generate a haptic effect at the user interface device, and

controlling the display device to overlay the visual design of the data tag on the image of the physical structure or the physical surface.

12. A non-transitory computer-readable medium having instructions thereon that, when executed by a processor of a user interface device, causes the processor to perform the following:

determining a current location of the user interface device, the user interface device having a haptic actuator, a display device, and a camera;

controlling the camera to capture an image of a physical structure or a physical surface at the current location of the user interface device;

determining, based on the current location of the user interface device, whether-the user interface device is at a physical location associated with a data tag stored on a host computer separate from the user interface device, wherein the data tag comprises a visual design that comprises one of a logo, a symbol, or a drawing,

in response to a determination that the user interface device is at the physical location associated with the data tag stored on the host computer;

determining that the haptic effect is to be generated by the haptic actuator of the user interface device,

controlling the haptic actuator to generate a haptic effect at the user interface device, and

controlling the display device to overlay the visual design of the data tag on the image of the physical structure or the physical surface.