Mobile devices and methods employing haptics

• US 9,836,929 B2
• Filed: 09/08/2015
• Issued: 12/05/2017
• Est. Priority Date: 07/09/2010
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Conwell, William Y.
• Primary Examiner(s): Tweel, Jr., John A
• Application Number:
  US14/847,677
• Publication Number:
  US 20160005278A1
• Time in Patent Office: 819 Days
• Field of Search: 3404072
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: H04M 1/725 Cordless telephones user interfaces specially adapted therefor H04M1/724
• CPC Class Codes: H04M 3/02 Calling substations, e.g. by ringing selective calling H04Q
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73

Abstract

A variety of haptic improvements useful in mobile devices are detailed. In one, a smartphone captures image data from a physical object, and discerns an object identifier from the imagery (e.g., using watermark, barcode, or fingerprint techniques). This identifier is sent to a remote data structure, which returns data defining a distinct haptic signature associated with that object. This smartphone then renders this haptic signal to the user. (Related embodiments identify the object using other means, such as location, or NFC chip.) In another arrangement, haptic feedback signals social network information about a product or place (e.g., the user'"'"'s social network friends “Like” a particular brand of beverage). In yet another arrangement, the experience of watching a movie on a television screen is augmented by tactile effects issued by a tablet computer on the viewer'"'"'s lap. In still another arrangement, commercial vendors bid for rights to employ different ones of a library of haptic signals on one or more users'"'"' smartphones, e.g., to alert such user(s) to their products/services. A great variety of other features and arrangements are also detailed.

Claims

1. A method involving a user'"'"'s camera-equipped mobile device interacting with a consumer product, the consumer product having been produced by a first party, the method comprising the acts:

with a camera of the mobile device, capturing image data from the consumer product;

recognition-processing the captured data to obtain identification data for the consumer product;

consulting a data structure with the identification data, to obtain data indicating a haptic signature associated with said consumer product, said haptic signature also having been defined by said first party; and

presenting the haptic signature to a user of the mobile device;

wherein the first party both creates the consumer product, and also defines the haptic signature presented to users interacting with that consumer product, wherein a single party defines both a look and feel associated with that product.

3. The method of claim 1 in which the recognition-processing comprises fingerprint extraction.

4. The method of claim 1 in which the recognition-processing comprises watermark decoding.

5. The method of claim 1 in which the recognition-processing comprises barcode decoding.

6. The method of claim 1 that includes consulting a data structure maintained by the first party, to obtain the data indicating the haptic signature associated with the consumer product.

7. The method of claim 1 that further includes the acts:

a microphone of the mobile device capturing sound data from a subject;

recognition-processing the captured sound data to obtain second identification data;

consulting a data structure with the second identification data to obtain data indicating a haptic signature associated with said subject; and

presenting said haptic signature associated with the subject to the user of the mobile device.

8. The method of claim 1 that further includes:

determining a location of the user'"'"'s mobile device;

presenting said haptic signature to the user when the camera captures image data from said consumer product at a first location; and

not presenting said haptic signature to the user when the camera captures image data from said consumer product at a second location;

wherein the mobile device is attuned to different objects based on its location.

9. The method of claim 1 that further includes:

discerning a feature on the consumer product;

by reference to said feature, displaying an augmented reality overlay on a screen of the mobile device;

in response to tilting of the mobile device, navigating along a path in said displayed augmented reality overlay; and

activating one or more of plural haptic transducers in the mobile device to provide directional feedback relating to the navigated path.

10. The method of claim 1 that includes presenting the haptic signature to the user by spaced-apart haptic actuators, at least one of which is remote from the camera-equipped mobile device.

11. The method of claim 10 that includes presenting the haptic signature to the user, at least in part, through haptic-equipped earbuds.

12. The method of claim 10 that includes presenting the haptic signature to the user, at least in part, through a haptic actuator within the user'"'"'s shoes.

13. The method of claim 10 that includes presenting the haptic signature to the user, at least in part, through a haptic actuator in eyewear worn by the user.

14. The method of claim 1 that includes rendering the haptic signature to the user at a speed determined in accordance with a user-set parameter, so that users who are adept at tactile sensing can set haptic signatures to be rendered at speeds above a default speed.

15. The method of claim 1 that further includes adapting a speed at which the haptic signature is rendered, based on environmental context.

16. The method of claim 1 that further includes adapting an amplitude at which the haptic signature is rendered, based on environmental context.

17. The method of claim 1 wherein said haptic signature associated with said consumer product was previously established through a process that included an auction.

18. The method of claim 1 in which the camera-equipped mobile device comprises a wearable computer.

2. A non-transitory computer readable medium containing software instructions for configuring a user'"'"'s mobile device programmed thereby to perform acts including:

capturing image data from a consumer product;

recognition-processing the captured data to obtain identification data for the consumer product;

consulting a data structure with the identification data, to obtain data indicating a haptic signature defined by a first party that produced the consumer product; and

presenting the haptic signature to a user of the mobile device.

19. A method comprising:

with a mobile device, capturing image data from a subject;

processing the captured data to obtain metadata corresponding to said subject, said processing including fingerprint extraction, watermark decoding or barcode decoding;

consulting a data structure with the metadata, to obtain data indicating a haptic signature associated with said subject; and

presenting the haptic signature to a user of the mobile device;

wherein the subject comprises a pharmaceutical or food product, and in which the haptic signature indicates a potentially adverse reaction to said pharmaceutical or food product.