CONTEXT-BASED SMARTPHONE SENSOR LOGIC

• US 20130150117A1
• Filed: 09/07/2012
• Published: 06/13/2013
• Est. Priority Date: 09/23/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Bai, Yang
• Granted Patent: US 9,196,028 B2
• Time in Patent Office: Days
• US Class Current: 455/550.1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 40/12 Fingerprints or palmprints
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion

Abstract

Methods employ sensors in portable devices (e.g., smartphones) both to sense content information (e.g., audio and imagery) and context information. Device processing is desirably dependent on both. For example, some embodiments activate certain processor intensive operations (e.g., content recognition) based on classification of sensed content and context. The context can control the location where information produced from such operations is stored, or control an alert signal indicating, e.g., that sensed speech is being transcribed. Some arrangements post sensor data collected by one device to a cloud repository, for access and processing by other devices. Multiple devices can collaborate in collecting and processing data, to exploit advantages each may have (e.g., in location, processing ability, social network resources, etc.). A great many other features and arrangements are also detailed.

Claims

1. A method comprising:

applying a first classification procedure to received audio and/or visual information, to identify a type of said received information from among plural possible types;

applying a second classification procedure to received second information, to identify a scenario, from among plural possible scenarios, the identified scenario comprising the confluence of at least three circumstances, the received second information being different than said received audio or visual information; and

activating one or more recognition modules based on outputs from the first and second classification procedures;

wherein at least one of said acts is performed by hardware configured to perform such act(s).

2. The method of claim 1 in which the first and second classification procedures identify a temporal-geographic-auditory scenario by reference to time, place, and audio information, and the method includes activating said one or more recognition modules based on said identified temporal-geographic-auditory scenario.

3. The method of claim 1 that includes activating a facial recognition module.

4. The method of claim 1 that includes:

applying a first classification procedure to received audio information, to identify the received audio information as comprising speech;

activating a speech recognition module to process the audio information; and

identifying a storage medium in which output from said speech recognition module should be stored, based, at least in part, on a location at which said audio information was captured.

5. The method of claim 1 that includes activating plural different recognition modules based on outputs from the first and second classification procedures.

6. The method of claim 1 that includes applying outputs from the first and second classification procedures to a rules module, which identifies one or more recognition modules to activate.

7. The method of claim 1 in which the received second information comprises information output from the first classification procedure.

8. The method of claim 1 in which the received second information comprises social network information.

9. The method of claim 1 in which the received second information comprises information about one or persons who are nearby.

38. The method of claim 1 in which the three circumstances are characterized by factors selected from a list consisting of:

time information, location information, ambient light information, information from a first camera of a user portable device, information from a second camera of the user portable device, accelerometer information, alarm information, calendar information, social network information, and information about a person in proximity to the user.

39. The method of claim 38 in which the identified scenario comprises the confluence of at least five circumstances selected from said list.

40. The method of claim 38 in which the identified scenario comprises the confluence of at least seven circumstances selected from said list.

41. The method of claim 1 that includes activating an audio fingerprint-based recognition module based on outputs from the first and second classification procedures.

42. The method of claim 1 that includes activating a digital watermark-based recognition module based on outputs from the first and second classification procedures.

10. A method comprising:

applying a first classification procedure to received audio and/or visual information, to identify a type of said received information from among two possible types;

a first type, and a second type;

applying a first combination of plural recognition technologies to the received information if the received information is identified as the first type; and

applying a second combination of plural recognition technologies to the received information if the received information is identified as the second type;

wherein at least one of the recognition technologies is a watermark- or fingerprint-based recognition technology, and the first, and second combinations are all different; and

wherein at least one of said acts is performed by hardware configured to perform such act(s).

11. The method of claim 10 that includes:

applying the first classification procedure to identify the type of said received information from among three possible types;

a first type, a second type, and a third type; and

applying a third combination of plural recognition technologies to the received information if the received information is identified as the third type;

wherein the first, second, and third combinations are all different.

12-37. -37. (canceled)

43. A portable user device including an image or sound sensor, a processor, and a memory, the memory containing software instructions causing the device to perform a method that includes:

applying a first classification procedure to information received by said image or sound sensor, to identify a type of said received information from among plural possible types;

applying a second classification procedure to received second information, to identify a scenario, from among plural possible scenarios, the identified scenario comprising the confluence of at least three circumstances, the received second information being different than said received audio or visual information; and

activating one or more recognition modules based on outputs from the first and second classification procedures.

44. The device of claim 43 in which the three circumstances are characterized by factors selected from a list consisting of:

time information, location information, ambient light information, information from a first camera of a user portable device, information from a second camera of the user portable device, accelerometer information, alarm information, calendar information, social network information, and information about a person in proximity to the user.

45. The device of claim 44 in which the identified scenario comprises the confluence of at least five circumstances selected from said list.

46. The device of claim 45 in which the identified scenario comprises the confluence of at least seven circumstances selected from said list.