CONTEXT-BASED SMARTPHONE SENSOR LOGIC

• US 20160232898A1
• Filed: 11/20/2015
• Published: 08/11/2016
• Est. Priority Date: 04/04/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Bai, Yang
• Granted Patent: US 9,595,258 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06V 40/12 Fingerprints or palmprints
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/285 Memory allocation or algorithm optimisation to reduce hardware requirements
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10

Abstract

Methods employ sensors in portable devices (e.g., smartphones) both to sense content information (e.g., audio and imagery) and context information. Device processing is desirably dependent on both. For example, some embodiments activate certain processor intensive operations (e.g., content recognition) based on classification of sensed content and context. The context can control the location where information produced from such operations is stored, or control an alert signal indicating, e.g., that sensed speech is being transcribed. Some arrangements post sensor data collected by one device to a cloud repository, for access and processing by other devices. Multiple devices can collaborate in collecting and processing data, to exploit advantages each may have (e.g., in location, processing ability, social network resources, etc.). A great many other features and arrangements are also detailed.

Claims

1-9. -9. (canceled)

10. A method comprising the acts:

applying a first classification procedure to received audio and/or visual information, to identify a type of said received information from among two possible types;

a first type, and a second type;

applying a first combination of plural recognition technologies to the received information if the received information is identified as the first type; and

applying a second combination of plural recognition technologies to the received information if the received information is identified as the second type;

wherein at least one of the recognition technologies is a watermark- or fingerprint-based recognition technology, and the first, and second combinations are all different; and

wherein at least one of said acts is performed by hardware configured to perform such act(s).

11. The method of claim 10 that includes:

applying the first classification procedure to identify the type of said received information from among three possible types;

a first type, a second type, and a third type; and

applying a third combination of plural recognition technologies to the received information if the received information is identified as the third type;

wherein the first, second, and third combinations are all different.

38. The method of claim 10 in which the received audio and/or visual information comprises audio information.

39. The method of claim 38 in which:

one recognition technology of said first combination of plural recognition technologies, is selected from a group consisting of;

a first watermark decoding technology, a second watermark decoding technology, speech recognition, and music recognition; and

one recognition technology of said second combination of plural recognition technologies is also selected from said group.

40. The method of claim 39 in which two recognition technologies of said first combination of plural recognition technologies, are selected from said group, and two recognition technologies of said second combination of plural recognition technologies, are also selected from said group.

41. The method of claim 38 in which the first classification procedure classifies the received audio visual information into one of at least three classes, including (1), speech, (2) music, and (3) background/indeterminate.

42. The method of claim 38 in which the first classification procedure classifies the received audio visual information into one of at least four classes, including (1), speech, (2) music, (3) background/indeterminate, and (4) silence.

43. The method of claim 10 in which the received audio and/or visual information comprises visual information.

44. The method of claim 43 in which:

one recognition technology of said first combination of plural recognition technologies, is selected from a group consisting of;

watermark decoding, fingerprint-based image recognition, optical character recognition, facial recognition, and barcode decoding; and

one recognition technology of said second combination of plural recognition technologies, is also selected from said group.

45. The method of claim 44 in which two recognition technologies of said first combination of plural recognition technologies are selected from said group, and two recognition technologies of said second combination of plural recognition technologies are also selected from said group.

46. The method of claim 10 that further includes:

applying a second classification procedure to second information to determine a contextual scenario type from among plural contextual scenario types, said second information including information different than said received audio and/or visual information;

applying said first combination of plural recognition technologies to the received audio and/or visual information if the received audio and/or visual information is identified as the first type, and the contextual scenario type is determined to be a first contextual scenario type; and

applying a third combination of plural recognition technologies to the received audio and/or visual information if the received audio and/or visual information is identified as the first type, and the contextual scenario is determined to be a second contextual scenario type different than said first contextual scenario type;

wherein the first and third combinations of plural recognition technologies are different.

47. The method of claim 46 that further includes:

applying a fourth combination of plural recognition technologies to the received audio and/or visual information if the received audio and/or visual information is identified as the first type, and the contextual scenario is determined to be a third contextual scenario type different than said first and second contextual scenario types;

wherein the first, third and fourth combinations of plural recognition technologies are different.

48. The method of claim 46 in which the second information includes an item of information selected from the group consisting of:

time of day, day, day of week, location, calendar data, clock alarm status, motion sensor data, social network status, number of persons nearby, and identities of persons nearby.

49. The method of claim 48 in which the second information comprises two items of information selected from said group.

51. The method of claim 49 in which:

the received audio and/or visual information comprises audio information;

one recognition technology of said first combination of plural recognition technologies is a watermark decoding technology; and

one recognition technology of said second combination of plural recognition technologies is music recognition.

52. The method of claim 49 in which:

the received audio and/or visual information comprises audio information;

one recognition technology of said first combination of plural recognition technologies, is selected from a group consisting of;

a first watermark decoding technology, a second watermark decoding technology, speech recognition, and music recognition; and

one recognition technology of said second combination of plural recognition technologies is also selected from said group.

53. The method of claim 52 in which two recognition technologies of said first combination of plural recognition technologies, are selected from said group, and two recognition technologies of said second combination of plural recognition technologies, are also selected from said group.

55. The method of claim 49 in which:

the received audio and/or visual information comprises visual information;

one recognition technology of said first combination of plural recognition technologies is watermark decoding; and

one recognition technology of said second combination of plural recognition technologies is fingerprint-based image recognition.

56. The method of claim 49 in which:

the received audio and/or visual information comprises visual information;

one recognition technology of said first combination of plural recognition technologies, is selected from a group consisting of;

watermark decoding, fingerprint-based image recognition, optical character recognition, facial recognition, and barcode decoding; and

one recognition technology of said second combination of plural recognition technologies, is also selected from said group.

57. The method of claim 56 in which two recognition technologies of said first combination of plural recognition technologies, are selected from said group, and two recognition technologies of said second combination of plural recognition technologies, are also selected from said group.

12-37. -37. (canceled)

50. (canceled)

54. (canceled)