METHODS AND ARRANGEMENTS EMPLOYING SENSOR-EQUIPPED SMART PHONES

• US 20110159921A1
• Filed: 12/30/2010
• Published: 06/30/2011
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Conwell, William Y., Rhoads, Geoffrey B., Rodriguez, Tony F., Davis, Bruce L.
• Granted Patent: US 9,143,603 B2
• Time in Patent Office: Days
• US Class Current: 455/556.1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 2200/1636 Sensing arrangement for detection of a tap gesture on the housing
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06Q 10/109 Time management, e.g. calendars, reminders, meetings or time accounting
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 19/047 Vibrating means for incoming calls
• CPC Class Codes: H04M 2250/02 including a Bluetooth interface
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04M 2250/74 with voice recognition means
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

The present technology concerns improvements to smart phones and related sensor-equipped systems. Some embodiments relate to smart phone-assisted commuting, e.g., by bicycle. Some involve novel human-computer interactions, e.g., using tactile grammars—some of which may be customized by users. Others involve spoken clues, e.g., by which a user can assist a smart phone in identifying what portion of imagery captured by a smart phone camera should be processed, or identifying what type of image processing should be conducted. Some arrangements include the degradation of captured content information in accordance with privacy rules, which may be location-dependent, or based on the unusualness of the captured content, or responsive to later consultation of the stored content information by the user. A great variety of other features and arrangements are also detailed.

Claims

1-5. -5. (canceled)

6. A method employing a portable user device having at least one microphone that captures audio, and at least one image sensor for capturing imagery, the method comprising the acts:

sending, to a speech recognition module, audio data corresponding to user speech captured by the microphone, and receiving recognized user speech data corresponding thereto; and

by reference to said recognized user speech data, determining an image portion of interest to the user within imagery captured by the image sensor.

7. The method of claim 6 that further includes, based at least in part on said determined image portion, selecting one or more operations to be applied in connection with the captured imagery, from a larger set of possible operations.

8. The method of claim 7 that further includes performing said selected operation(s).

9. The method of claim 6 in which the audio data sent to the speech recognition module includes audio data corresponding to a verb and a noun of user speech.

10. The method of claim 6 in which the audio data sent to the speech recognition module includes audio data corresponding to a noun of user speech, and the method further includes consulting a data structure to identify visual information associated with said noun, and using said identified visual information to determine said image portion of interest.

11. A method employing a portable user device having at least one microphone that captures audio, and at least one image sensor for capturing imagery, the method comprising the acts:

sending, to a speech recognition module, first audio data corresponding to first user speech captured by the microphone, and receiving first response data from the speech recognition module;

inviting the user to provide a further spoken clue;

thereafter sending, to the speech recognition module, second audio data corresponding to second user speech captured by the microphone, and receiving second response data from the speech recognition module; and

by reference to said received data, determining an image portion of interest to the user, within imagery captured by the image sensor.

12. A method employing a portable user device having at least one microphone that captures audio, and at least one image sensor for capturing imagery, the method comprising the acts:

sending, to a speech recognition module, audio data corresponding to user speech captured by the microphone, and receiving recognized user speech data corresponding thereto, the received user speech data including a word related to a subject depicted in imagery captured by the image sensor; and

based at least in part on said word, selecting one or more operations to be applied in connection with the captured imagery, from a larger set of possible operations.

13. The method of claim 12 in which the word is descriptive of a color of the subject.

14. The method of claim 12 in which the word is descriptive of a shape of the subject.

15. The method of claim 12 in which the word is a name of the subject.

16-31. -31. (canceled)

32. A method comprising:

storing content captured at a first location by a portable user device, the content including audio and/or imagery;

after passage of a first set interval of time, automatically degrading the audio and/or imagery content, in accordance with a stored privacy rule relating to retention of captured content;

wherein the method further includes performing at least one of the following content processing operations before degrading the content, and retaining a result of said content processing operation(s) for at least a further interval of time following said degrading;

(a) recognizing, from audio content, speech of a known person from audio content, and producing transcription data therefrom;

(b) recognizing a face of a known person, and producing associated name information;

(c) extracting barcode data from image content;

(d) decoding watermark data from image or audio content; and

(e) computing a fingerprint function from image or audio content.

33. The method of claim 32 in which the known person is a proprietor of said user device.

34. The method of claim 32 in which the known person is a social network acquaintance of a proprietor of said user device.

35. The method of claim 32 in which the degrading comprises discarding.

36. A method comprising:

storing content captured at a location by a portable user device, the content including audio and/or imagery;

after passage of an interval of time, automatically degrading the audio and/or imagery content;

wherein said interval of time has a length that is dependent on one or more of the following factors;

(a) a location at which the content was captured;

(b) a history of later recall of the stored captured content;

(c) a time elapsed between capture of content and said later recall;

(d) a salience of the captured content; and

(e) a social network factor.