Intuitive Computing Methods and Systems

• US 20120165046A1
• Filed: 02/21/2012
• Published: 06/28/2012
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Geoffrey B. Rhoads, Tony F. Rodriguez
• Original Assignee: Geoffrey B. Rhoads, Tony F. Rodriguez
• Inventors: Rhoads, Geoffrey B., Rodriguez, Tony F.
• Granted Patent: US 8,422,994 B2
• Time in Patent Office: Days
• US Class Current: 455/456.3
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone senses audio, imagery, and/or other stimulus from a user'"'"'s environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone'"'"'s camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user'"'"'s apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Claims

1-28. -28. (canceled)

29. A processing device including a processor, memory, a touch screen, a location determination module, and at least one audio or image sensor, the memory storing instructions configuring the processor to present on the touch screen a user interface, a first area of the user interface presenting information derived from the sensor, and a second, different area of the user interface simultaneously presenting information related to a location of the device.

30. The device of claim 29 wherein the information related to a location of the device comprises a map depicting a nearby vicinity, and wherein the instructions configure the processor to present pins indicating historical actions of a user on said map.

36. The processing device of claim 29 wherein the first area of the user interface presents information derived from an audio sensor.

37. The processing device of claim 36 in which the first area of the user interface presents audio scene visualization information, including information indicating a direction of an audio stream.

38. The processing device of claim 29 wherein the first area of the user interface presents information derived from an image sensor.

31. A processing device including a processor, memory, a screen, and an image sensor, the memory storing instructions configuring the processor to present on the touch screen data corresponding to imagery sensed by the image sensor, the processor further presenting a sweeping radar trace effect on the touch screen to indicate device activity in processing image data.

32. The device of claim 31 wherein the instructions configure the processor to present, trailing the sweeping radar trace, indicia about orientation of an object imaged by the image sensor.

33. The device of claim 32 wherein the indicia indicates orientation of a digital watermark in sensed image data.

34. A method of sound source localization, comprising the acts:

sampling ambient audio using plural wireless phones within an environment;

sending audio information sensed by a first phone to a second phone;

discerning location data relating the position of the first phone to the second location; and

in the second phone, processing the location data, audio information received from the first phone, and audio sampled by the second phone, to discern a sound source direction relative to the second phone.

35. The method of claim 34 in which the sending comprises sending information that has not been made temporally ambiguous by windowed frequency domain-based compression.

39. The method of claim 34 in which the discerning includes discerning 6D pose information for at least one phone.