Intuitive computing methods and systems

• US 8,422,994 B2
• Filed: 02/21/2012
• Issued: 04/16/2013
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Rodriguez, Tony F.
• Primary Examiner(s): MIZRAHI, DIANE D
• Application Number:
  US13/401,332
• Publication Number:
  US 20120165046A1
• Time in Patent Office: 420 Days
• Field of Search: 455/456.3, 455/456.1, 455/411, 345/8, 345/156, 345/158, 345/633, 707/746, 707/769, 707/812, 715/202, 715/810, 709/217, 348/14.03
• US Class Current: 455/411
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone senses audio, imagery, and/or other stimulus from a user'"'"'s environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone'"'"'s camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user'"'"'s apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Claims

1. A processing device including a processor, memory, a touch screen, a location determination module, and at least one image sensor that produces sensor data about an object in an environment of the device, the memory storing instructions configuring the processor to present on the touch screen a user interface, a first area of the user interface presenting discovery information derived from the image sensor data, the discovery information comprising object-identifying information that identifies the object in said environment, and a second, different area of the user interface simultaneously presenting a representation of a geographic area around the device based on information from the location determination module, the second part of the user interface further displaying markings corresponding to objects previously encountered by a user of the device, and including a time-related control by which said markings can be filtered based on times of such previous encounters.

2. The device of claim 1 wherein said representation of the geographic area around the device comprises a map.

3. The device of claim 1 in which said representation of a geographic area comprises a map depicting at least a street or a business location.

4. The device of claim 3 in which the map includes pins that indicate historical actions of a user.

5. The device of claim 3 in which the map includes pins that indicate locations where different objects were previously identified.

6. The device of claim 1 wherein the first and second areas of the user interface are separated by a straight border.

7. A non-transitory computer readable storage medium containing software instructions that, when executed by a device processor, cause the device to perform acts including:

analyze sensed image data to yield identifying information about an object;

obtain information identifying a location; and

display two parts of a user interface simultaneously, a first part of the user interface presenting the identifying information about the object, and a second part of the user interface presenting a representation of a geographic area around said location, the second part also including markings corresponding to objects previously encountered by a user, and a time-related control by which said markings can be filtered based on times of such previous encounters.

8. The computer readable storage medium of claim 7 in which the second part of the user interface comprises a map.

9. The computer readable storage medium of claim 7 in which a said parts of the user interface are separated by a straight border.

10. A method comprising:

sensing image data from a user'"'"'s environment;

analyzing, using a mobile device, the sensed image data to yield identifying information about an object in the user'"'"'s environment;

sensing a location of the user;

displaying a user interface, a first part of the user interface presenting the identifying information about the object in the user'"'"'s environment, and a second, different part of the user interface presenting a representation of a geographic area around the sensed location of the user;

wherein the method includes displaying said parts of the user interface simultaneously; and

wherein the second part of the user interface displays markings corresponding to objects previously encountered by a user, and includes a time-related control by which said markings can be filtered based on times of such previous encounters.

11. The method of claim 10 that includes, in the second part of the user interface presenting a map.

12. The method of claim 10 wherein said parts of the user interface are separated by a straight border.