Intuitive computing methods and systems

• US 9,888,105 B2
• Filed: 03/23/2017
• Issued: 02/06/2018
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B.
• Primary Examiner(s): Marc, McDieunel
• Application Number:
  US15/467,819
• Publication Number:
  US 20170257474A1
• Time in Patent Office: 320 Days
• Field of Search: 701517, 455411, 4554561
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone senses audio, imagery, and/or other stimulus from a user'"'"'s environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone'"'"'s camera. Recognition tasks, selected with the aid of context, are allocated increased or decreased resources based on data comprising (a) user input data indicating express or implied encouragement or discouragement of the recognition task and/or (b) a detection state metric, representing a quantified likelihood that the recognition goal sought by the recognition task will be reached. A great number of other features and arrangements are also detailed.

Claims

1. An image processing method comprising the acts:

capturing imagery with a camera portion of a mobile system, the system also including a processor controlled by configuration instructions stored in a memory;

selecting a first recognition process to apply to the captured imagery, from among several recognition processes that the system is equipped to apply, said selection of the first recognition process being based on context information rather than on express user instruction;

launching the selected first recognition process, employing said processor, to discern information about a subject depicted in the captured imagery;

receiving first data, after said selected first recognition process has been launched, said first data comprising (a) user input data indicating express or implied user encouragement or discouragement of said first recognition process by the user, and/or (b) a detection state metric, representing a quantified likelihood that a recognition goal sought by the first recognition process will be reached; and

allocating increased or decreased resources to said first recognition process based on said received first data, said allocation of increased resources being apart from increased resource usage due to any increasing base complexity of successive stages of the selected first recognition process.

2. The method of claim 1 in which the received first data comprises user input data indicating user encouragement of said process, and the method includes allocating increased resources to said first recognition process based on said indication of encouragement, wherein the mobile system seems to respond intuitively to a user interest in said subject.

3. The method of claim 1 in which the received first data indicates comprises user input data indicating user discouragement of said process, and the method includes allocating reduced resources to said first recognition process based on said indication of discouragement, wherein the mobile system seems to respond intuitively to a user dis-interest in said subject.

4. The method of claim 1 in which the received first data comprises a detection state metric representing a quantified likelihood that the recognition goal sought by the first recognition process will be reached, and the method includes allocating increased resources to the first recognition process as a consequence of said detection state metric.

5. The method of claim 1 in which the received first data comprises a detection state metric representing a quantified likelihood that the recognition goal sought by the first recognition process will be reached, and the method includes allocating decreased resources to the first recognition process as a consequence of said detection state metric.

6. The method of claim 1 that includes:

receiving user data indicating express or implied user encouragement or discouragement of the first recognition process; and

receiving a detection state metric representing a quantified likelihood that the recognition goal sought by the first recognition process will be reached;

wherein the method includes allocating resources to said first recognition process based on both (1) said received user data; and

(2) said received detection state metric.

7. The method of claim 1 in which the context information on which selection of the first recognition process is based, comprises image information.

8. The method of claim 7 that includes selecting said first recognition process, from the several recognition processes the system is equipped to apply, based on results of baseline image processing operations applied to the captured imagery.

9. The method of claim 7 that includes selecting said first recognition process, from the several recognition processes the system is equipped to apply, also based on context data distinct from image data.

10. The method of claim 7 in which the context information on which selection of the first recognition process is based, also comprises location information.

11. The method of claim 1 in which the context information on which selection of the first recognition process is based, comprises location information.

12. The method of claim 1 that includes selecting said first recognition process, from the several recognition processes the system is equipped to apply, based on context data distinct from image data.

13. The method of claim 1 that includes:

selecting both first and second recognition processes to apply to the captured imagery, from among said several recognition processes the system is equipped to apply, based on context information rather than on express user instruction;

launching both the first and second recognition processes;

receiving second data, after said second recognition process has been launched, said second data comprising (a) user input data indicating express or implied user encouragement or discouragement of said second recognition process by the user, and/or (b) a detection state metric, representing a quantified likelihood that a recognition goal sought by the second recognition process will be reached; and

based on the received first and second data, allocating increased resources to the first recognition process, and allocating reduced resources to the second recognition process.

14. The method of claim 1 in which allocating increased resources to said first recognition process comprises making GPU resources available instead of CPU resources, allocating more scratchpad memory for calculations, allowing access to faster-responding memory, allowing increased power consumption, or allowing access to a faster network connection.

15. A non-transitory computer readable storage medium containing software instructions to configure a mobile system, equipped with a processor and a camera, to perform acts including:

selecting a first recognition process to apply to imagery captured by the camera, from among several recognition processes that the system is equipped to apply, said selection of the first recognition process being based on context information rather than on express user instruction;

launching the selected first recognition process, to discern information about a subject depicted in the captured imagery;

receiving first data, after said selected first recognition process has been launched, said first data comprising (a) user input data indicating express or implied user encouragement or discouragement of said first recognition process by the user, and/or (b) a detection state metric, representing a quantified likelihood that a recognition goal sought by the first recognition process will be reached; and

allocating increased or decreased resources to said first recognition process based on said received first data, said allocation of increased resources being apart from increased resource usage due to any increasing base complexity of successive stages of the selected first recognition process.

16. The computer readable storage medium of claim 15 in which said instructions configure the system to allocate increased resources to said first recognition process based on user input data indicating express or implied user encouragement of said first recognition process by the user.

17. The computer readable storage medium of claim 15 in which said instructions configure the system to allocate decreased resources to said first recognition process based on user input data indicating express or implied user discouragement of said first recognition process by the user.

18. The computer readable storage medium of claim 15 in which said instructions configure the system to allocate increased resources to said first recognition process based on said detection state metric.

19. The computer readable storage medium of claim 15 in which said instructions configure the system to allocate decreased resources to said first recognition process based on said detection state metric.

20. A mobile system comprising a processor, memory and camera, the memory further comprising software instructions to configure the mobile system to perform acts including:

selecting a first recognition process to apply to imagery captured by the camera, from among several recognition processes that the system is equipped to apply, said selection of the first recognition process being based on context information rather than on express user instruction;

launching the selected first recognition process, to discern information about a subject depicted in the captured imagery;

receiving first data, after said selected first recognition process has been launched, said first data comprising (a) user input data indicating express or implied user encouragement or discouragement of said first recognition process by the user, and/or (b) a detection state metric, representing a quantified likelihood that a recognition goal sought by the first recognition process will be reached; and

allocating increased or decreased resources to said first recognition process based on said received first data, said allocation of increased resources being apart from increased resource usage due to any increasing base complexity of successive stages of the selected first recognition process.