Intuitive computing methods and systems

• US 9,609,107 B2
• Filed: 07/02/2013
• Issued: 03/28/2017
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Shaw, Gilbert B., Conwell, William Y.
• Primary Examiner(s): Marc, McDieunel
• Application Number:
  US13/933,874
• Publication Number:
  US 20130294648A1
• Time in Patent Office: 1,365 Days
• Field of Search: 701/517, 455/411, 455/456.1
• US Class Current: 1/1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone senses audio, imagery, and/or other stimulus from a user'"'"'s environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone'"'"'s camera. In one detailed arrangement, image processing tasks applied to the scene are selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone applies more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user'"'"'s apparent interest in the task. In another detailed arrangement, data is referred to the cloud for analysis, or for gleaning. In still another detailed arrangement, cognition, and identification of appropriate device response(s), is aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Claims

1. A method comprising the acts:

identifying at least first and second different, alternate operations, either of which operations can yield output data of a desired functional type when applied to first input data, the first input data comprising data derived from camera-captured imagery;

determining a first score for the first operation, said first score depending on at least two parameters;

determining a second score for the second operation, the second score depending on at least two parameters; and

identifying, from said determined scores, one of said alternate operations to be used;

wherein said determining acts are performed by a hardware processor configured to perform such acts.

2. The method of claim 1 that includes a dispatch process invoking the identified one of said alternate operations.

3. The method of claim 1 in which the output data of a desired functional type comprises edge detection data.

4. The method of claim 3 in which the first operation comprises a Canny filtering operation, and the second operation comprises an Infinite Symmetric Exponential filtering operation.

5. The method of claim 1 in which said determining acts are performed by a hardware processor in a camera-equipped cell phone, said imagery having been captured by said camera-equipped cell phone.

6. The method of claim 1 that includes:

performing the method of claim 1 a first time and invoking the first operation as a consequence;

performing the method of claim 1 a second time, and invoking the second operation as a consequence.

7. The method of claim 1 in which at least one of the parameters on which the first score depends comprises a parameter based on data retrieved from a data structure, the data structure storing information relating to the first operation.

8. The method of claim 7 in which the data retrieved from the data structure comprises data indicating a degree of functional correspondence between the alternate operations.

9. The method of claim 1 in which at least one of the parameters on which the first score depends comprises a computing context parameter.

10. The method of claim 9 in which the computing context parameter comprises a power constraint.

11. The method of claim 1 in which the first score depends on a particular parameter, and the second score also depends on said same particular parameter.

12. The method of claim 1 that includes determining the first score by a process that includes weighting and summing the two parameters on which the first score depends, in accordance with a polynomial equation.

13. The method of claim 1 that further includes capturing said imagery with the camera, and deriving data therefrom.

14. The method of claim 1 in which said identifying act includes:

determining that output data of the desired functional type, produced by the first operation, is available in a data structure; and

deciding, based on said scores, to use said output data available in the data structure rather than invoke the second operation.

15. The method of claim 14 wherein the output data available in the data structure was produced by the first operation applied to second input data different than the first input data, and the method further includes determining that the second input data is comparable in content to the first input data.

16. The method of claim 15 wherein the first and second input data comprise non-consecutive first and second frames of imagery captured in a video sequence, and said determining comprises determining that a region of interest that appears in the first frame also appears in the second frame, and also appears in each intervening frame in said sequence between said first and second frames.

17. The method of claim 1 in which said identifying act includes:

determining that output data of the desired functional type, produced by the first operation, is available in a data structure; and

deciding, based on said scores, to invoke the second operation rather than use said output data available in the data structure.

18. A non-transitory computer readable medium having software instructions thereon that cause a camera-equipped cell phone configured thereby to perform actions including:

identify at least first and second different, alternate operations, either of which operations can yield output data of a desired functional type when applied to first input data, the first input data comprising data derived from camera-captured imagery;

determine a first score for the first operation, said first score depending on at least two parameters;

determine a second score for the second operation, the second score depending on at least two parameters; and

identify, from said determined scores, one of said alternate operations to be performed.

19. A cell phone comprising:

a camera, a microphone, a touchscreen, a wireless interface, a processor, and memory, the memory containing software instructions that configure the cell phone to perform actions including;

identify at least first and second different, alternate operations, either of which operations can yield output data of a desired functional type when applied to first input data, the first input data comprising data derived from camera-captured imagery;

determine a first score for the first operation, said first score depending on at least two parameters;

determine a second score for the second operation, the second score depending on at least two parameters; and

identify, from said determined scores, one of said alternate operations to be performed.

20. The cell phone of claim 19 in which said software instructions further configure the cell phone to:

determine that output data of the desired functional type, produced by the first operation, is available in a data structure; and

decide, based on said scores, to use said output data available in the data structure rather than invoke the second operation.