Intuitive computing methods and systems

• US 8,886,222 B1
• Filed: 08/05/2014
• Issued: 11/11/2014
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Rhoads, Geoffrey B.
• Primary Examiner(s): MIZRAHI, DIANE D
• Application Number:
  US14/452,282
• Publication Number:
  US 20140351765A1
• Time in Patent Office: 98 Days
• Field of Search: 455/456.1, 455/411
• US Class Current: 455/456.1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone senses audio and/or imagery from a user'"'"'s environment, and acts to fulfill inferred or requested user desires. In one arrangement, a smart phone comprises: a touch screen display; a camera for capturing imagery; a microphone for capturing audio; a physical discovery button for receiving a user'"'"'s indication to initiate a discovery mode; one or more processors; and memory storing software instructions for configuring said one or more processors. The software configures the processors for: responsive to a user'"'"'s request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera; causing two different areas to be displayed on the touch screen display, in which a first of the two areas comprises a sensor pane in which visual information associated with captured imagery is displayed along with augmented reality graphics, and in which a second of the two areas comprises user-selectable graphic icons; and causing a user-selectable graphic icon to be displayed in the first of the two areas to allow a user to switch modes from image to audio. A great number of other features and arrangements are also detailed.

Claims

1. A smart phone comprising:

a touch screen display;

a camera for capturing imagery;

a microphone for capturing audio;

a physical discovery button for receiving a user'"'"'s indication to initiate a discovery mode, in which the discovery mode includes processing smart phone sensor-captured audio or imagery to discern identification or meaning therefrom;

one or more processors; and

memory storing software instructions for configuring said one or more processors for;

responsive to a user'"'"'s request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera;

causing two different areas to be displayed on the touch screen display, in which a first of the two areas comprises a sensor pane in which visual information associated with captured imagery is displayed along with augmented reality graphics, and in which a second of the two areas comprises user-selectable graphic icons; and

causing a user-selectable graphic icon to be displayed in the first of the two areas to allow a user to switch discovery modes from image to audio.

2. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for highlighting features identified within a visual scene of a camera view displayed within the sensor pane.

3. The smart phone of claim 1 in which said physical discovery button comprises hardwiring or programming to receive a user'"'"'s indication to initiate a discovery mode.

4. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for allowing a user to control relative sizes of the first area and the second area.

5. The smart phone of claim 1 in which the processing imagery comprises communicating data associated with captured imagery to a remotely located server.

6. The smart phone of claim 1 in which the processing imagery comprises a filtering operation.

7. The smart phone of claim 1 in which the processing imagery comprises generating a greyscale image associated with captured imagery.

8. The smart phone of claim 1 in which the processing imagery minimizes communication bandwidth requirements associated with captured imagery.

9. The smart phone of claim 1 in which the second area comprises a geolocation pane.

10. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for:

once the user-selectable graphic icons in the second area are user-selected, providing information about a prior audio or visual discovery.

11. A portable device comprising:

a touch screen display;

a camera for capturing imagery;

a microphone for capturing audio;

a physical discovery button for receiving a user'"'"'s indication to initiate a discovery mode, in which the discovery mode includes analyzing sensor-captured audio or imagery to discern identification or meaning therefrom;

one or more processors configured for;

responsive to a user'"'"'s request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera;

causing two different panes to be displayed on the touch screen display, in which a first of the two panes comprises a sensor pane in which visual information associated with captured imagery is displayed along with augmented reality graphics, and in which a second of the two panes comprises user-selectable graphic icons; and

causing a user-selectable graphic icon to be displayed in the first of the two panes to allow a user to switch discovery modes from image to audio.

12. The portable device of claim 11 in which said one or more processors are configured for highlighting features identified within a visual scene of a camera view displayed within the sensor pane.

13. The portable device of claim 11 in which said physical discovery button comprises hardwiring or programming to receive a user'"'"'s indication to initiate a discovery mode.

14. The portable device of claim 11 in which said one or more processors are configured for allowing a user to control relative sizes of the first pane and the second pane.

15. The portable device of claim 11 in which the processing imagery comprises communicating data associated with captured imagery to a remotely located server.

16. The portable device of claim 11 in which the processing imagery comprises a filtering operation.

17. The portable device of claim 11 in which the processing imagery comprises generating a greyscale image associated with captured imagery.

18. The portable device of claim 11 in which the processing imagery minimizes communication bandwidth requirements associated with captured imagery.

19. The portable device of claim 11 in which the second pane comprises a geolocation pane.

20. The portable device of claim 11 in which said one or more processors are configured for:

once the user-selectable graphic icons in the second pane are user-selected, providing information about a prior audio or visual discovery.

21. A system comprising:

i. a smart phone comprising;

a touch screen display;

a camera for capturing imagery;

a microphone for capturing audio;

a physical discovery button for receiving a user'"'"'s indication to initiate a discovery mode, in which the discovery mode includes analyzing smart phone sensor-captured audio or imagery to discern identification or meaning therefrom;

one or more processors configured for;

responsive to a user'"'"'s request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera;

causing two different areas to be displayed on the touch screen display, in which a first of the two areas comprises a sensor pane in which visual information associated with captured imagery is displayed along with augmented reality graphics, and in which a second of the two areas comprises user-selectable graphic icons; and

causing a user-selectable graphic icon to be displayed in the first of the two areas to allow a user to switch discovery modes from image to audio; and

ii. one or more servers configured for;

receiving data associated with imagery processed by said smart phone;

processing the data to extract identifying information therefrom;

using the identifying information to obtain response information; and

communicating the response information to the smart phone.

22. The system of claim 21 in which said one or more processors are configured for highlighting features identified within a visual scene of a camera view displayed within the sensor pane.

23. The system of claim 21 in which said physical discovery button comprises hardwiring or programming to facilitate receipt of a user'"'"'s indication to initiate a discovery mode.

24. The system of claim 21 in which said one or more processors are configured for allowing a user to control relative sizes of the first area and the second area.

25. The system of claim 21 in which the smart phone'"'"'s processing imagery comprises communicating data associated with captured imagery to said one or more servers.

26. The system of claim 21 in which the smart phone'"'"'s processing imagery comprises a filtering operation.

27. The system of claim 21 in which the smart phone'"'"'s processing imagery comprises generating a greyscale image associated with captured imagery.

28. The system of claim 21 in which the smart phone'"'"'s processing imagery minimizes communication bandwidth requirements associated with captured imagery.

29. The system of claim 21 in which the second area comprises a geolocation pane.

30. The system of claim 21 in which said one or more processors are configured for:

once the user-selectable graphic icons in the second area are user-selected, providing response information associated with a prior audio or visual discovery.