Intuitive computing methods and systems

• US 8,977,293 B2
• Filed: 10/07/2014
• Issued: 03/10/2015
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Rhoads, Geoffrey B.
• Primary Examiner(s): MIZRAHI, DIANE D
• Application Number:
  US14/508,681
• Publication Number:
  US 20150024800A1
• Time in Patent Office: 154 Days
• Field of Search: 455/456.1
• US Class Current: 455/456.1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

A smart phone senses audio and/or imagery from a surrounding environment, and acts to provide graphical user interfaces and toggling discovery modes. In one arrangement, a smart phone comprises: a wireless communications interface; a touch screen display; a camera for capturing imagery; a microphone for capturing audio; a physical discovery button for initiating a discovery mode, in which the discovery mode includes processing smart phone sensor-captured imagery or sensor-captured audio to obtain identification of an object depicted in sensor-captured imagery or of captured audio, the identification obtained through the wireless communications interface; one or more processors; and memory storing software instructions for configuring said one or more processors for: responsive to a request received via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera; causing a sensor pane to be displayed on the touch screen display, in which captured imagery is displayed along with augmented reality graphics within the sensor pane, and causing user-selectable graphic icons to be displayed within a first area of the touch screen display; highlighting an object represented in captured imagery that is displayed within the sensor pane; and causing a user-selectable graphic icon to be displayed on the touch screen display to facilitate switching discovery modes from image to audio. A great number of other features and arrangements are also detailed.

Claims

1. A smart phone comprising:

a wireless communications interface;

a touch screen display;

a camera for capturing imagery;

a microphone for capturing audio;

a physical discovery button for initiating a discovery mode, in which the discovery mode includes processing smart phone camera-captured imagery or microphone-captured audio to obtain identification of an object depicted in camera-captured imagery or to obtain identification of microphone-captured audio, an identification obtained through said wireless communications interface;

one or more processors; and

memory storing software instructions for configuring said one or more processors for;

responsive to a request received via said physical discovery button, initiating a discovery mode including processing imagery captured with said camera;

causing a sensor pane to be displayed on said touch screen display, in which captured imagery is displayed along with augmented reality graphics within the sensor pane, and causing user-selectable graphic icons to be displayed within a first area of said touch screen display;

causing highlighting of an object represented in captured imagery that is displayed within the sensor pane; and

causing a user-selectable graphic icon to be displayed on said touch screen display to facilitate switching discovery modes from image to audio.

2. The smart phone of claim 1 in which the first area overlays the sensor pane, in which said memory stores software instructions for configuring said one or more processors for touch screen control of a size of the first area displayed on said touch screen display relative to the sensor pane.

3. The smart phone of claim 2 in which the first area comprises a geolocation pane.

4. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for displaying a visual bar on said touch screen display, the visual bar separating the first area from the sensor pane.

5. The smart phone of claim 1 in which said physical discovery button comprises hardwiring or programming to receive a user'"'"'s indication to initiate a discovery mode.

6. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for:

responsive to the request received via said physical discovery button, initiating the discovery mode including processing imagery captured with said camera by communicating data associated with captured imagery to a remotely located server.

7. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for:

responsive to the request received via said physical discovery button, initiating the discovery mode including processing imagery captured with said camera with an edge filtering operation to identify image features.

8. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for:

responsive to the request received via said physical discovery button, initiating the discovery mode including processing imagery captured with said camera by generating a greyscale image associated with captured imagery.

9. The smart phone of claim 1 in which said memory stores software instructions for configuring said one or more processors for:

responsive to the request received via said physical discovery button, initiating the discovery mode including processing imagery by omitting image information.

10. A portable device comprising:

a wireless communications interface;

a touch screen display;

a camera for capturing imagery;

a microphone for capturing audio;

a physical discovery button for initiating a discovery mode, in which the discovery mode includes processing smart phone sensor-captured imagery or sensor-captured audio to obtain identification of an object depicted in sensor-captured imagery or to obtain identification of captured audio, an identification obtained through said wireless communications interface;

one or more processors configured for;

responsive to a request received via said physical discovery button, initiating a discovery mode including processing imagery captured with said camera,

the processing imagery comprises transforming data representing the imagery captured with said camera into greyscale imagery;

causing captured imagery including video to be displayed in a first area on said touch screen display along with augmented reality graphics;

causing user-selectable graphic icons to be displayed in a second area on said touch screen display, in which the user-selectable graphic icons include visible indicia to graphically represent content; and

causing a user-selectable graphic icon to be displayed on said touch screen display to facilitate switching discovery modes from image to audio.

11. The portable device of claim 10 in which said one or more processors are configured for causing highlighting of an object represented in captured imagery that is displayed in the first area on said touch screen display.

12. The portable device of claim 10 in which said physical discovery button comprises hardwiring or programming to initiate a discovery mode.

13. The portable device of claim 10 in which the second area overlays the first area, and in which said one or more processors are configured for touch screen control of a size of the second area displayed on said touch screen display relative to the first area.

14. The portable device of claim 10 in which said one or more processors are configured for displaying a visual bar on said touch screen display, the visual bar separating the first area from the second area.

15. The portable device of claim 10 in which said one or more processors are configured for processing imagery captured with said camera by communicating data associated with captured imagery to a remotely located server.

16. The portable device of claim 10 in which said one or more processors are configured for processing imagery captured with said camera by an edge filtering operation to identify image features.

17. The portable device of claim 10 in which said one or more processors are configured for processing imagery captured with said camera by omitting image information.

18. The portable device of claim 10 in which the second area comprises a geolocation pane.

19. A smart phone comprising:

a camera;

a microphone;

a wireless communications interface;

a physical discovery button for receiving an indication to initiate a discovery mode, in which the discovery mode includes analyzing sensor-captured audio or imagery to obtain identification of an object depicted in sensor-captured imagery or to obtain identification of captured audio, an identification obtained through said wireless communications interface;

processing means for, responsive to a request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera;

smart phone user interface means for displaying two different areas, in which a first of the two areas comprises an area in which visual information associated with captured imagery is displayed along with augmented reality graphics, and in which a second of the two areas comprises user-selectable graphic icons; and

smart phone mode switching means for displaying a user-selectable graphic icon to allow a user to switch discovery modes from image to audio.

20. The smart phone of claim 19 comprising means for causing highlighting of an object represented in captured imagery that is within a visual scene of a camera view displayed within the first of the two areas.

21. The smart phone of claim 19 comprising means for controlling relative sizes of the first of the two areas and the second of the two areas.

22. The smart phone of claim 19 in which said processing means for, responsive to a user'"'"'s request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera includes communicating data associated with captured imagery to a remotely located server.

23. The smart phone of claim 19 in which said processing means for, responsive to a user'"'"'s request via the physical discovery button, initiating a discovery mode including processing imagery captured with said camera includes omitting image information.

24. The smart phone of claim 19 in which the second of the two areas comprises a geolocation pane.

25. A computer program product being embodied in non-transitory computer readable storage media and comprising smartphone executable instructions for:

initiating a discovery mode, in response to a request received via a smartphone input, in which the discovery mode includes processing smartphone camera-captured imagery to obtain identification of an object depicted in the camera-captured imagery through a wireless communications interface of the smartphone;

causing captured imagery including video to be displayed in a first area on a smartphone touch screen display along with augmented reality graphics;

controlling a user interface to display user-selectable graphic icons in a second area on the touch screen display, in which the user-selectable graphic icons include visible indicia to graphically represent content; and

controlling the user interface to display a user-selectable graphic icon on the touch screen display to facilitate switching discovery modes from image to audio,

in which an audio discovery mode processes microphone-captured audio to determine identification therefrom.

26. The computer program product of claim 25 in which said smartphone executable instructions for initiating a discovery mode, in response to a request received via a smartphone input, comprise instructions for processing smartphone camera-captured imagery by transforming data representing the imagery into greyscale imagery.

27. The computer program product of claim 25 further comprising smartphone executable instructions for causing highlighting of an object represented in captured imagery that is displayed in the first area on the touch screen display.

28. The computer program product of claim 25 in which the second area overlays the first area, and in which the computer program product further comprises smartphone executable instructions for providing touch screen control of a size of the second area displayed on the touch screen display relative to the first area.

29. The computer program product of claim 25 in which in which said smartphone executable instructions for initiating a discovery mode, in response to a request received via a smartphone input, comprise instructions for processing smartphone sensor-captured imagery by communicating data associated with captured imagery to a remotely located server.

30. The computer program product of claim 25 in which the second area comprises a geolocation pane.