Intuitive computing methods and systems

• US 9,444,924 B2
• Filed: 03/09/2015
• Issued: 09/13/2016
• Est. Priority Date: 10/28/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Rhoads, Geoffrey B.
• Primary Examiner(s): MIZRAHI, DIANE D
• Application Number:
  US14/642,296
• Publication Number:
  US 20150244850A1
• Time in Patent Office: 554 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/36 Input/output arrangements for on-board computers
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04N 23/00 : Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04W 4/02 : Services making use of location information
• CPC Class Codes: H04W 4/027 : using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 : Location-based management or tracking services

Abstract

The present disclosure describes portable devices (e.g., smart phones and tablets), related methods and systems. One claim relates to a portable computing device including a device input for initiating a discovery mode. The discovery mode includes processing camera-captured video imagery or microphone-captured audio to obtain identification of an object depicted in camera-captured video imagery or to obtain identification of microphone-captured audio. The portable computing device operates to initiate a discovery mode including processing real-time video imagery captured with a camera, cause a sensor pane to be displayed on a touch screen display, in which real-time captured video imagery is displayed along with augmented reality graphics within the sensor pane, and cause user-selectable graphic icons to be displayed within a first area of the touch screen display, and cause a user-selectable graphic icon to be displayed on touch screen display to facilitate switching discovery modes from image to audio.

Claims

1. A portable tablet computing device comprising:

a wireless communications interface;

a touch screen display;

a camera for capturing video imagery;

a microphone for capturing audio;

a device input for initiating a discovery mode, in which the discovery mode includes processing camera-captured video imagery or microphone-captured audio to obtain identification of an object depicted in camera-captured video imagery or to obtain identification of microphone-captured audio, an identification obtained through the wireless communications interface;

one or more multi-core processors; and

memory storing software instructions for configuring said one or more multi-core processors for;

responsive to a request received via said device input, initiating a discovery mode including processing real-time video imagery captured with said camera;

causing a sensor pane to be displayed on said touch screen display, in which real-time captured video imagery is displayed along with augmented reality graphics within the sensor pane, and causing user-selectable graphic icons to be displayed within a first area of said touch screen display;

causing highlighting of an object represented in captured real-time video imagery that is displayed within the sensor pane; and

causing a user-selectable graphic icon to be displayed on said touch screen display to facilitate switching discovery modes from image to audio.

2. The portable tablet computing device of claim 1 in which the first area overlays the sensor pane, in which said memory stores software instructions for configuring said one or more multi-core processors for touch screen control of a size of the first area displayed on said touch screen display relative to the sensor pane.

3. The portable tablet computing device of claim 1 in which said memory stores software instructions for configuring said one or more multi-core processors for displaying a visual bar via said touch screen display, the visual bar separating the first area from the sensor pane.

4. The portable tablet computing device of claim 1 in which said device input comprises hardwiring to receive a user'"'"'s indication to initiate a discovery mode.

5. The portable tablet computing device of claim 1 in which said memory stores software instructions for configuring said one or more multi-core processors for:

responsive to the request received via said device input, initiating the discovery mode including processing video imagery captured with said camera by communicating data associated with captured video imagery to a remotely located server.

6. The portable tablet computing device of claim 1 in which said memory stores software instructions for configuring said one or more multi-core processors for:

responsive to the request received via said device input, initiating the discovery mode including processing video imagery captured with said camera with an edge filtering operation to identify image features.

7. The portable tablet computing device of claim 1 in which said memory stores software instructions for configuring said one or more multi-core processors for:

responsive to the request received via said device input, initiating the discovery mode including processing video imagery captured with said camera by generating a greyscale image associated with captured video imagery.

8. The portable tablet computing device of claim 1 in which said memory stores software instructions for configuring said one or more multi-core processors for:

responsive to the request received via said device input, initiating the discovery mode including processing imagery by omitting image information.

9. The portable tablet computing device of claim 2 in which the first area comprises a geolocation pane.

10. A portable computing device comprising:

a wireless communications interface;

a touch screen display;

a camera for capturing video imagery;

a microphone for capturing audio;

a device input for initiating a discovery mode, in which the discovery mode includes processing camera-captured video imagery or microphone-captured audio to obtain identification of an object depicted in camera-captured video imagery or to obtain identification of microphone-captured audio, an identification obtained through the wireless communications interface;

means for, responsive to a request received via said device input, initiating a discovery mode including processing real-time video imagery captured with said camera;

means for causing a sensor pane to be displayed on said touch screen display, in which real-time captured video imagery is displayed along with augmented reality graphics within the sensor pane, and causing user-selectable graphic icons to be displayed within a first area of said touch screen display;

means for causing highlighting of an object represented in captured real-time video imagery that is displayed within the sensor pane; and

means for causing a user-selectable graphic icon to be displayed on said touch screen display to facilitate switching discovery modes from image to audio.

11. The portable computing device of claim 10 in which the first area overlays the sensor pane, further comprising means for touch screen control of a size of the first area displayed on said touch screen display relative to the sensor pane.

12. The portable computing device of claim 10 further comprising means for displaying a visual bar via said touch screen display, the visual bar separating the first area from the sensor pane.

13. The portable computing device of claim 10 in which said device input comprises hardwiring to receive a user'"'"'s indication to initiate a discovery mode.

14. The portable computing device of claim 10 further comprising means for, responsive to the request received via said device input, initiating the discovery mode including processing video imagery captured with said camera by communicating data associated with captured imagery to a remotely located server.

15. The portable computing device of claim 10 further comprising means for, responsive to the request received via said device input, initiating the discovery mode including processing video imagery captured with said camera with an edge filtering operation to identify image features.

16. The portable computing device of claim 10 further comprising means for, responsive to the request received via said device input, initiating the discovery mode including processing video imagery captured with said camera by generating a greyscale image associated with captured video imagery.

17. The portable computing device of claim 10 further comprising means for, responsive to the request received via said device input, initiating the discovery mode including processing imagery by omitting image information.

18. The portable computing device of claim 11 in which the first area comprises a geolocation pane.

19. A computer program product being embodied in non-transitory computer readable storage media and comprising portable computing device executable instructions for:

initiating a discovery mode, in response to a request received via a device input, in which the discovery mode includes processing camera-captured imagery to obtain identification of an object depicted in the camera-captured imagery through a wireless communications interface of the portable computing device;

causing captured imagery including video to be displayed in a first area on a portable computing device touch screen display along with augmented reality graphics;

controlling a user interface to display user-selectable graphic icons in a second area on the touch screen display, in which the user-selectable graphic icons include visible indicia to graphically represent content; and

controlling the user interface to display a user-selectable graphic icon on the touch screen display to facilitate switching discovery modes from image to audio,

in which an audio discovery mode processes microphone-captured audio to determine identification therefrom.

20. The computer program product of claim 19 in which said executable instructions comprise instructions to control processing camera-captured imagery by transforming data representing the imagery into greyscale imagery.

21. The computer program product of claim 19 further comprising executable instructions to control highlighting of an object represented in captured imagery that is displayed in the first area on the touch screen display.

22. The computer program product of claim 19 in which the second area overlays the first area, and in which the computer program product further comprises executable instructions to provide touch screen control of a size of the second area displayed on the touch screen display relative to the first area.

23. The computer program product of claim 19 in which in which said executable instructions comprise instructions to provide processing sensor-captured imagery by communicating data associated with captured imagery to a remotely located server.

24. The computer program product of claim 19 in which the second area overlays the first area, comprising executable instructions to provide touch screen control of a size of the second area displayed on the touch screen display relative to the first area.

25. The computer program product of claim 19 in which the second area comprises a geolocation pane.

26. The computer program product of claim 19 in which the device input comprises a user selectable graphic icon.