SMARTPHONE-BASED METHODS AND SYSTEMS

• US 20150163345A1
• Filed: 08/05/2014
• Published: 06/11/2015
• Est. Priority Date: 12/06/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Cornaby, Colin P., Weaver, Matthew M.
• Granted Patent: US 9,354,778 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

Arrangements involving portable devices (e.g., smartphones and tablet computers) are disclosed. One arrangement includes a smartphone comprising: a touchscreen display; a camera for capturing image data; a microphone for capturing audio data; memory for storing software instructions; and one or more processors programmed with said software instructions. The software instruction cause the processors to: process image data captured by said camera; communicate data associated with the processed image data to a remotely located server; in response to the communication, receive identifying information associated with an object depicted in the image data, the identifying information comprising an image or graphic; cause the touchscreen display to simultaneously display: i) stacked tiles comprising a first tile including an image or graphic, the first tile associated with a first image-recognized object, and a second tile including an image or graphic, the second tile associated with a second image-recognized object; and ii) a viewfinder for displaying live image data captured by the camera while the smartphone remains in an image discovery mode. A great variety of other features and arrangements are also detailed.

Claims

1. A method utilizing a portable device comprising a camera and a touchscreen display, said method comprising the acts of:

receiving image data captured by the camera of the portable device;

communicating data associated with the image data to a remotely located server;

in response to said communicating, receiving identifying information associated with an object depicted in the image data, the identifying information comprising an image or graphic;

causing the touchscreen display to simultaneously display;

i) a first tile comprising an image or graphic, the first tile associated with a first image-recognized object, and a second tile comprising an image or graphic, the second tile associated with a second image-recognized object, the first tile and second tile displayed in a stacked arrangement; and

ii) live image data captured by the camera while the portable device remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data.

2. The method of claim 1 in which the portable device comprises a movement sensor, and said method comprises making an inference from data obtained from the movement sensor of a preference for operating the portable device in an image discovery mode rather than an audio discovery mode, in which the audio discovery mode operates to identify audio captured with a microphone.

3. The method of claim 2 in which the movement sensor comprises an accelerometer or gyroscope.

4. The method of claim 1 in which the first tile, second tile and live image data are displayed on the same plane of the touchscreen display.

5. The method of claim 1 in which, in response to input received through the touchscreen display, displaying a third tile in the stacked arrangement.

6. The method of claim 1 in which the live image data is displayed in a viewfinder window.

7. The method of claim 6 in which upon the viewfinder window being occluded, said method further comprises deactivating the camera.

8. The method of claim 6 further comprising causing the portable device to vibrate to confirm recognition of an object depicted in the viewfinder window.

9. The method of claim 1 further comprising, causing the stacked arrangement to grow in tiles until a predetermined height is reached, and then tiles resulting from further content discovery cause a bottom tile in the stacked arrangement to sink out of sight on the touchscreen display.

10. A smartphone comprising:

a touchscreen display;

a camera for capturing image data;

a microphone for capturing audio data;

memory for storing software instructions;

one or more processors programmed with said software instructions for;

processing image data captured by said camera;

communicating data associated with the processed image data to a remotely located server;

in response to said communicating, receiving identifying information associated with an object depicted in the image data, the identifying information comprising an image or graphic;

causing the touchscreen display to simultaneously display;

i) stacked tiles comprising a first tile including an image or graphic, the first tile associated with a first image-recognized object, and a second tile including an image or graphic, the second tile associated with a second image-recognized object; and

ii) a viewfinder area for displaying live image data captured by said camera while the smartphone remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data.

11. The smartphone of claim 10 further comprising a movement sensor, and in which said one or more processors are programmed for making an inference from data obtained from the movement sensor of a preference for operating said smartphone in an image discovery mode rather than an audio discovery mode, in which the audio discovery mode operates to identify audio captured with said microphone.

12. The smartphone of claim 11 in which said movement sensor comprises an accelerometer or gyroscope.

13. The smartphone of claim 10 in which the first tile, second tile and viewfinder area are displayed on the same plane of said touchscreen display.

14. The smartphone of claim 10 in which said one or more processors are programmed for, in response to input received through said touchscreen display, displaying a third tile in the stacked tiles.

15. The smartphone of claim 10 in which upon the viewfinder area being occluded, said one or more processors are programmed for deactivating said camera.

16. The smartphone of claim 10 in which said one or more processors are programmed for controlling a vibration to confirm recognition of an object depicted in the viewfinder area.

17. The smartphone of claim 10 in which said one or more processors are programmed for causing the stacked tiles to grow in tiles until a predetermined height is reached, and then tiles resulting from further content discovery cause a bottom tile in the stack to sink out of sight on the touchscreen display.

18. A system comprising:

A. a smartphone comprising;

a touchscreen display;

a camera for capturing image data;

a microphone for capturing audio data;

one or more processors configured for;

communicating data associated with captured image data to a remotely located server;

in response to said communicating, receiving payoff information associated with an object depicted in the image data, the payoff information comprising an image or graphic;

causing the touchscreen display to simultaneously display;

i) stacked tiles comprising a first tile including an image or graphic, the first tile associated with a first image-recognized object, and a second tile including an image or graphic, the second tile associated with a second image-recognized object; and

ii) a viewfinder window where live image data captured by said camera is displayed while said smartphone remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data; and

B. a server comprising one or more processors configured for;

processing data associated with remotely captured imagery to extract identifying information therefrom;

indexing the identifying information in a data repository to find payoff information associated with an object depicted in the captured imagery; and

control communication of the payoff information to said smartphone.

19. The system of claim 18 in which said smartphone further comprises a movement sensor, and in which said smartphone'"'"'s one or more processors are configured for making an inference from data obtained from said movement sensor of a preference for operating said smartphone in an image discovery mode rather than an audio discovery mode, in which the audio discovery mode operates to identify audio captured with said microphone.

20. The system of claim 19 in which said movement sensor comprises an accelerometer or gyroscope.

21. The system of claim 18 in which the first tile, second tile and viewfinder window are displayed on the same plane of said touchscreen display.

22. The system of claim 18 in which said smartphone'"'"'s one or more processors are configured for, in response to input received through said touchscreen display, displaying a third tile in the stacked tiles.

23. The system of claim 18 in which upon the viewfinder window being occluded, said smartphone'"'"'s one or more processors are configured for deactivating said camera.

24. The system of claim 18 in which said smartphone'"'"'s one or more processors are configured for controlling a vibration to confirm recognition of an object depicted in the viewfinder window.

25. The system of claim 18 in which said smartphone'"'"'s one or more processors are programmed for causing the stacked tiles to grow in tiles until a predetermined height is reached, and then tiles resulting from further content discovery cause a bottom tile in the stack to sink out of sight on said touchscreen display.

26. A smartphone comprising:

means for displaying imagery and graphics;

means for capturing image data;

means for capturing audio data;

means for processing image data captured by said means for capturing image data;

means for communicating data associated with processed image data to a remotely located server;

means for, in response to communicating data associated with processed image data, receiving identifying information associated with an object depicted in the image data, the identifying information comprising an image or graphic;

means for simultaneously displaying on said means for displaying imagery and graphics;

i) stacked tiles comprising a first tile including an image or graphic, the first tile associated with a first image-recognized object, and a second tile including an image or graphic, the second tile associated with a second image-recognized object; and

ii) a viewfinder window where live image data captured by said means for capturing imagery is displayed while said smartphone remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data.

27. The smartphone of claim 26 in which said smartphone comprises means for displaying a third tile in the stacked tiles, in response to input received through said means for simultaneously displaying.

28. The smartphone of claim 26 further comprising means for deactivating said means for capturing image data, which operates upon the viewfinder window being occluded.

29. The smartphone of claim 26 further comprising means for controlling a vibration to confirm recognition of an object depicted in the viewfinder window.

30. The smartphone of claim 26 in which the first tile, second tile and viewfinder window are displayed on the same plane of said means displaying imagery and graphics.