Methods of and systems for content search based on environment sampling

• US 9,736,524 B2
• Filed: 01/06/2012
• Issued: 08/15/2017
• Est. Priority Date: 01/06/2011
• Status: Active Grant

• Current Assignee: Veveo, Inc. (Adeia Inc.)
• Original Assignee: Veveo, Inc. (Adeia Inc.)
• Inventors: Rajasekharan, Ajit, Aravamudan, Murali
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US13/345,119
• Publication Number:
  US 20120176509A1
• Time in Patent Office: 2,048 Days
• Field of Search: None
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/25 Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies maintenance or administration in data networks H04L41/00
• CPC Class Codes: H04N 21/2542 for selling goods, e.g. TV shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/258 Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data arrangements for services using the result on the distributing side of broadcast systems H04H60/66; profiles in network data switching protocols H04L67/30
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4524 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; systems specially adapted for using geographical information in broadcast systems H04H60/70; protocols in which the network application is adapted for the location of the user terminal in communication control or processing H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12

Abstract

The present disclosure provides user interface methods of and systems for displaying at least one available action overlaid on an image, comprising displaying an image; selecting at least one action and assigning a ranking weight thereto based on at least one of (1) image content, (2) current device location, (3) location at which the image was taken, (4) date of capturing the image; (5) time of capturing the image; and (6) a user preference signature representing prior actions chosen by a user and content preferences learned about the user; and ranking the at least one action based on its assigned ranking weight.

Claims

1. A computer-implemented user interface method of displaying at least one available action overlaid on an image, the method comprising:

generating for display, a live image and a visual guide overlaid on the live image;

identifying an object of interest in the live image based on a proximity of the object of interest to the visual guide;

identifying, by a processor, without receiving user input, a first plurality of actions of different types from a second plurality of actions for subsequent selection by a user, the first plurality of actions being identified automatically based at least in part on the object of interest and at least one of (1) current device location, (2) location at which the live image was taken, (3) date of capturing the live image, (4) time of capturing the live image, and (5) a user preference signature representing prior actions selected by a user and content preferences learned about the user associated with particular times or locations at which the prior actions were selected by the user;

assigning a ranking weight to the first plurality of actions based on a non-textual portion of the identified object of interest;

ranking the first plurality of actions based on its assigned ranking weight; and

presenting the first plurality of actions to a user as selectable options.

2. The method of claim 1, wherein the presenting of the first plurality of actions to the user as selectable options includes displaying the first plurality of actions in an order based on the ranking.

3. The method of claim 2, further comprising updating the user preference signature to include information about the action chosen by the user from among the first plurality of actions.

4. The method of claim 1, wherein the live image is an image of a portion of an environment surrounding the user.

5. The method of claim 1, wherein the identifying a first plurality of actions and assigning a ranking weight thereto includes determining the ranking weight by a machine learning process.

6. The method of claim 1, further comprising selecting the highest ranked action within the first plurality of actions in response to activation of a hardware camera button.

7. The method of claim 1, further comprising analyzing the live image to learn about the object of interest.

8. The method of claim 7, further comprising using at least one of the location of the device and the location at which the live image was taken to augment the analyzing the live image to learn about the object of interest.

9. The method of claim 8, wherein the first plurality of actions includes an action to purchase an item corresponding to the object of interest from an online storefront corresponding to a physical storefront, if the device'"'"'s location is proximate to the physical storefront.

10. The method of claim 7, wherein the analyzing the live image to learn about the object of interest includes comparing the live image against a collection of at least one sample image to determine the object of interest.

11. The method of claim 10, wherein the analyzing the live image to learn about the object of interest includes using optical character recognition to learn about textual image content.

12. The method of claim 10, wherein the analyzing the live image to learn about the object of interest includes analyzing at least one partial image selected based on a proximity of the at least one partial image to a visual field of interest for the user.

13. The method of claim 1, wherein the first plurality of actions are presented at a first point in time, further comprising:

storing the live image to a memory along with data about at least one of the location of the device, the date at which the live image was captured, and the time at which the live image was captured; and

presenting the first plurality of actions at a second point in time in an order based on the ranking when the user later acts upon the stored live image after the first point in time.

14. The method of claim 1, the method further comprising:

subsequent to identifying the first plurality of actions, identifying a second object in the live image, wherein the second object is farther from the visual guide than the object of interest;

identifying an alternate action associated with the second object; and

presenting the alternate action to a user as a selectable option.

15. A system for displaying at least one available action overlaid on an image, the system comprising:

a memory device that stores instructions; and

a processor circuitry that executes the instructions and is configured to;

generate, for display, a live image and a visual guide overlaid on the live image;

identify an object of interest in the live image based on the proximity of the object of interest to the visual guide;

identify, without receiving user input, a first plurality of actions of different types from a second plurality of actions for subsequent selection by the user, the first plurality of actions being identified automatically based at least in part on the object of interest and at least one of (1) current device location, (2) location at which the live image was taken, (3) date of capturing the live image, (4) time of capturing the live image, and (5) a user preference signature representing prior actions selected by a user and content preferences learned about the user associated with particular times or locations at which prior actions were selected by the user;

assign a ranking weight to the first plurality of actions based on a non-textual portion of the identified object of interest;

rank the first plurality of actions based on its assigned ranking weight; and

present the first plurality of actions to a user as selectable options.

16. The system of claim 15, wherein the processor circuitry is further configured to present the first plurality of actions to a user as selectable options by displaying the first plurality of actions in an order based on the ranking.

17. The system of claim 16, the processor circuitry further being configured to cause the computer system to update the user preference signature to include information about the action chosen by the user from among the first plurality of actions.

18. The system of claim 15, wherein the live image is an image of a portion of an environment surrounding the user.

19. The system of claim 15, wherein the processor circuitry is further configured to determine the ranking weight by a machine learning process.

20. The system of claim 15, the processor circuitry being further configured to cause the computer system to select the highest ranked action within the first plurality of actions in response to activation of a hardware camera button.

21. The system of claim 15, the processor circuitry being further configured to cause the computer system to analyze the live image to learn about the object of interest.

22. The system of claim 21, the processor circuitry being further configured to cause the computer system to use at least one of the location of the device and the location at which the live image was taken to augment the analyzing of the live image to learn about the object of interest.

23. The system of claim 21, wherein the processor circuitry is further configured to compare the live image against a collection of at least one sample image to determine the object of interest.

24. The system of claim 23, wherein the processor circuitry is further configured to use optical character recognition to learn about textual image content.

25. The system of claim 23, wherein the processor circuitry is further configured to analyze at least one partial image selected based on a proximity of the at least one partial image to a visual field of interest for the user.

26. The system of claim 15, wherein the processor circuitry is further configured to present the first plurality of actions at a first point in time, and the processor circuitry is further configured to:

cause the computer system to store the live image to a memory along with data about at least one of the location of the device, the date at which the live image was captured, and the time at which the live image was captured; and

cause the computer system to present the first plurality of actions again at a second point in time in-an order based on the ranking when the user later acts upon the stored live image after the first point in time.