Methods for providing information services related to visual imagery

• US 7,873,911 B2
• Filed: 08/01/2006
• Issued: 01/18/2011
• Est. Priority Date: 08/31/2004
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Kumar C. Gopalakrishnan
• Inventors: Gopalakrishnan, Kumar C.
• Primary Examiner(s): Bashore; William L
• Assistant Examiner(s): ZAHR, ASHRAF A
• Application Number:
  US11/461,713
• Publication Number:
  US 20060268007A1
• Time in Patent Office: 1,631 Days
• Field of Search: 715/764, 715/716, 715/972
• US Class Current: 715/764
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/383 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/538 Presentation of query results
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation

Abstract

Methods for providing information services related to visual imagery and operational details of various methods are described. The methods enable the capture of visual imagery, the request and presentation of information services related to the visual imagery, augmentation of visual imagery and continued interaction with the information services.

Claims

1. A method comprising:

upon detecting a click of a button at a wireless camera phone, starting recording of a first video of a scene of interest, wherein the scene of interest comprises text;

upon detecting a release of the button, ending recording of the first video and sending the first video to a server system, wherein the first video comprises a sequence of images, wherein a first image of the sequence comprises a first portion of the text, and a second image of the sequence comprises a second portion of the text;

extracting the first portion of the text from the first image using a text recognition engine;

extracting the second portion of the text from the second image using the text recognition engine;

generating at the server system a plurality of contexts based on the extracted first and second portions of the text;

querying a database using the plurality of contexts to generate a list of information services;

sending the list of information services to the wireless camera phone; and

when playing the first video on the wireless camera phone after the sending the list of information services to the wireless camera phone, overlaying a second video comprising the list of information services on the first video.

2. The method of claim 1 comprising during the extracting the first and second portions of the text using a text recognition engine, the generating at the server system a plurality of contexts, and the querying a database using the plurality of contexts, playing a video of an advertisement on the wireless camera phone.

3. The method of claim 1 comprising during the playing the first video on the wireless camera phone, detecting motion of the wireless camera phone via an accelerometer of the wireless camera phone;

and

after the detecting motion, pausing the playing of the first video.

4. The method of claim 3 wherein the after the detecting motion comprises pausing playing of the second video comprising the list of information services.

5. The method of claim 1 wherein during the overlaying a second video comprising the list of information services, the method comprises:

receiving an indication to filter the list of information services; and

after the receiving the indication, selecting a filter; and

applying the selected filter to the list of information services.

6. The method of claim 5 wherein the indication comprises a speech input.

7. The method of claim 6 wherein the speech input comprises filter criteria.

8. The method of claim 5 wherein the selected filter to apply to the list of information services is based on user preferences.

9. The method of claim 5 wherein the selected filter to apply to the list of information services is based on knowledge of a user'"'"'s interaction history with at least a subset of information services in the list of information services.

10. The method of claim 5 wherein the receiving an indication to filter the list of information services is via a graphical user interface component displayed on the wireless camera phone.

11. The method of claim 1 wherein the overlaying a second video comprising the list of information services on the first video, comprises displaying an icon for each information service in the list of information services to indicate an information service type.

12. A method comprising:

upon detecting a click of a button at a wireless camera phone, starting recording of a first video of a scene of interest, wherein the scene of interest comprises text;

upon detecting a release of the button, ending recording of the first video and sending the first video to a server system, wherein the first video comprises a sequence of images, wherein a first image of the sequence comprises a first portion of the text, and a second image of the sequence comprises a second portion of the text;

extracting the first portion of the text from the first image using a text recognition engine;

extracting the second portion of the text from the second image using the text recognition engine;

generating at the server system a plurality of contexts based on the extracted first and second portions of the text;

querying a database using the plurality of contexts to generate a list of information services;

sending the list of information services to the wireless camera phone; and

when playing the first video on the wireless camera phone after the sending the list of information services to the wireless camera phone, overlaying graphics comprising the list of information services on the first video.

13. The method of claim 12 wherein the button is a graphical button displayed on a display of the wireless camera phone.

14. The method of claim 12 wherein the button is a physical button of the wireless camera phone.

15. The method of claim 12 wherein the generating at the server system a first plurality of contexts based on the extracted first and second portions of the text comprises using location information of the wireless camera phone.

16. The method of claim 12 wherein the generating at the server system a first plurality of contexts based on the extracted first and second portions of the text comprises using knowledge of a user'"'"'s interaction history.

17. The method of claim 12 wherein the first video comprises speech input and the generating at the server system a plurality of contexts comprises using the speech input.

18. The method of claim 17 wherein the speech input comprises speech from a user of the wireless camera phone and speech from a person in the scene of interest.

19. The method of claim 12 wherein the graphics are semi-transparent.

20. The method of claim 12 wherein the graphics are opaque.