Dynamic Presentation of Targeted Information in a Mixed Media Reality Recognition System

• US 20150324848A1
• Filed: 07/20/2015
• Published: 11/12/2015
• Est. Priority Date: 10/01/2004
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey, Hull, Jonathan J.
• Granted Patent: US 10,007,928 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/3326 using relevance feedback from the user, e.g. relevance feedback on documents, documents sets, document terms or passages
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/437 Administration of user profiles, e.g. generation, initialisation, adaptation, distribution
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06F 18/285 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0241 Advertisements
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06V 30/19113 Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
• CPC Class Codes: G06V 30/1916 Validation; Performance evaluation
• CPC Class Codes: G06V 30/1918 Fusion techniques, i.e. combining data from various sources, e.g. sensor fusion
• CPC Class Codes: G06V 30/414 : Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

A context-aware targeted information delivery system comprises a mobile device, an MMR matching unit, a plurality of databases for user profiles, user context and advertising information, a plurality of comparison engines and a plurality of weight adjusters. The mobile device is coupled to deliver an image patch to the MMR matching unit which in turn performs recognition to produce recognize text. The recognized text is provided to a first and second comparison engines to produce relevant topics and relevant ads. The relevant topics and relevant ads are adjusted with information from a user context database including information such as location, date, time, and other information from a user profile. The third comparison engine compares the relevant topics and relevant ads to produce a set of final ads that are most related to the topics of interest for the user and delivered for display on to the mobile device.

Claims

1. A computer-implemented method comprising:

receiving an input image;

performing, with one or more processors, a visual search using the input image to produce recognized text and a document context;

receiving targeted information;

generating, with the one or more processors, relevant targeted information based on the recognized text, the document context, and the targeted information.

2. The computer-implemented method of claim 1, wherein receiving the input image comprises receiving the input image from a user device.

3. The computer-implemented method of claim 1, wherein the targeted information includes advertisements.

4. The computer-implemented method of claim 1, further comprising transmitting the relevant targeted information for presentation on a user device.

5. The computer-implemented method of claim 1, wherein the document context includes at least one from the group of:

text appearing in the image patch;

text surrounding the image patch;

previous targets the user hovered over; and

hotspots surrounding the image patch.

6. The computer-implemented method of claim 1, further comprising:

determining a word relevancy weight for each word of the recognized text, wherein generating relevant targeted information is based on the word relevancy weight for each word of the recognized text.

7. The computer-implemented method of claim 6, wherein the word relevancy weight is based on a distance from the center of the input image.

8. A system comprising:

one or more processors; and

a memory storing instructions, which when executed, cause the one or more processors to;

receive an input image;

perform a visual search using the input image to produce recognized text and a document context;

receive targeted information;

generate relevant targeted information based on the recognized text, the document context, and the targeted information.

9. The system of claim 8, wherein the system receives the input image from a user device.

10. The system of claim 8, wherein the targeted information includes advertisements.

11. The system of claim 8, wherein the instructions cause the one or more processors to transmit the relevant targeted information for presentation on a user device.

12. The system of claim 8, wherein the document context includes at least one from the group of:

text appearing in the image patch;

text surrounding the image patch;

previous targets the user hovered over; and

hotspots surrounding the image patch.

13. The system of claim 8, wherein the instructions cause the one or more processors to determine a word relevancy weight for each word of the recognized text, wherein the one or more processors uses the word relevancy weight for each word of the recognized text to generate relevant targeted information.

14. The system of claim 13, wherein the word relevancy weight is based on a distance from the center of the input image.

15. A computer program product comprising a non-transitory computer useable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

receive an input image;

perform a visual search using the input image to produce recognized text and a document context;

receive targeted information;

generate relevant targeted information based on the recognized text, the document context, and the targeted information.

16. The computer program product of claim 15, wherein the targeted information includes advertisements.

17. The computer program product of claim 15, wherein the computer readable program causes the computer to transmit the relevant targeted information for presentation on a user device.

18. The computer program product of claim 15, wherein the document context includes at least one from the group of:

text appearing in the image patch;

text surrounding the image patch;

previous targets the user hovered over; and

hotspots surrounding the image patch.

19. The computer program product of claim 15, wherein the computer readable program causes the computer to determine a word relevancy weight for each word of the recognized text, wherein the computer uses the word relevancy weight for each word of the recognized text to generate relevant targeted information.

20. The computer program product of claim 19, wherein the word relevancy weight is based on a distance from the center of the input image.