Analyzing usage of visual content to determine relationships indicating unsuccessful attempts to retrieve the visual content

• US 9,870,388 B2
• Filed: 06/10/2013
• Issued: 01/16/2018
• Est. Priority Date: 07/31/2006
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Erol, Berna, Hull, Jonathan J.
• Primary Examiner(s): Paula, Cesar
• Assistant Examiner(s): Blackwell, James H
• Application Number:
  US13/914,417
• Publication Number:
  US 20130346431A1
• Time in Patent Office: 1,681 Days
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

An MMR system includes an analytics module for monitoring and analyzing the creation and usage of visual content search. The analytics module is part of an MMR gateway and provides a mechanism for administrators to determine the usage patterns and other information about users of the MMR system. The analytics module comprises a monitoring module, a usage database, an access analysis module and a presentation module. The monitoring module collects information about the retrieval request, image queries, context information and recognition results. The usage database stores the information collected by the monitoring module.

Claims

1. A system comprising:

one or more processors;

a monitoring module stored on a memory and executable by the one or more processors, the monitoring module configured to monitor and capture access information related to accessing an electronic database of features, the access information including metadata about user interactions with a hotspot embedded in a recognition result that is retrieved based on features in the electronic database of features that match features of a captured image;

a storage device coupled with the monitoring module, the storage device for storing the access information;

an access analysis module coupled with the storage device, the access analysis module configured to;

retrieve a first plurality of records from the access information, wherein the first plurality of records are records that include a successful retrieval result;

retrieve a second plurality of records from the access information, wherein the second plurality of records are records that include an unsuccessful retrieval result;

identify a relationship between the first plurality of records and the second plurality of records using mathematical calculations;

and

generate an analysis result based on the relationship between the first plurality of records and the second plurality of records, the analysis result comprising an indication of a number of unsuccessful visual retrieval attempts, based on capturing images of an object, before an average user ceases to capture additional images of the object; and

a presentation module coupled with the access analysis module, the presentation module configured to initiate an action based on the analysis result.

2. The system of claim 1, wherein the action comprises at least one of outputting the analysis result, connecting to a web page, emailing an article, playing a music clip, playing a video clip, and downloading an operating time for a business.

3. The system of claim 1, wherein the analysis result further comprises a conversion rate of the user purchasing a product related to the features that match features of captured images.

4. The system of claim 1, wherein the access information comprises indexing information, addition of hotspot data, and hotspot association information.

5. The system of claim 1, wherein the analysis result further comprises information relating to how retrieval performance affects user behavior.

6. The system of claim 1, wherein the analysis result further comprises a location and a direction in which a user is traveling.

7. The system of claim 1, wherein the presentation module combines the analysis result with one from a group of a matched feature and a map image.

8. A method comprising:

monitoring and capturing access information by one or more processors, the access information related to access to an electronic database of features, the access information including metadata about user interactions with a hotspot embedded in a recognition result that is retrieved based on features in the electronic database of features that match features of a captured image;

storing the access information in a storage device;

retrieving a first plurality of records from the access information, wherein the first plurality of records are records that include a successful retrieval result;

retrieving a second plurality of records from the access information, wherein the second plurality of records are records that include an unsuccessful retrieval result;

identifying a relationship between the first plurality of records and the second plurality of records using mathematical calculations;

generating an analysis result based on the relationship between the first plurality of records and the second plurality of records, the analysis result comprising an indication of a number of unsuccessful visual retrieval attempts, based on capturing images of an object, before an average user ceases to capture additional images of the object; and

initiating an action based on the analysis result.

9. The method of claim 8, wherein the action comprises at least one of outputting the analysis result, connecting to a web page, emailing an article, playing a music clip, playing a video clip, and downloading an operating time for a business.

10. The method of claim 9, wherein outputting the result comprises combining the analysis result with one from a group of a matched feature and a map image.

11. The method of claim 8, wherein the analysis result further comprises a conversion rate of the user purchasing a product related to the features that match features of captured images.

12. The method of claim 8, wherein the access information comprises indexing information, addition of hotspot data, and hotspot association information.

13. The method of claim 8, wherein the analysis result further comprises information relating to how retrieval performance affects user behavior.

14. The method of claim 8, wherein the analysis result further comprises a location and a direction in which a user is traveling.

15. A computer program product comprising a non-transitory computer useable medium including a computer-readable program, wherein the computer-readable program when executed on a computer causes the computer to:

monitor and capture access information by one or more processors, the access information related to access to an electronic database of features, the access information including metadata about user interactions with a hotspot embedded in a recognition result that is retrieved based on features in the electronic database of features that match features of a captured image;

store the access information in a storage device;

retrieve a first plurality of records from the access information, wherein the first plurality of records are records that include a successful retrieval result;

retrieve a second plurality of records from the access information, wherein the second plurality of records are records that include an unsuccessful retrieval result;

identify a relationship between the first plurality of records and the second plurality of records using mathematical calculations;

generate an analysis result based on the relationship between the first plurality of records and the second plurality of records, the analysis result comprising an indication of a number of unsuccessful visual retrieval attempts, based on capturing images of an object, before an average user ceases to capture additional images of the object; and

initiate an action based on the analysis result.

16. The computer program product of claim 15, wherein the action comprises at least one of outputting the analysis result, connecting to a web page, emailing an article, playing a music clip, playing a video clip, and downloading an operating time for a business.

17. The computer program product of claim 15, wherein the analysis result further comprises a conversion rate of the user purchasing a product related to the features that match features of captured images.

18. The method of claim 15, wherein the access information comprises indexing information, addition of hotspot data, and hotspot association information.

19. The method of claim 15, wherein the analysis result further comprises information relating to how retrieval performance affects user behavior.

20. The method of claim 15, wherein the analysis result further comprises a location and a direction in which a user is traveling.