Web-based content detection in images, extraction and recognition

• US 8,385,589 B2
• Filed: 05/15/2008
• Issued: 02/26/2013
• Est. Priority Date: 05/15/2008
• Status: Active Grant

• Current Assignee: Piece Future Pte., Ltd.
• Original Assignee: Ricoh Company Limited
• Inventors: Erol, Berna, Hull, Jonathan J.
• Primary Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US12/121,275
• Publication Number:
  US 20090285444A1
• Time in Patent Office: 1,748 Days
• Field of Search: 382/100, 382/176, 382/229, 382/181, 382/199
• US Class Current: 382/100
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables

Abstract

A web-based image detection system includes: an image capture device, a web browser, a plug-in, and a recognition server. A captured image is processed by the plug-in to assess whether a pre-determined content type is contained with in the image. Once an initial analysis at the browser plug-in detects a pre-determined content type in an image, the image is transmitted to a recognition server for content recognition of the image by content type. The system thus enables users to link documents or other image types to electronic data using only a capture device and a web browser.

Claims

1. A method for web-based image content type detection using a browser plug-in, the method comprising:

receiving an image via a capture device associated with a client;

analyzing at the browser plug-in the image for detection of a content type;

computing a gradient of adjacent pixels in the image;

detecting the content type by performing edge detection based on the gradient;

indicating how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type; and

in response to the analyzing and detecting the content type within the image, transmitting the image to a server.

2. The method of claim 1, wherein the content type is at least one from a group of text, a barcode, a QR code, a fingerprint and a discernible object.

3. The method of claim 1, wherein the browser plug-in comprises a .swf file that runs in a Flash player.

4. The method of claim 1, wherein the capture device is a webcam.

5. The method of claim 1, further comprising:

receiving from the server recognition results associated with the image; and

displaying content associated with the recognition results.

6. The method of claim 1, further comprising receiving a webpage from the server that is associated with the image, the webpage including at least one of a comment, a marked section, a link to a uniform resource locator, a link to a document and a media clip.

7. The method of claim 1, wherein the client comprises a mobile device.

8. A method for web-based content type detection, the method comprising:

receiving an image via a capture device associated with a client;

analyzing the image at the client for detecting a content type within the image;

computing a gradient of adjacent pixels in the image;

detecting the content type by performing edge detection based on the gradient;

indicating how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type;

in response to the analyzing and detecting of the content type within the image, identifying, at the client, text image features for searching; and

transmitting from the client the text image features for searching a document server.

9. The method of claim 8, wherein the text image features are words.

10. The method of claim 8, further comprising:

receiving from the document server recognition results comprising a document matching the text image features; and

displaying content associated with the recognition results.

11. The method of claim 8, further comprising receiving a webpage from the document server that is associated with the text image features, the webpage including at least one of a comment, a marked section, a link to a uniform resource locator, a link to a document and a media clip.

12. The method of claim 8, wherein the edge detection comprises calculating vertical and horizontal gradients.

13. The method of claim 8, wherein the client comprises a mobile device.

14. A method for web-based content type detection, the method comprising:

receiving an image via a capture device associated with a client;

analyzing the image at the client for detecting a content type within the image;

computing a gradient of adjacent pixels in the image;

detecting the content type by performing edge detection based on the gradient;

indicating how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type;

in response to the analyzing and detecting the content type within the image, identifying, at the client, text image features for searching; and

performing a desktop search using the text image features as keywords.

15. The method of claim 14, further comprising receiving recognition results comprising a document matching the text image features.

16. The method of claim 15, wherein the document is selected from the group consisting of a file, an email, and a web history entry.

17. The method of claim 15, wherein the document is stored on the client.

18. The method of claim 15, further comprising displaying a thumbnail representation of the document matching the text image features and a link to the document matching the text image features.

19. The method of claim 14, wherein the edge detection comprises calculating vertical and horizontal gradients.

20. The method of claim 14, wherein the client comprises a mobile device.

21. A method for web-based content type detection using a browser plug-in, the method comprising:

providing by a server the browser plug-in for running on a client;

receiving an image at the server captured via a capture device associated with the client that used the browser plug-in;

analyzing the image at the server for detecting a content type within the image;

computing a gradient of adjacent pixels in the image;

detecting the content type by performing edge detection based on the gradient;

transmitting an indication to the client of how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type;

in response to the analyzing and detecting the content type within the image, identifying, at the server, features for searching; and

performing, at the server, a document recognition search using the features.

22. The method of claim 21, wherein the browser plug-in comprises a .swf file that runs in a Flash player.

23. The method of claim 21, wherein the edge detection comprises calculating vertical and horizontal gradients.

24. The method of claim 21, further comprising displaying a thumbnail representation of a document recognized using the features and a link to the document recognized using the features.

25. A system for web-based image content type detection using a browser plug-in, the system comprising:

a capture device associated with a client for receiving an image; and

the browser plug-in for analyzing the image, for computing a gradient of adjacent pixels in the image, for detecting the content type by performing edge detection based on the gradient, for indicating how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type, and for, in response to the analyzing and detecting the content type within the image, transmitting the image to a server.

26. The system of claim 25, wherein the content type is at least one from a group of text, a barcode, a QR code, a fingerprint and a discernible object.

27. The system of claim 25, wherein the browser plug-in comprises a .swf file that runs in a Flash player.

28. The system of claim 25, wherein the capture device is a webcam.

29. The system of claim 25, wherein the browser plug-in receives from the server recognition results associated with the image and the browser plug-in displays content associated with the recognition results.

30. The system of claim 25, wherein the browser plug-in receives a webpage from the server that is associated with the image, the webpage including at least one of a comment, a marked section, a link to a uniform resource locator, a link to a document and a media clip.

31. The system of claim 25, wherein the client comprises a mobile device.

32. A system for web-based content type detection, the system comprising:

a capture device associated with a client for receiving an image;

a browser plug-in for analyzing the image at the client, computing a gradient of adjacent pixels in the image, detecting a content type by performing edge detection based on the gradient, indicating how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type and, in response to the analyzing and detecting the content type within the image, identifying, at the client, text image features for searching; and

wherein the client transmits the text image features for searching a document server.

33. The system of claim 32, wherein the text image features are words.

34. The system of claim 32, wherein the browser plug-in receives from the document server recognition results comprising a document matching the text image features and the browser plug-in displays a thumbnail representation of the document matching the text image features and a link to the document matching the text image features.

35. The system of claim 32, wherein the browser plug-in receives a webpage from the document server that is associated with the text image features, the webpage including at least one of a comment, a marked section, a link to a uniform resource locator, a link to a document and a media clip.

36. The system of claim 32, wherein the edge detection comprises calculating vertical and horizontal gradients.

37. The system of claim 32, wherein the client comprises a mobile device.

38. A system for web-based content type detection, the system comprising:

a capture device associated with a client for receiving an image;

a browser plug-in for analyzing the image at the client, computing a gradient of adjacent pixels in the image, detecting a content type by performing edge detection based on the gradient, for indicating how much of the content type is identified from the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type and, in response to the analyzing and detecting the content type within the image, identifying, at the client, text image features for searching; and

a desktop search application interface for performing a desktop search using the text image features as keywords.

39. The system of claim 38, wherein the browser plug-in receives recognition results comprising a document matching the text image features.

40. The system of claim 39, wherein the document is selected from the group consisting of a file, an email, and a web history entry.

41. The system of claim 39, wherein the document is stored on the client.

42. The system of claim 39, wherein the browser plug-in displays a thumbnail representation of the document matching the text image features and a link to the document matching the text image features.

43. The system of claim 38, wherein the edge detection comprises calculating vertical and horizontal gradients.

44. The system of claim 38, wherein the client comprises a mobile device.

45. A method comprising:

receiving a browser plug-in from an intermediary server responsive to transmitting a request to connect to the intermediary server from a browser associated with a client;

modifying the browser plug-in based on a parameter associated with the client so that the performance of the client is optimized by the modified browser plug-in;

receiving an image associated with a capture device, the capture device associated with the client and controlled by the browser plug-in;

analyzing, by the browser plug-in, the image for detection of a content type;

computing, by the browser plug-in, a gradient of adjacent pixels in the image, the gradient computed for a predetermined step of adjacent pixels in the image;

detecting, by the browser plug-in, the content type by performing edge detection on the gradient;

indicating, by the browser plug-in, how much of the content type is identified in the image, the indicating visually depicting an amount of the content type detected in the image ranging from none to all of the content type; and

filtering, by the browser plug-in, the image for transmission to a recognition server, the filtering including converting the image to a bit map.

46. The method of claim 45, wherein the browser plug-in comprises a .swf file that runs in a Flash player.

47. The method of claim 45, wherein the edge detection comprises calculating vertical and horizontal gradients.

48. The method of claim 45, wherein the browser plug-in displays a thumbnail representation of a document recognized using features extracted from the image and a link to the document recognized using the features.