Invisible Junction Feature Recognition For Document Security or Annotation

• US 20090016615A1
• Filed: 03/31/2008
• Published: 01/15/2009
• Est. Priority Date: 07/11/2007
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Uda, Shigeharu, Hull, Jonathan J., Erol, Berna, Ke, Qifa
• Granted Patent: US 9,373,029 B2
• Time in Patent Office: Days
• US Class Current: 382/217
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

The present invention uses invisible junctions which are a set of local features unique to every page of the electronic document to match the captured image to a part of an electronic document. The present invention includes: an image capture device, a feature extraction and recognition system and database. When an electronic document is printed, the feature extraction and recognition system captures an image of the document page. The features in the captured image are then extracted, indexed and stored in the database. Given a query image, the features in the query image are extracted and compared against those stored in the database to identify the query image. The feature extraction and recognition system of the present invention is integrated into a multifunction peripheral. This allows the feature extraction and recognition system to be used in conjunction with other modules to provide security and annotation applications.

Claims

1. A method for utilizing invisible junction features for performing an action, the method comprising:

receiving an input image;

recognizing an electronic document corresponding to the input image using invisible junctions; and

performing the action related to the recognized electronic document.

2. The method of claim 1, wherein the steps of receiving, recognizing and performing are performed by a multifunction peripheral.

3. The method of claim 1, wherein the action is enforcing a security procedure related the electronic document.

4. The method of claim 1, wherein performing the action comprises:

determining a security procedure associated with the electronic document; and

processing the scanned image according to the security procedure.

5. The method of claim 1, wherein performing the action comprises:

requesting a password;

receiving input from a user;

comparing the input from the user to a password for the electronic document;

processing the scanned image if the input from the user matches the password for the electronic document input.

6. The method of claim 1, wherein processing the scanned image comprises one from the group of outputting a copy of the scanned image, printing the electronic document, sending a copy of the scanned image by as a facsimile message, sending the electronic document as part of a by facsimile message, emailing a copy of the scanned image, emailing the electronic document, sending a message including the scanned image and sending a message including the electronic document.

7. The method of claim 1, wherein performing the action comprises storing a record indicating the action taken.

8. The method of claim 7, wherein the record includes one from the group of a time, a number of copies output, type of output, a recipient and whether image or document was acted upon.

9. The method of claim 1, wherein performing the action is inhibiting output of the scanned image or electronic document.

10. The method of claim 1, wherein performing the action is generating and sending a notification indicating the action taken and the electronic document.

11. The method of claim 1, wherein performing the action is printing the electronic document.

12. The method of claim 11, wherein:

the input image is one from the group of a document that has been updated since the input scanned image was produced, a page or portion of a much longer document and a previously copied or printed version of the document; and

the electronic document is one from the group of an updated version of the electronic document, an original electronic document and an entire electronic document.

13. The method of claim 1, wherein the action is updating a database with an entry related to the electronic document.

14. The method of claim 13, wherein updating the database is modifying a MMR record in the database to add one from the group of text, voice, video or metadata.

15. The method of claim 13, wherein updating the database includes:

prompting a user for updating instructions;

receiving updating instructions from the user; and

updating the database according to the received updating instructions.

16. The method of claim 13, wherein updating the database includes:

prompting a user for processing instructions;

receiving processing instructions from the user; and

processing the electronic document or the input image according to the received processing instructions.

17. The method of claim 1, wherein the action is adding an annotation for the electronic document.

18. The method of claim 17, wherein adding the annotation is modifying a MMR record in the database corresponding to the input image to add one from the group of text, voice, video or metadata.

19. The method of claim 17, wherein adding the annotation includes:

prompting a user for processing instructions;

receiving processing instructions from the user;

receiving an annotation from the user; and

processing the input image and the annotation according to the received processing instructions.

20. The method of claim 1, wherein recognizing the electronic document corresponding to the input image using invisible junctions includes creating a patch map.

21. The method of claim 20, wherein creating the patch map comprises:

defining a grid having a plurality of cells;

generating a query using invisible junction features of a cell of the grid,

applying the query to a database to generate a list of candidate documents defining a cell in the patch map corresponding having the list of candidate documents.

22. The method of claim 1, wherein recognizing the electronic document corresponding to the input image using invisible junctions includes preprocessing invisible junction features prior to generation of the patch map.

23. The method of claim 1, wherein recognizing the electronic document corresponding to the input image using invisible junctions includes post-processing using the patch map to determine at least one secure document a significant portion of which is included in the input image.

24. A system for utilizing invisible junction features for performing an action, the system comprising:

a recognizer unit having an input and an output for identifying an electronic document using an invisible junction feature descriptor from an input image, the input of the recognizer coupled to receive the input image, the output of the recognizer providing information about the electronic document; and

a security module having an input and an output for enforcing a security procedure on the electronic document, the input of the security module coupled to the output of recognizer to receive the information about the electronic document.

25. The system of claim 24, wherein the system is a multifunction peripheral, and the security module controls whether the multifunction peripheral can be used to copy or transmit the input image or the electronic document.

26. The system of claim 24, wherein the system includes a scanner, a processor, a input/output device, a printer and a database and the security module coupled to cooperate with the processor, the input/output device, the printer and the database to implement a security procedure.

27. The system of claim 24, wherein the security procedure is one from the group of requiring a password to perform the action, inhibiting printing, storing a record of the action, and generating and sending a notification of the action.

28. The system of claim 24, wherein recognizer unit creates and uses a patch map.

29. The system of claim 24, wherein recognizer unit preprocesses the invisible junction features prior to generation of the patch map.

30. The method of claim 24, wherein recognizer unit performs post-processing using the patch map to determine at least one secure document a significant portion of which is included in the input image.

31. A system for utilizing invisible junction features for performing an action, the system comprising:

a recognizer unit having an input and an output for identifying an electronic document using an invisible junction feature descriptor from an input image, the input of the recognizer coupled to receive the input image, the output of the recognizer providing information about the electronic document; and

a database updating unit having an input and an output for processing the input image and updating an entry related to the electronic document.

32. The system of claim 31 further comprising a database storing data including electronic documents, feature indexes and feature descriptors.

33. The system of claim 32 wherein the database also stores mixed-media reality documents.

34. The system of claim 31 wherein the database updating unit receives the input image and stores the input image in the database associated with the electronic document.

35. The system of claim 31 wherein the database updating unit receives an annotation related to the electronic document and stores the information associated with the electronic document in a database.

36. The system of claim 31 wherein the annotation is one from the group of text, audio, video and metadata.

37. The system of claim 31 wherein the database updating unit creates the information and the electronic document in a database.

38. The system of claim 31 wherein the database updating unit is software and routines operable on a portable computing device.

39. A method for utilizing invisible junction features to provide additional content, the method comprising:

receiving an input image;

recognizing an electronic document corresponding to the input image using invisible junctions;

sending a message to a user offering the additional content;

receiving input from the user; and

responsive to the input from the user, providing the additional content.

40. The method of claim 39, comprising publishing a document including a fragment, wherein the input image includes an image of the fragment, and wherein the electronic document corresponds to the fragment.

41. The method of claim 40, wherein the recognizing the electronic document uses the image of the fragment and a patch map to recognize the electronic document.

42. The method of claim 39, wherein the input from the user includes a request to provide additional content and an authorization for payment.

43. A method for utilizing invisible junction features to print an electronic document from a database, the method comprising:

receiving an image of a document ticket;

recognizing a electronic document corresponding to the image of a document ticket using invisible junctions; and

printing the recognized electronic document.

44. The method of claim 43, wherein the document ticket is any portion of a printed document this is a printed version of the electronic document stored in the database, the portion of the printed document having greater than a predefined size.

45. The method of claim 43, comprising storing the electronic document in the database.