Data access based on human-produced images

• US 5,745,610 A
• Filed: 10/13/1995
• Issued: 04/28/1998
• Est. Priority Date: 07/22/1993
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Johnson, Walter A. L.
• Primary Examiner(s): Couso, Jose L.
• Assistant Examiner(s): BELLA, MATTHEW C
• Application Number:
  US08/543,232
• Time in Patent Office: 928 Days
• Field of Search: 382/309, 382/311, 358/403, 358/436, 358/440, 358/462, 358/464, 379/100
• US Class Current: 382/309
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/5854 using shape and object relationship

Abstract

An item of data, such as data defining an image of a document, and identifying image data are stored in memory. The identifying image data define a human-produced identifying image for the item of data. In response to query image data defining a human-produced query image similar to the identifying image, the query image data and the identifying image data are used to access the item of data. The identifying image and query image can be produced making marks by hand, by controlling a machine to make marks, or by interactively using a machine to produce data defining an image. The identifying image can be in a field on a form requesting a store operation, and the form can be part of an image set that includes images of a document that the identifying image identifies. Similarly, the query image can be in a field on an access form requesting an access operation. In response to the access form, a machine can automatically create a retrieval request form listing identifying images that are sufficiently similar to the query image to meet a criterion. In response to the marked retrieval request form, the machine can access and retrieve each requested document.

Claims

1. A method of operating a machine that includes:

image input circuitry for obtaining data defining images as input;

image output circuitry for providing data defining images as output;

image memory connected for receiving data defining images from the image input circuitry and for providing data defining images to the image output circuitry;

a processor connected for accessing data stored in the image memory; and

program memory storing instruction data indicating instructions the processor can execute;

the processor further being connected for accessing the instruction data in the program memory;

the method comprising;

receiving from the image input circuitry and storing in the image memory first input image data;

the first input image data defining a document image and a first form image;

the first form image including an identifying image for the document image;

the identifying image being a human-produced image;

operating the processor to use the first input image data to obtain document image data defining the document image and identifying image data defining the identifying image;

storing the document image data and the identifying image data in the image memory, associated so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

receiving from the image input circuitry and storing in the image memory second input image data;

the second input image data defining a second form image;

the second form image including a query image;

the query image being a human-produced image;

operating the processor to use the second input image data to obtain query image data defining the query image;

operating the processor to use the query image data and the identifying image data to access the document image data in the image memory;

operating the processor to use the document image data to obtain output image data defining a version of the document image; and

providing the output image data to the image output circuitry.

2. The method of claim 1 in which the identifying image data include pixel data indicating a value for each pixel of the identifying image and the query image data include pixel data indicating a value for each pixel of the query image;

the act of operating the processor to use the query image data and the identifying image data comprising comparing the pixel data of the identifying image data and of the query image data to determine that the identifying image and the query image are similar.

3. The method of claim 2 in which the act of operating the processor to use the query image data and the identifying image data further comprises obtaining data indicating a Hausdorff distance between the query image and the identifying image.

4. The method of claim 1 in which the identifying image data include a descriptor of the identifying image;

the act of operating the processor to use the query image data and the identifying image data comprising;

using the query image data to obtain a descriptor of the query image; and

comparing the descriptor of the query image with the descriptor of the identifying image to determine that the query image and the identifying image are similar.

5. The method of claim 4 in which the descriptor of the query image and the descriptor of the identifying image are one-dimensional signal descriptors.

6. The method of claim 1 in which the first form image shows a storage request form that includes the identifying image the first input image data including information indicating a request to store the document image data and the identifying image data;

the act of operating the processor to use the first input image data comprising;

operating the processor to use the first input image data to obtain the identifying image data; and

operating the processor to use the first input image data to obtain request identifying data indicating a request to store the document image data and the identifying image data.

7. The method of claim 1 in which the first form image shows a sheet that includes the identifying image;

the act of operating the processor to use the first input image data comprising;

operating the processor to use the first form image to obtain segment image data defining a segment of the image of the sheet that includes the identifying image; and

operating the processor to use the segment image data to obtain the identifying image data.

8. The method of claim 7 in which the sheet is a form that includes a field;

the field including the identifying image;

the segment image data defining the field.

9. The method of claim 7 in which the sheet also includes the document image;

the segment being a relatively small part of the document image.

10. The method of claim 7 in which the sheet shows a boundary mark indicating a boundary on the sheet;

the segment image data defining a part of the sheet within the boundary.

11. The method of claim 7 in which the identifying image includes marks made by a human by hand;

the act of operating the processor to use the segment image data to obtain the identifying image data being performed without recognizing the marks.

12. The method of claim 1 in which the act of operating the processor to use the first input image data comprises:

using the first input image data to obtain pixel data indicating a value for each pixel in the identifying image.

13. The method of claim 1 in which the act of operating the processor to use the query image data and the identifying image data to access the document image data is performed automatically in response to the second input image data.

14. The method of claim 13 in which the second input image data include information indicating a request to retrieve document image data with an identifying image similar to the query image;

the act of operating the processor to use the second input image data comprising;

using the second input image data to obtain request identifying data indicating a request to retrieve document image data with an identifying image similar to the query image.

15. The method of claim 13 in which the second input image data further include information indicating an image destination;

the act of providing the output image data to the image output circuitry comprising;

operating the processor to provide the output image data to the image output circuitry for transmission to the image destination.

16. The method of claim 13 in which the second form image shows a sheet that includes the query image;

the query image data including sheet image data defining an image of the sheet;

the act of operating the processor to use the query image data and the identifying image data comprising;

using the sheet image data to obtain segment image data defining a segment of the image of the sheet that includes the query image; and

using the segment image data and the identifying image data to determine that the query image is similar to the identifying image.

17. The method of claim 16 in which the sheet is a form that includes a field;

the field including the query image;

the segment image data defining the field.

18. The method of claim 16 in which the sheet shows a boundary mark indicating a boundary on the sheet;

the segment image data defining a part of the sheet within the boundary.

19. The method of claim 16 in which the query image includes marks made by a human by hand;

the act of using the segment image data and the identifying image data being performed without recognizing the marks.

20. The method of claim 1 in which the act of operating the processor to use the query image data and the identifying image data comprises:

using the query image data to obtain pixel data indicating a value for each pixel in the query image.

21. The method of claim 1 in which the act of operating the processor to use the query image data and the identifying image data further comprises:

using the identifying image data to create a retrieval request form that includes the identifying image and a check box;

providing form output data defining the retrieval request form to the image output circuitry;

receiving form input data defining a marked version of the retrieval request form;

the marked version including a mark in the check box; and

in response to the form input data, accessing the document image data.

22. The method of claim 1 in which the machine further includes a database system for associating data stored in the image memory;

the act of storing the document image data and the identifying image data comprising operating the processor to associate the document image data and the identifying image data in the database system.

23. A machine comprising:

image input circuitry for obtaining data defining images as input;

image memory connected to said image input circuitry, said image memory for storing data defining images;

a processor connected to said image memory;

program memory storing instruction data indicating instructions the processor can execute;

the processor further being connected for accessing the instruction data in the program memory;

the processor, in executing the instructions;

accessing first input image data in the image memory;

the first input image data being received from the image input circuitry;

the first input image data defining a document image and a first form image;

the first form image including an identifying image for the document image;

the identifying image being a human-produced image;

using the first input image data to obtain document image data defining the document image and identifying image data defining the identifying image;

storing the document image data and the identifying image data so that the document image data can be accessed using the identifying image data;

the document image data and the identifying image data being associated so that the processor, upon accessing the identifying image data can then access the document image data;

accessing second input image data in the image memory;

the second input image data being received from the image input circuitry;

the second input image data defining a second form image;

the second form image including a query image;

the query image being a human-produced image;

using the second input image to obtain a query image data defining the query image; and

using the query image data and the identifying image data to access the document image data in the image memory.

24. An article of manufacture for use in a machine that includes:

image input circuitry for obtaining data defining images as input;

image memory connected to said image input circuitry, said image memory for storing data defining images;

a processor connected to said image memory;

the article comprising;

a storage medium that can be accessed by the storage medium access device when the article is used in the system; and

data stored by the storage medium so that the storage medium access device can provide the stored data to the processor when the article is used in the system;

the stored data comprising instruction data indicating instructions the processor can execute;

the processor, in executing the instructions;

accessing first input image data in the image memory;

the first input image data being received from the image input circuitry;

the first input image data defining a document image and a first form image;

the first form image including an identifying image for the document image;

the identifying image being a human-produced image;

using the first input image data to obtain document image data defining the document image and identifying image data defining the identifying image;

storing the document image data and the identifying image data so that the document image data can be accessed using the identifying image data;

the document image data and the identifying image data being associated so that the processor, upon accessing the identifying image data, can then access the document image data;

accessing second input image data in the image memory;

the second input image data being received from the image input circuitry;

the second input image data defining a second form image;

the second form image including a query image;

the query image being a human-produced image;

using the second input image to obtain a query image data defining the query image; and

using the query image data and the identifying image data to access the document image data in the image memory.

25. A method of operating a machine that includes:

image input circuitry for obtaining data defining images as input;

memory for storing data; and

a processor connected for receiving data defining images from the image input circuitry and connected for accessing the memory;

the method comprising;

operating the processor to receive storage request image data from the image input circuitry;

the storage request image data defining a first image set that shows a document image and an identifying image for the document image;

the identifying image being a human-produced image;

in response to the storage request image data, operating the processor to;

automatically use the storage request image data to obtain document image data defining the document image and identifying image data defining the identifying image; and

automatically store in memory the document image data and the identifying image data, associated so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

the identifying image data including pixel data indicating a value for each pixel of the identifying image;

operating the processor to receive query image data from the image input circuitry, the query image data defining a query image that is similar to the identifying image;

the query image being a human-produced image;

the query image data including pixel data indicating a value for each pixel of the query image;

in response to the query image data, operating the processor to automatically use the query image data and the identifying image data to access the document image data stored in memory;

the act of operating the processor to automatically use the query image data and the identifying image data comprising;

comparing the pixel data of the query image data and the pixel data of the identifying image data without obtaining character codes identifying characters in the query image or the identifying image;

the act of comparing the pixel data of the query image data and the pixel data of the identifying image data determining whether the query image and the identifying image are similar without performing recognition on the query image or the identifying image; and

if the query image and the identifying image are determined to be similar, accessing the document image data;

operating the processor to use the document image data to obtain second image data;

the second image data defining a version of the document image.

26. A method of operating a machine that includes:

image input circuitry for obtaining data defining images as input;

memory for storing data;

a processor connected for receiving data defining images from the image input circuitry and connected for accessing the memory; and

a database system stored in the memory;

the method comprising;

storing in memory document image data defining a document image and identifying image data defining an identifying image for the document image;

the identifying image being a human-produced image;

the identifying image data including pixel data indicating a value for each pixel of the identifying image;

operating the processor to associate the document image data and the identifying image data in the database system;

the document image data and the identifying image data being associated in the database system so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

operating the processor to receive query image data from the image input circuitry, the query image data defining a query image that is similar to the identifying image;

the query image data including pixel data indicating a value for each pixel of the query image;

the query image being a human-produced image; and

in response to the query image data, operating the processor to automatically use the query image data and the identifying image data to access the document image data stored in memory;

the act of operating the processor to automatically use the query image data and the identifying image data comprising;

comparing the pixel data of the query image data and the pixel data of the identifying image data without obtaining character codes identifying characters in the query image or the identifying image;

the act of comparing the pixel data of the query image data and the pixel data of the identifying image data determining whether the query image and the identifying image are similar without performing recognition on the query image or the identifying image; and

if the query image and the identifying image are determined to be similar, using the database system to access the document image data.

27. A method of operating a machine that includes:

image input circuitry for obtaining data defining images as input;

memory for storing data;

a processor connected for receiving data defining images from the image input circuitry and connected for accessing the memory; and

a file system in the memory;

the method comprising;

storing in memory document image data defining a document image and identifying image data defining an identifying image for the document image;

the identifying image being a human-produced image;

the identifying image data including pixel data indicating a value for each pixel of the identifying image;

operating the processor to associate the document image data and the identifying image data in the file system;

the document image data and the identifying image data being associated in the file system so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

operating the processor to receive query image data from the image input circuitry, the query image data defining a query image that is similar to the identifying image;

the query image data including pixel data indicating a value for each pixel of the query image;

the query image being a human-produced image; and

in response to the query image data, operating the processor to automatically use the query image data and the identifying image data to access the document image data stored in memory;

the act of operating the processor to automatically use the query image data and the identifying image data comprising;

comparing the pixel data of the query image data and the pixel data of the identifying image data without obtaining character codes identifying characters in the query image or the identifying image;

the act of comparing the pixel data of the query image data and the pixel data of the identifying image data determining whether the query image and the identifying image are similar without performing recognition on the query image or the identifying image; and

if the query image and the identifying image are determined to be similar, using the file system to access the document image data.

28. A method of operating a machine that includes:

image input circuitry for obtaining data defining images as input;

memory for storing data; and

a processor connected for receiving data defining images from the image input circuitry and connected for accessing the memory;

the method comprising;

storing in the memory document image data defining a document image and identifying image data defining an identifying image for the document image;

the identifying image data including pixel data indicating a value for each pixel of the identifying image;

the identifying image being a human-produced image;

the document image data and the identifying image data being associated so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

operating the processor to receive query image data from the image input circuitry, the query image data defining a query image that is similar to the identifying image;

the query image data including pixel data indicating a value for each pixel of the query image;

the query image being a human-produced image;

operating the processor to compare the pixel data of the identifying image data and the pixel data of the query image data without obtaining character codes identifying characters in the query image or the identifying image;

the act of operating the processor to compare the pixel data of the identifying image data and the pixel data of the query image data determining whether the query image and the identifying image are similar without performing recognition on the query image or the identifying image; and

if the processor determines that the query image and the identifying image are similar, operating the processor to access the document image data.

29. A method of operating a machine that includes:

image input circuitry for obtaining data defining images as input;

memory for storing data; and

a processor connected for receiving data defining images from the image input circuitry and connected for accessing the memory;

the method comprising;

storing in the memory document image data defining a document image and identifying image data defining an identifying image for the document image;

the document image data and the identifying image data being associated so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

the identifying image data including a one dimensional signal descriptor of the identifying image;

the identifying image being a human-produced image;

operating the processor to receive query image data from the image input circuitry, the query image data defining a query image that is similar to the identifying image;

the query image being a human-produced image;

operating the processor to use the query image data to obtain a one-dimensional signal descriptor of the query image;

operating the processor to compare the one-dimensional signal descriptor from the identifying image data and the one-dimensional signal descriptor obtained from the query image data without obtaining character codes identifying characters in the query image or the identifying image;

the act of operating the processor to compare the one-dimensional signal descriptor from the identifying image data and the one-dimensional signal descriptor from the query image data determining whether the query image and the identifying image are similar without performing recognition on the query image or the identifying image; and

if the processor determines that the query image and the identifying image are similar, operating the processor to access the document image data.

30. A machine comprising:

image input circuitry for obtaining data defining images as input;

image output circuitry for providing data defining images as output;

image memory connected for receiving data defining images from the image input circuitry and for providing data defining images to the image output circuitry;

a processor connected for accessing data stored in the image memory; and

program memory storing instruction data indicating instructions the processor can execute;

the processor further being connected for accessing the instruction data in the program memory;

the processor, in executing the instructions;

accessing first input image data in the image memory;

the first input image data being received from the image input circuitry;

the first input image data defining a document image and a first form image;

the first form image including an identifying image for the document image;

the identifying image being a human-produced image;

using the first input image data to obtain document image data defining the document image and identifying image data defining the identifying image;

storing the document image data and the identifying image data, associated so that the processor, upon subsequently accessing the identifying image data, can then access the document image data;

accessing second input image data in the image memory;

the second input image data being received from the image input circuitry;

the second input image data defining a second form image;

the second form image including a query image;

the query image being a human-produced image;

using the second input image data to obtain query image data defining the query image;

using the query image data and the identifying image data to access the document image data in the image memory;

using the document image data to obtain output image data defining a version of the document image; and

providing the output image data to the image output circuitry.

31. The machine of claim 30 in which the processor, in using the query image data and the identifying image data, does not obtain character codes.

32. The machine of claim 30 in which the identifying image data include pixel data indicating a value for each pixel of the identifying image and the query image data include pixel data indicating a value for each pixel of the query image;

the processor, in using the query image data and the identifying image data, comparing the pixel data of the identifying image data and of the query image data to determine that the identifying image and the query image are similar.

33. The machine of claim 32 in which the processor, in using the query image data and the identifying image data, further obtains data indicating a Hausdorff distance between the query image and the identifying image.

34. The machine of claim 30 in which the identifying image data include a descriptor of the identifying image;

the processor, in using the query image data and the identifying image data;

using the query image data to obtain a descriptor of the query image; and

comparing the descriptor of the query image with the descriptor of the identifying image to determine that the query image and the identifying image are similar.

35. The machine of claim 34 in which the descriptor of the query image and the descriptor of the identifying image are one-dimensional signal descriptors.

36. The machine of claim 30 in which the input image circuitry is connected for receiving facsimile transmissions.

37. The machine of claim 30 in which the output image circuitry is connected for providing facsimile transmissions.

38. The machine of claim 30, further comprising a database system;

the database system associating the document image data and the identifying image data so that the processor, upon accessing the identifying image data, can access the document image data.

39. A method of operating a machine that includes a memory for storing data;

the method comprising;

storing in the memory document image data defining a document image and identifying image data, associated so that, when the identifying image data is subsequently accessed, the document image data can then be accessed;

the identifying image data defining an identifying image for the document image;

the identifying image being a human-produced image that includes a first graphics feature that is not a character capable of being represented by a character code;

receiving query image data defining a query image that is similar to the identifying image;

the query image being a human-produced image that includes a second graphics feature that is not a character capable of being represented by a character code;

the second graphics feature being similar to the first graphics feature;

accessing the identifying image data;

comparing the query image data and the identifying image data to determine whether the query image and the identifying image are similar;

the act of comparing being performed without recognizing the first graphics feature or the second graphics feature; and

if the act of comparing determines that the query image and the identifying image are similar, accessing the document image data.

40. The method of claim 39 in which the query image includes marks made by a human by hand.

41. The method of claim 39 in which the query image includes marks made by a machine under control of a human.

42. The method of claim 39 in which the query image data is received by a machine that includes a user input device;

the machine receiving the query image data in response to signals from the user input device.

43. The method of claim 42 in which the machine also includes an image output device for presenting images to a user;

the user providing the signals to cause presentation of a desired image by the image output device.

44. The method of claim 42 in which the machine also includes, for each of a set of character types, character image data defining an image of a version of the character type made by a human by hand;

the signals from the user input device indicating a sequence of characters each of which is an instance of a character type in the set;

the query image showing the sequence of characters with each character being a version defined by the character image data of one of the set of character types.

45. The method of claim 39 in which the item of data defines a first image, the method further comprising using the item of data to present the first image to a user.

46. The method of claim 39 in which the first and second graphics features each include marks made by a human by hand.