Interactively predicting fields in a form
First Claim
1. A method for predicting a location of a field on a form based on an image of the form, the method comprising:
- receiving, by a computer system, binary data that represents the image of the form;
creating a data structure based on the binary data by;
analyzing the binary data, by the computer system, to determine a plurality of grid points, wherein the plurality of grid points define a plurality of rectangular boxes, and wherein each corner of each of the rectangular boxes is coincident with a grid point of the plurality of grid points;
creating the data structure, by the computer system, so that each element of the data structure maps to a different one of the rectangular boxes;
identifying, by the computer system, a plurality of visible line segments of the image by executing an image analysis algorithm that reads the binary data, that identifies visual data based on the binary data, and that analyzes the visual data to determine which of the visual data represent line segments; and
for each line segment of the identified plurality of visible line segments;
mapping the line segment, by the computer system, to a selected element of the data structure based on a determination that the line segment intersects a selected rectangular box that maps to the selected element;
identifying the location of the field of the form by;
determining, by the computer system, a location of a cursor being displayed at a display of the computer system after a user caused the cursor to move;
determining, by the computer system, that the location of the cursor intersects a portion of the image that is being displayed at the display, and that maps to a first rectangular box;
identifying a nearby line segment of the visible line segments, by the computer system, based on a proximity of a nearby rectangular box, that is mapped to the nearby line segment, to the first rectangular box;
determining the location of the field based on a location of the nearby line segment; and
determining an extent of the field based on an analysis of other nearby line segments; and
determining coordinates of a rectangular box that represents the field based on the location of the field and the extent of the field; and
displaying the rectangular box at the display.
2 Assignments
0 Petitions
Accused Products
Abstract
Methods, apparatuses, and embodiments related to interactively predicting fields in a form. A computer system received an image of a form. A user moves a cursor to a first field of the form, and the computer system automatically displays a predicted location of the field, including a bounding box that represents a boundary of the field. The computer system further predicts the field name/label based on text in the document. The user clicks on the field to indicate that he wants to digitize the field. When needed, the user interactively modifies the size of the bounding box that represents the extent of the field, changes the name/label of the field. Once finalized, the user can cause the field information (e.g., the bounding box coordinate, the bounding box location, the name/label of the field, etc.) to be written to a database.
16 Citations
11 Claims
-
1. A method for predicting a location of a field on a form based on an image of the form, the method comprising:
-
receiving, by a computer system, binary data that represents the image of the form; creating a data structure based on the binary data by; analyzing the binary data, by the computer system, to determine a plurality of grid points, wherein the plurality of grid points define a plurality of rectangular boxes, and wherein each corner of each of the rectangular boxes is coincident with a grid point of the plurality of grid points; creating the data structure, by the computer system, so that each element of the data structure maps to a different one of the rectangular boxes; identifying, by the computer system, a plurality of visible line segments of the image by executing an image analysis algorithm that reads the binary data, that identifies visual data based on the binary data, and that analyzes the visual data to determine which of the visual data represent line segments; and for each line segment of the identified plurality of visible line segments; mapping the line segment, by the computer system, to a selected element of the data structure based on a determination that the line segment intersects a selected rectangular box that maps to the selected element; identifying the location of the field of the form by; determining, by the computer system, a location of a cursor being displayed at a display of the computer system after a user caused the cursor to move; determining, by the computer system, that the location of the cursor intersects a portion of the image that is being displayed at the display, and that maps to a first rectangular box; identifying a nearby line segment of the visible line segments, by the computer system, based on a proximity of a nearby rectangular box, that is mapped to the nearby line segment, to the first rectangular box; determining the location of the field based on a location of the nearby line segment; and determining an extent of the field based on an analysis of other nearby line segments; and determining coordinates of a rectangular box that represents the field based on the location of the field and the extent of the field; and displaying the rectangular box at the display. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method comprising:
-
receiving, by a computer system, binary data that represents an image of a form; executing an image analysis algorithm, by the computer system, that reads the binary data, that identifies visual data based on the binary data, and that analyzes the visual data to determine which of the visual data represent visual line segments; creating a data structure, by the computer system, based on an identification of the visual line segments; receiving data, by the computer system, that indicates a location of a cursor on a display and that indicates a particular location of the image; predicting a location and a boundary of a field of the form, by the computer system, by; accessing a first element of the data structure, based on a location indicated by a cursor, to identify a visual line segment to use in a determination of a predicted location of the field; and accessing a second element of the data structure, based on a location of the identified visual line segment or the location indicated by the cursor, to obtain information regarding other visual line segments to use in a determination of a predicted boundary of the field; causing the predicted boundary of the field to be displayed at a location of the image determined based on the predicted location of the field; executing an optical character recognition (OCR) algorithm on a portion of the image that is determined based on coordinates of the predicted boundary; determining a predicted label of the field based on results of said executing of the OCR algorithm and based on a character input by a user to indicate a label of the field; causing the predicted label to be displayed; in response to receipt of data that indicates an intent of the user to create the field, writing data that indicate the predicted boundary and the predicted label to a database that is associated with the image; receiving a second image of a filled out version of the form; accessing the database to determine the coordinates of the predicted boundary; executing the OCR algorithm on a portion of the second image that is determined based on the coordinates; determining a value of the field based on said executing the OCR algorithm on the portion of the second image; and writing the value to the database to associate the value with the predicted label.
-
-
9. A method comprising:
-
receiving, by a computer system, binary data that represents an image of a form; executing an image analysis algorithm, by the computer system, that reads the binary data, that identifies visual data based on the binary data, and that analyzes the visual data to determine which of the visual data represent visual line segments; creating a data structure, by the computer system, based on an identification of the visual line segments; receiving data, by the computer system, that indicates a location of a cursor on a display and that indicates a particular location of the image; predicting a location and a boundary of a field of the form, by the computer system, by; accessing a first element of the data structure, based on a location indicated by a cursor, to identify a visual line segment to use in a determination of a predicted location of the field; and accessing a second element of the data structure, based on a location of the identified visual line segment or the location indicated by the cursor, to obtain information regarding other visual line segments to use in a determination of a predicted boundary of the field; causing the predicted boundary of the field to be displayed at a location of the image determined based on the predicted location of the field; writing data that indicates coordinates of the predicted boundary of the field to a database; receiving a second image of a filled out version of the form; accessing the database, based on the second image, to determine the coordinates of the predicted boundary of the field; executing an algorithm on a portion of the second image that is determined based on the coordinates, wherein the portion of the second image being determined based on the coordinates includes being determined based on an oversizing of the predicted boundary; determining a value of the field based on said executing the algorithm on the portion of the second image; and writing the value to the database to associate the value with the predicted label.
-
-
10. A computing system comprising:
-
a processor; a networking interface coupled to the processor; and a memory coupled to the processor and storing instructions which, when executed by the processor, cause the computing system to perform operations including; receiving, via the networking interface, binary data that represents an image of a form; executing an image analysis algorithm that reads the binary data, that identifies visual data based on the binary data, and that analyzes the visual data to determine which of the visual data represent visual line segments; writing data that represents one of the visual line segments to a selected element of a data structure, wherein the data structure is organized so that adjacent portions of the image map to adjacent elements of the data structure; receiving data that indicates a particular location of the image; predicting a location and a boundary of a field of the form by; accessing a first element of the data structure, based on a location indicated by a cursor, to identify a visual line segment to use in a determination of a predicted location of the field; and accessing a second element of the data structure, based on a location of the identified visual line segment or the location indicated by the cursor, to obtain information regarding other visual line segments to use in a determination of a predicted boundary of the field; and causing the predicted boundary of the field to be displayed at a location of the image determined based on the predicted location of the field, wherein a first element of the data structure has an index, wherein a second element of the data structure has an index, wherein the index of the second element is a unit increment of the index of the first element, and wherein the first element and the second element are adjacent elements of the data structure because the index of the second element is a unit increment of the index of the first element. - View Dependent Claims (11)
-
Specification