Systems and methods for tax data capture and use

• US 9,639,900 B2
• Filed: 02/08/2016
• Issued: 05/02/2017
• Est. Priority Date: 02/28/2013
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Huang, Nankun, Eftekhari, Amir, Howe, Carol A., Tifford, Alan B., Ludwig, Jeffrey P.
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/018,662
• Publication Number:
  US 20160155202A1
• Time in Patent Office: 449 Days
• CPC Class Codes: G06Q 40/123 Tax preparation or submission
• CPC Class Codes: G06V 30/412 Layout analysis of documents structured with printed lines or input boxes, e.g. business forms or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

A computer-implemented method of acquiring tax data for use in tax preparation application includes acquiring an image of at least one document containing tax data therein with an imaging device. A computer extracts one or more features from the acquired image of the at least one document and compares the extracted one or more features to a database containing a plurality of different tax forms. The database may include a textual database and/or geometric database. The computer identifies a tax form corresponding to the at least one document from the plurality of different tax forms based at least in part on a confidence level associated with the comparison of the extracted one or more features to the database. At least a portion of the tax data from the acquired image is transferred into corresponding fields of the tax preparation application.

Claims

1. A computer-implemented method for preparing at least a portion of an electronic tax return with a computerized tax preparation application, the computer-implemented method comprising:

a computing device receiving an image of one or more documents containing tax data therein with an imaging device;

the computing device processing the image by;

extracting a plurality of features from the image of the one or more documents, wherein optical character recognition is performed on at least one of the features to output text,

comparing the extracted one or more features and output text to a graphical database and a textual database, respectively, containing a plurality of different tax forms, and

identifying a tax form corresponding to the one or more documents from the plurality of different tax forms based on a comparison of the extracted one or more features and the output text to the graphical database and the textual database, respectively; and

the computing device transferring at least a portion of the tax data determined from the image into corresponding fields of the computerized tax preparation application to automatically prepare at least a portion of the electronic tax return.

2. The method of claim 1, wherein the extracted one or more features comprise at least one of a title, a separator, a whitespace, a paragraph, an image and associated location information of an extracted feature.

3. The method of claim 1, wherein a confidence level associated with a particular tax form of the plurality of tax forms is generated based at least in part on the comparison of the extracted one or more features to the graphical database.

4. The method of claim 3, wherein a confidence level associated with a particular tax form of the plurality of different tax forms is generated based at least in part on the comparison of the output text to the textual database.

5. The method of claim 4, wherein when the graphical database and the textual database identify different tax forms, the tax form having the highest confidence value is selected.

6. The method of claim 4, wherein when the graphical database and the textual database identify different tax forms, an arbiter identifies the tax form to be selected based on a pre-trained set of documents.

7. The method of claim 1, wherein the computing device identifies a tax form corresponding to the one or more documents from the plurality of different tax forms based at least in part on a dataset of previously imaged documents.

8. The method of claim 1, wherein the imaging device comprises one of a mobile phone or tablet.

9. The method of claim 8, wherein the computing device is contained in the mobile phone or tablet.

10. The method of claim 8, wherein the computing device is remotely located with respect to the mobile phone or tablet and the image of the at least one document containing tax data is transferred to the computing device over a network for processing of the image.

11. The method of claim 1, wherein the receive image is in a format selected from the group consisting of JPEG, GIF, BMP, PNG, TIFF, RAW, PDF and RTF.

12. The method of claim 1, wherein image processing comprises identification of dark pixels within the received image, wherein feature extraction is based at least in part upon identification of the dark pixels.

13. The method of claim 1, wherein the received image is an image of a tangible, paper document that contains tax data and that is visually perceptible to the human eye.

14. A computer-implemented method for preparing at least a portion of an electronic tax return, the computer-implemented method comprising:

a computing device receiving an image of at least one document containing tax data therein with an imaging device;

the computing device processing the image by;

dividing the image the into a plurality of different image regions, wherein each image region contains image data relevant to a specific tax form,

extracting one or more features from at least one of the image regions of the image based at least in part upon analysis of pixels of the region,

comparing the extracted one or more features to a database containing a plurality of different tax forms, and

identifying a tax form corresponding to the at least one image region from the plurality of different tax forms based at least in part on a confidence level associated with the comparison of the extracted one or more features to the database; and

the computing device automatically transferring respective tax data determined from the at least one image region into respective fields of the electronic tax return to automatically prepare at least a portion of the electronic tax return.

15. The method of claim 14, further comprising:

the computing device extracting one or more features from one of the remaining image regions based at least in part upon analysis of pixels of the remaining image regions, the computing device comparing the extracted one or more features to a database containing a plurality of different tax forms;

the computing device identifying a tax form corresponding to the remaining image regions from the plurality of different tax forms based at least in part on a confidence level associated with the comparison of the extracted one or more features to the database; and

the computing device automatically transferring respective tax data determined from the image into respective fields of the electronic tax return to automatically prepare at least a portion of the electronic tax return.

16. The method of claim 14, wherein the extracted one or more features comprise at least one of a title, a separator, a whitespace, a paragraph, an image and associated location information of an extracted feature.

17. The method of claim 14, wherein the database comprises a graphical database.

18. The method of claim 14, wherein the database comprises a textual database.

19. The method of claim 14, wherein the imaging device comprises one of a mobile phone or tablet.

20. The method of claim 19, wherein the computing device is contained in the mobile phone or tablet.

21. The method of claim 14, wherein the receive image is in a format selected from the group consisting of JPEG, GIF, BMP, PNG, TIFF, RAW, PDF and RTF.

22. The method of claim 14, wherein image processing comprises identification of dark pixels within the received image, wherein feature extraction is based at least in part upon identification of the dark pixels.

23. The method of claim 14, wherein the received image is an image of a tangible, paper document that contains tax data and that is visually perceptible to the human eye.

24. A system for automatically preparing at least a portion of an electronic tax return, the system comprising:

a computing device configured to receive an image of one or more documents containing tax data therein, the computing device being configured to extract a plurality of features from the image of the one or more documents, wherein optical character recognition is performed on at least one of the features to output text;

a graphical database containing a plurality of different tax forms, wherein the computing device compares the extracted features to contents of the graphical database and outputs a first confidence level for one or more of the different tax forms;

a textual database containing a plurality of different tax forms, wherein the computing device compares the output text to contents of the textual database and outputs a second confidence level for one or more of the different tax forms;

an arbiter executed by the computing device for resolving conflicts between the first confidence level and the second confidence level, wherein the computing device identifies a tax form corresponding to the one or more documents based on the first and second confidence levels, and

wherein the computing device is further configured to populate at least one field of an interview screen generated by a computerized tax preparation application with at least a portion of the tax data determined from the received image of the one or more documents to automatically prepare at least a portion of the electronic tax return.

25. The system of claim 24, wherein the computing device further comprises an imaging device for imaging the one or more documents.

26. The system of claim 24, wherein the computerized tax preparation application is located on a mobile communications device.

27. The system of claim 24, wherein the computing device is configured to process the received image in a format selected from the group consisting of JPEG, GIF, BMP, PNG, TIFF, RAW, PDF and RTF.

28. The system of claim method of claim 24, wherein the computing device is configured to perform image processing comprising identifying dark pixels within the received image, wherein feature extraction is based at least in part upon identification of the dark pixels.

29. The method of claim 24, wherein the received image is an image of a tangible, paper document that contains tax data and that is visually perceptible to the human eye.