Methods and systems for mobile document acquisition and enhancement

• US 8,947,453 B2
• Filed: 04/01/2011
• Issued: 02/03/2015
• Est. Priority Date: 04/01/2011
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Chen, Lawrence Shao-hsien, Campbell, Richard John, Ferman, Ahmet Mufit
• Primary Examiner(s): CHOW, JEFFREY J
• Application Number:
  US13/078,070
• Publication Number:
  US 20120249554A1
• Time in Patent Office: 1,404 Days
• Field of Search: 345/629, 345/156, 348/333.02, 348/190, 348/195
• US Class Current: 345/629
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/147 Determination of region of interest

Abstract

Aspects of the present invention are related to systems and methods for correcting artifacts in a camera-captured image of a document or image of an object exhibiting document-like content. A mobile device may capture an image and send the image to a cloud computing system for processing. According to a first aspect of the present invention, the mobile device may provide real-time feedback cues to assist in the capture of an image. The mobile device may detect a region-of-interest in the captured image, and a user may refine or confirm the detected region-of-interest. The captured image, information identifying the region-of-interest and a metadata tag, referred to as a region-of-interest modification tag, indicating whether, or not, the region-of-interest was refined by a user may be sent to the cloud. The cloud may process the image giving priority to the region-of-interest received from the handset when the region-of-interest modification tag indicates that the region-of-interest was refined by a user over a cloud determined region-of-interest. The cloud may transmit, to the handset, the processing results.

Claims

1. A method for image correction, said method comprising:

capturing an image in a handset, wherein said capturing comprises;

generating real-time capture feedback comprising an acquisition-factor value; and

providing said real-time capture feedback for display on a display associated with said handset;

determining an acceptability indicator associated with said acquisition-factor value;

in said handset, determining a region-of-interest in said image;

forming a region-of-interest modification metadata tag, wherein said region-of-interest modification metadata tag indicates whether, or not, said region-of-interest is a modified region-of-interest relative to an automatically determined region-of-interest; and

transmitting, to a cloud, said image, a location identifier associated with said region-of-interest and said region-of-interest modification metadata tag.

2. A method as described in claim 1 further comprising receiving from said cloud a cloud-processed result.

3. A method as described in claim 2, wherein said cloud-processed result is a version of said image processed by at least one process selected from the group of processes consisting of illumination correction, perspective distortion correction, tone correction, sharpening, de-noising, de-blurring and resolution boosting.

4. A method as described in claim 2 further comprising routing, from said handset to an additional location, said cloud-processed result.

5. A method as described in claim 1, wherein said determining comprises:

detecting a first region-of-interest;

displaying said first region-of-interest on said display associated with said handset;

receiving a region-of-interest confirmation associated with said first region-of-interest; and

setting said region-of-interest to said first region-of-interest.

6. A method as described in claim 1, wherein said determining comprises:

detecting a first region-of-interest;

displaying said first region-of-interest on said display associated with said handset;

receiving a modified region-of-interest; and

setting said region-of-interest to said modified region-of-interest.

7. A method as described in claim 1 further comprising:

generating a low-resolution version of said image;

processing said low-resolution version of said image to form a preview image; and

displaying said preview image on said display associated with said handset.

8. A method as described in claim 7, wherein said processing said low-resolution version of said image comprises at least one process selected from the group of processes consisting of illumination correction, perspective distortion correction, tone correction, sharpening, de-noising, de-blurring and resolution boosting.

9. A method as described in claim 1 further comprising transmitting to said cloud a first metadata tag associated with routing information.

10. A method as described in claim 1 further comprising transmitting to said cloud a first metadata tag associated with an image acquisition parameter.

11. A method as described in claim 1 further comprising displaying said real-time capture feedback and said acceptability indicator on said display associated with said handset.

12. A method for image correction, said method comprising:

receiving, in a cloud from a handset, an image, a region-of-interest identifier and a region-of-interest modification metadata tag, wherein said region-of-interest modification metadata tag indicates whether, or not, said region-of-interest is a modified region-of-interest relative to an automatically determined region-of-interest;

processing said image; and

transmitting, to said handset, a cloud-processed result.

13. A method as described in claim 12 further comprising transmitting said cloud-processed result to an additional location.

14. A method as described in claim 13, wherein routing information identifying said additional location is sent as a first metadata from said handset to said cloud.

15. A method as described in claim 12, wherein:

said processing determines a cloud-detected region-of-interest when said region-of-interest modification metadata tag indicates said region-of-interest is not a modified region-of-interest; and

said processing uses said transmitted region-of-interest when said region-of interest modification metadata tag indicates said region-of-interest is a modified region-of-interest.

16. A method as described in claim 12, wherein:

said processing determines a cloud-detected region-of-interest when said region-of-interest modification metadata tag indicates said region-of-interest is not a modified region-of-interest; and

said processing refines said transmitted region-of-interest when said region-of-interest modification metadata tag indicates said region-of-interest is a modified region-of-interest.

17. A method as described in claim 12, wherein said processing comprises at least one process selected from the group of processes consisting of illumination correction, perspective distortion correction, tone correction, sharpening, de-noising, de-blurring and resolution boosting.

18. A method for image correction, said method comprising:

capturing an image in a handset, wherein said capturing comprises;

generating real-time capture feedback comprising an acquisition-factor value; and

providing said real-time capture feedback for display on a display associated with said handset;

determining an acceptability indicator associated with said acquisition-factor value;

in said handset, determining a region-of-interest in said image;

forming a region-of-interest modification metadata tag, wherein said region-of-interest modification metadata tag indicates whether, or not, said region-of-interest is a modified region-of-interest relative to an automatically determined region-of-interest;

transmitting, to a cloud, said image, a location identifier associated with said region-of-interest and said region-of-interest modification metadata tag;

receiving, in said cloud from said handset, said image, said location identifier associated with said region-of-interest and said region-of-interest modification metadata tag;

processing, in said cloud, said image; and

transmitting, to said handset from said cloud, a cloud-processed result.

19. A method as described in claim 18 further comprising receiving, at said handset from said cloud, said cloud-processed result.

20. A method as described in claim 18, wherein said processing comprises at least one process selected from the group of processes consisting of illumination correction, perspective distortion correction, tone correction, sharpening, de-noising, de-blurring and resolution boosting.

21. A method as described in claim 18 further comprising routing, from said handset to an additional location, said cloud-processed result.

22. A method as described in claim 18, wherein said determining comprises:

detecting a first region-of-interest;

displaying said first region-of-interest on said display associated with said handset;

receiving a region-of-interest confirmation associated with said first region-of-interest; and

setting said region-of-interest to said first region-of-interest.

23. A method as described in claim 18, wherein said determining comprises:

detecting a first region-of-interest;

displaying said first region-of-interest on said display associated with said handset;

receiving a modified region-of-interest; and

setting said region-of-interest to said modified region-of-interest.

24. A method as described in claim 18 further comprising:

generating a low-resolution version of said image;

processing said low-resolution version of said image to form a preview image; and

displaying said preview image on said display associated with said handset.

25. A method as described in claim 24, wherein said processing said low-resolution version of said image comprises at least one process selected from the group of processes consisting of illumination correction, perspective distortion correction, tone correction, sharpening, de-noising, de-blurring and resolution boosting.

26. A method as described in claim 18 further comprising transmitting to said cloud a first metadata tag associated with routing information.

27. A method as described in claim 18 further comprising transmitting to said cloud a first metadata tag associated with an image acquisition parameter.

28. A method as described in claim 18 further comprising displaying said real-time capture feedback and said acceptability indicator on said display associated with said handset.

29. A method as described in claim 18 further comprising transmitting said cloud-processed result to an additional location from said cloud.

30. A method as described in claim 18, wherein:

said processing determines a cloud-detected region-of-interest when said region-of-interest modification metadata tag indicates said region-of-interest is not a modified region-of-interest; and

said processing uses said transmitted region-of-interest when said region-of interest modification metadata tag indicates said region-of-interest is a modified region-of-interest.

31. A method as described in claim 18, wherein:

said processing determines a cloud-detected region-of-interest when said region-of-interest modification metadata tag indicates said region-of-interest is not a modified region-of-interest; and

said processing refines said transmitted region-of-interest when said region-of interest modification metadata tag indicates said region-of-interest is a modified region-of-interest.