Methods and apparatus to determine the dimensions of a region of interest of a target object from an image using target object landmarks

• US 10,235,585 B2
• Filed: 09/07/2016
• Issued: 03/19/2019
• Est. Priority Date: 04/11/2016
• Status: Active Grant

• Current Assignee: The Nielsen Company LLC (MOOD Media Corporation)
• Original Assignee: Nielsen Company (US) LLC (RR Donnelley & Sons Company)
• Inventors: Deng, Kevin
• Primary Examiner(s): Niu, Feng
• Application Number:
  US15/258,325
• Publication Number:
  US 20170293819A1
• Time in Patent Office: 923 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/147 Determination of region of interest
• CPC Class Codes: G06V 30/19007 Matching; Proximity measures
• CPC Class Codes: G06V 30/19067 Matching configurations of points or features, e.g. constellation matching
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images

Abstract

Methods and apparatus to determine the dimensions of a region of interest of a target object and a class of the target object from an image using target object landmarks are disclosed herein. An example method includes identifying a landmark of a target object in an image based on a match between the landmark and a template landmark; classifying a target object based on the identified landmark; projecting dimensions of the template landmark based on a location of the landmark in the image; and determining a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

Claims

1. A method comprising:

identifying, by executing instructions with a processor, a landmark of a target object in an image based on a match between the landmark and a template landmark;

determining, by executing instructions with the processor, a position and an orientation of the landmark relative to the image;

projecting, by executing instructions with the processor, dimensions of the template landmark based on the position and the orientation of the landmark in the image by;

generating, by executing instructions with the processor, a left side search area, the left side search area being a first region including a first edge of the landmark;

determining, by executing instructions with the processor, a first landmark edge by analyzing first transition points in the left side search area;

generating, by executing instructions with the processor, a right side search area, the right side search area being a second region including a second edge of the landmark; and

determining, by executing instructions with the processor, a second landmark edge by analyzing second transition points in the right side search area; and

defining, by executing instructions with the processor, a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

2. The method of claim 1, further including comparing the landmark to a plurality of template landmarks.

3. The method of claim 1, wherein the template landmark is transformed to correspond to a size and the orientation of the landmark.

4. The method of claim 1, further including:

determining a first corner of the region of interest based on a first intersection between a third edge of the template landmark and the first landmark edge; and

determining a second corner of the region of interest based on a second intersection between the third edge of the template landmark and the second landmark edge.

5. The method of claim 4, further including determining a left side edge of the target object and a right side edge of the target object, the left side edge corresponding to the first landmark edge and the right side edge corresponding to the second landmark edge.

6. The method of claim 5, wherein the left side edge and the right side edge of the target object has a restricted range in direction and area.

7. The method of claim 5, further including determining text lines within a third region, the third region corresponding to the third edge of the landmark, the left side edge, the right side edge, and a bottom image line corresponding to a bottom of the image.

8. The method of claim 7, further including:

determining a third corner of the region of interest based on a third intersection between the left side edge and a text line closest to the bottom image line; and

determining a fourth corner of the region of interest based on a fourth intersection between the right side edge and the text line closest to the bottom image line.

9. The method of claim 8, wherein the first, second, third, and fourth corners of the region of interest improve an accuracy of text recognition techniques.

10. The method of claim 1, further including determining a class of the target object based on the matched template landmark.

11. The method of claim 1, wherein the landmark includes a text header.

12. The method of claim 1, wherein the image includes an image of a receipt.

13. The method of claim 12, further including determining data based on the printed text in the region of interest.

14. An apparatus comprising:

a landmark detector to identify a landmark of a target object in an image based on a match between the landmark and a template landmark; and

a corner determiner to;

determine a position and an orientation of the landmark relative to the image,

project dimensions of the template landmark based on the position and the orientation of the landmark relative to the image by;

generating a left side search area, the left side search area being a first region including a first edge of the landmark;

determining a first landmark edge by analyzing first transition points in the left side search area;

generating a right side search area, the right side search area being a second region including a second edge of the landmark; and

determining a second landmark edge by analyzing second transition points in the right side search area; and

define a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

15. The apparatus of claim 14, wherein the landmark detector is to compare the landmark to a plurality of template landmarks.

16. The apparatus of claim 14, wherein the corner determiner is to transform the template landmark to match a size and orientation of the landmark.

17. The apparatus of claim 14, wherein the corner determiner is to:

determine a first corner of the region of interest based on a first intersection between a third edge of the template landmark and the first landmark edge; and

determine a second corner of the region of interest based on a second intersection between the third edge of the template landmark and the second landmark edge.

18. The apparatus of claim 17, wherein the corner determiner is to determine a left side edge of the target object and a right side edge of the target object, the left side edge corresponding to the first landmark edge and the right side edge corresponding to the second landmark edge.

19. The apparatus of claim 18, wherein the left side edge and the right side edge of the target object has a restricted range in direction and area.

20. The apparatus of claim 18, wherein the corner determiner is to determine text lines within a third region, the third region corresponding to the third edge of the template landmark, the left side edge, the right side edge, and a bottom image line corresponding to a bottom of the image.

21. A tangible computer readable storage medium comprising instructions which, when executed, cause a processor to at least:

identify a landmark of a target object in an image based on a match between the landmark and a template landmark;

determine a position and an orientation of the landmark relative to the image;

project dimensions of the template landmark based on the position and the orientation of the landmark relative to the image by;

generating a left side search area, the left side search area being a first region including a first edge of the landmark;

determining a first landmark edge by analyzing first transition points in the left side search area;

generating a right side search area, the right side search area being a second region including a second edge of the landmark; and

determining a second landmark edge by analyzing second transition points in the right side search area; and

define a region of interest based on the projected dimensions, the region of interest corresponding to text printed on the target object.

22. The tangible computer readable storage medium of claim 21, wherein the instructions, that when executed, further cause the processor to:

determine a first corner of the region of interest based on a first intersection between a third edge of the landmark and the first landmark edge, and

determine a second corner of the region of interest based on a second intersection between the third edge of the landmark and the second landmark edge.