Computer-readable storage medium, image recognition apparatus, image recognition system, and image recognition method

• US 8,625,898 B2
• Filed: 08/04/2011
• Issued: 01/07/2014
• Est. Priority Date: 02/24/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Osako, Satoru, Maruko, Ryota
• Primary Examiner(s): Bhatnagar, Anand
• Assistant Examiner(s): PARK, SOO JIN
• Application Number:
  US13/198,140
• Publication Number:
  US 20120219228A1
• Time in Patent Office: 887 Days
• Field of Search: 382/103, 382/181, 382190-197, 382199-206, 382/209, 382216-222
• US Class Current: 382/181
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/243 by compensating for image skew or non-uniform image deformations
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Abstract

First, a plurality of vertices of a contour of an object or of a design are detected from an image. Then, a predetermined number of division points are generated on each of sides connecting the plurality of detected vertices, so as to divide each side of at least one pair of two opposing sides into unequal parts. Then, a plurality of sample points are determined on the basis of straight lines connecting the division points on the two opposing sides to one another, and on the basis of pixel values of the sample points, it is determined whether or not a predetermined object or design is displayed in an area surrounded by the plurality of vertices in the image.

Claims

1. A non-transitory computer-readable storage medium having stored thereon an image recognition program which, when executed by a computer of an information processing apparatus, causes the computer to perform operations comprising:

acquiring an image;

detecting from the image a plurality of vertices of a contour of an object or of a design;

generation means generating a predetermined number of division points on each of sides connecting the plurality of vertices to each other, so as to divide each side of at least one pair of two opposing sides into unequal parts, wherein the division points on each side of at least one pair of two opposing sides are generated such that the closer to one end of each side of the two opposing sides, the denser the division points;

determining a plurality of sample points on the basis of straight lines connecting the division points on the two opposing sides to one another; and

determining, on the basis of pixel values of the sample points, whether or not an object or design is displayed in an area surrounded by the plurality of vertices in the image.

2. The non-transitory computer-readable storage medium according to claim 1, wherein the operations further comprise

dividing each side of at least one pair of two opposing sides into unequal parts in accordance with a ratio of:

a distance between one end of one side of the two opposing sides and a corresponding one end of the other side, to a distance between the other ends.

3. The non-transitory computer-readable storage medium according to claim 1, the operations further comprise:

determining whether or not, among the plurality of sides connecting the plurality of vertices to each other, given two opposing sides are parallel to each other;

generating, when it is determined that any two opposing sides are parallel to each other, the division points so as to divide each side of the two opposing sides into equal parts; and

generating, when it is determining that any two opposing sides are not parallel to each other, the division points so as to divide each side of the two opposing sides into unequal parts.

4. The non-transitory computer-readable storage medium according to claim 1, wherein the operations further comprising

calculating a first vanishing point, the first vanishing point being an intersection of straight lines extending from a first side and a second side, respectively, the first side and the second side opposing each other;

calculating a second vanishing point, the second vanishing point being an intersection of straight lines extending from a third side and a fourth side, respectively, the third side and the fourth side opposing each other;

calculating a second straight line, the second straight line being parallel to a first straight line connecting the first vanishing point to the second vanishing point, the second straight line passing through the vertex furthest from the first straight line;

generating, on a line segment connecting an intersection of the straight line passing through the first side and the second straight line to an intersection of the straight line passing through the second side and the second straight line, a plurality of first equal division points so as to divide the line segment into equal parts;

generating, on a line segment connecting an intersection of the straight line passing through the third side and the second straight line to an intersection of the straight line passing through the fourth side and the second straight line, a plurality of second equal division points so as to divide the line segment into equal parts;

generating, on the basis of straight lines connecting the first vanishing point to the first equal division points, the division points on each of the third side and the fourth side so as to divide each of the third side and the fourth side into unequal parts; and

generating, on the basis of straight lines connecting the second vanishing point to the second equal division points, the division points on each of the first side and the second side so as to divide each of the first side and the second side into unequal parts.

5. The non-transitory computer-readable storage medium according to claim 1, wherein the operations further comprise

repeatedly acquiring images, and

detecting the plurality of vertices from the repeatedly acquired images.

6. The non-transitory computer-readable storage medium according to claim 1, wherein the operation further comprise

determining, by comparing the pixel values of the sample points with pixel values corresponding to the object or design that are stored in advance, whether or not the object or design is displayed in the area surrounded by the plurality of vertices in the image.

7. An image recognition apparatus comprising:

memory; and

at least one processing circuitry communicatively coupled to the memory, and configured to;

receive an image;

detect from the image a plurality of vertices of a contour of an object or of a design;

generate a predetermined number of division points on each of sides connecting the plurality of vertices to each other, so as to divide each side of at least one pair of two opposing sides into unequal parts, wherein the division points on each side of at least one pair of two sides are generated such that the closer to one end of each side of the two opposing sides, the denser the division points;

determine a plurality of sample points on the basis of straight lines connecting the division points on the two opposing sides to one another; and

determine, for on the basis of pixel values of the sample points, whether or not a object or design is displayed in an area surrounded by the plurality of vertices in the image.

8. The image recognition apparatus according to claim 7, wherein the at least one processor is further configured to:

divide each side of at least one pair of two opposing sides into unequal parts in accordance with a ratio of;

a distance between one end of one side of the two opposing sides and a corresponding one end of the other side, to a distance between the other ends.

9. The image recognition apparatus according to claim 7, wherein the at least one processor is further configured to:

determine whether or not, among the plurality of sides connecting the plurality of vertices to each other, given two opposing sides are parallel to each other;

when it has been determined that any two opposing sides are parallel to each other, generating the division points so as to divide each side of the two opposing sides into equal parts; and

when it has been determined that any two opposing sides are not parallel to each other, generating the division points so as to divide each side of the two opposing sides into unequal parts.

10. The image recognition apparatus according to claim 7, wherein the at least one processor is further configured to:

calculate a first vanishing point, the first vanishing point being an intersection of straight lines extending from a first side and a second side, respectively, the first side and the second side opposing each other;

calculate a second vanishing point, the second vanishing point being an intersection of straight lines extending from a third side and a fourth side, respectively, the third side and the fourth side opposing each other;

calculate a second straight line, the second straight line being parallel to a first straight line connecting the first vanishing point to the second vanishing point, the second straight line passing through the vertex furthest from the first straight line;

generate, on a line segment connecting an intersection of the straight line passing through the first side and the second straight line to an intersection of the straight line passing through the second side and the second straight line, a plurality of first equal division points so as to divide the line segment into equal parts;

generate, on a line segment connecting an intersection of the straight line passing through the third side and the second straight line to an intersection of the straight line passing through the fourth side and the second straight line, a plurality of second equal division points so as to divide the line segment into equal parts;

generate, on the basis of straight lines connecting the first vanishing point to the first equal division points, generating the division points on each of the third side and the fourth side so as to divide each of the third side and the fourth side into unequal parts; and

generate, on the basis of straight lines connecting the second vanishing point to the second equal division points, the division points on each of the first side and the second side so as to divide each of the first side and the second side into unequal parts.

11. An image recognition method comprising:

receiving an image;

detecting from the image a plurality of vertices of a contour of an object or of a design;

generating a predetermined number of division points on each of sides connecting the plurality of vertices to each other, so as to divide each side of at least one pair of two opposing sides into unequal parts, wherein the division points on each side of at least one pair of two opposing sides are generated such that the closer to one end of each side of the two opposing sides, the denser the division points;

determining a plurality of sample points on the basis of straight lines connecting the division points on the two opposing sides to one another; and

determine, on the basis of pixel values of the sample points, whether or not a object or design is displayed in an area surrounded by the plurality of vertices in the image.

12. The image recognition method according to claim 11, further comprising:

dividing each side of at least one pair of two opposing sides into unequal parts in accordance with a ratio of;

a distance between one end of one side of the two opposing sides and a corresponding one end of the other side, to a distance between the other ends.

13. The image recognition method according to claim 11, further comprising:

determining whether or not, among the plurality of sides connecting the plurality of vertices to each other, given two opposing sides are parallel to each other;

when it has been determined that any two opposing sides are parallel to each other, generating the division points so as to divide each side of the two opposing sides into equal parts; and

when it has been determined that any two opposing sides are not parallel to each other, generating the division points so as to divide each side of the two opposing sides into unequal parts.

14. The image recognition method according to claim 11, further comprising:

calculating a first vanishing point, the first vanishing point being an intersection of straight lines extending from a first side and a second side, respectively, the first side and the second side opposing each other;

calculating a second vanishing point, the second vanishing point being an intersection of straight lines extending from a third side and a fourth side, respectively, the third side and the fourth side opposing each other;

calculating a second straight line, the second straight line being parallel to a first straight line connecting the first vanishing point to the second vanishing point, the second straight line passing through the vertex furthest from the first straight line;

generating, on a line segment connecting an intersection of the straight line passing through the first side and the second straight line to an intersection of the straight line passing through the second side and the second straight line, a plurality of first equal division points so as to divide the line segment into equal parts;

generating, on a line segment connecting an intersection of the straight line passing through the third side and the second straight line to an intersection of the straight line passing through the fourth side and the second straight line, a plurality of second equal division points so as to divide the line segment into equal parts;

generating, on the basis of straight lines connecting the first vanishing point to the first equal division points, generating the division points on each of the third side and the fourth side so as to divide each of the third side and the fourth side into unequal parts; and

generating, on the basis of straight lines connecting the second vanishing point to the second equal division points, the division points on each of the first side and the second side so as to divide each of the first side and the second side into unequal parts.

15. An image recognition system comprising:

an image capture device for capturing an image; and

at least one processor communicatively coupled to the image capture device and configured to;

receive the image;

detect from the image a plurality of vertices of a contour of an object or of a design;

generate a predetermined number of division points on each of sides connecting the plurality of vertices to each other, so as to divide each side of at least one pair of two opposing sides into unequal parts, wherein the division points on each side of at least one pair of two opposing sides are generated such that the closer to one end of each side of the two opposing sides, the denser the division points;

determine a plurality of sample points on the basis of straight lines connecting the division points on the two opposing sides to one another; and

determine, on the basis of pixel values of the sample points, whether or not a object or design is displayed in an area surrounded by the plurality of vertices in the image.

16. An image recognition system including an image recognition apparatus and a marker in which a design is drawn, the image recognition apparatus comprising:

a capturing section for capturing the marker; and

at least one processor communicatively coupled to the image capture device and configured to;

receive an image from the capturing section;

detect from the image a plurality of vertices of a contour of the marker or of the design;

generate a predetermined number of division points on each of sides connecting the plurality of vertices to each other, so as to divide each side of at least one pair of two opposing sides into unequal parts, wherein the division points on each side of at least one pair of two opposing sides are generated such that the closer to one end of each side of the two opposing sides, the denser the division points;

determine a plurality of sample points on the basis of straight lines connecting the division points on the two opposing sides to one another; and

determine, on the basis of pixel values of the sample points, whether or not a design is displayed in an area surrounded by the plurality of vertices in the image.

17. The image recognition system according to claim 16, wherein the at least one processor is further configured to divide each side of at least one pair of two opposing sides into unequal parts in accordance with a ratio of:

a distance between one end of one side of the two opposing sides and a corresponding one end of the other side, to a distance between the other ends.

18. The image recognition system according to claim 16, wherein the at least one processor is further configured to:

determine whether or not, among the plurality of sides connecting the plurality of vertices to each other, given two opposing sides are parallel to each other;

when it has been determined that any two opposing sides are parallel to each other, generate the division points so as to divide each side of the two opposing sides into equal parts; and

when it has been determined that any two opposing sides are not parallel to each other, generate the division points so as to divide each side of the two opposing sides into unequal parts.

19. The image recognition system according to claim 16, wherein the at least one processor is further configured to:

calculate a first vanishing point, the first vanishing point being an intersection of straight lines extending from a first side and a second side, respectively, the first side and the second side opposing each other;

calculate a second vanishing point, the second vanishing point being an intersection of straight lines extending from a third side and a fourth side, respectively, the third side and the fourth side opposing each other;

calculate a second straight line, the second straight line being parallel to a first straight line connecting the first vanishing point to the second vanishing point, the second straight line passing through the vertex furthest from the first straight line;

calculate, on a line segment connecting an intersection of the straight line passing through the first side and the second straight line to an intersection of the straight line passing through the second side and the second straight line, a plurality of first equal division points so as to divide the line segment into equal parts;

calculate, on a line segment connecting an intersection of the straight line passing through the third side and the second straight line to an intersection of the straight line passing through the fourth side and the second straight line, a plurality of second equal division points so as to divide the line segment into equal parts;

calculate, on the basis of straight lines connecting the first vanishing point to the first equal division points, generating the division points on each of the third side and the fourth side so as to divide each of the third side and the fourth side into unequal parts; and

calculate, on the basis of straight lines connecting the second vanishing point to the second equal division points, the division points on each of the first side and the second side so as to divide each of the first side and the second side into unequal parts.