Computer-readable storage medium, image recognition apparatus, image recognition system, and image recognition method

• US 8,705,869 B2
• Filed: 08/02/2011
• Issued: 04/22/2014
• Est. Priority Date: 02/24/2011
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Osako, Satoru
• Primary Examiner(s): CHANG, JON CARLTON
• Application Number:
  US13/196,299
• Publication Number:
  US 20120219226A1
• Time in Patent Office: 994 Days
• Field of Search: None
• US Class Current: 382/199
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/469 Contour-based spatial representations, e.g. vector-coding
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 10/752 Contour matching

Abstract

First, it is determined whether or not a contour or vertices have been detected from a certain image. Then, when it is determined that a contour or vertices have been detected from the certain image, a pattern matching process is performed on, in the image, an area surrounded by the contour or the vertices detected from the image. Then, when it is determined that a predetermined object or a predetermined design is displayed in the area, detection position information indicating a position related to the area is stored in a storage area. On the other hand, when it is determined that a contour or vertices have not been detected from the certain image, a pattern matching process is performed on the image on the basis of the detection position information stored in the storage area.

Claims

1. A non-transitory computer-readable storage medium having stored thereon an image recognition program which, when executed by a computer of an information processing apparatus, causes the computer to perform operations comprising:

sequentially acquiring images;

sequentially detecting a contour or vertices from the sequentially acquired images;

determining whether or not a contour or vertices are detected from an image of the sequentially acquired images;

when the determining determines that a contour or vertices are detected from the image, performing a pattern matching process on an area, in the image, surrounded by the contour or the vertices detected from the image, to determine whether or not an object or a design is present in the area;

when the pattern matching determines that the object or the design is present in the area, storing, in a storage area, detection position information indicating a position related to the area; and

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of detection position information previously stored in the storage area, to determine whether or not, in the image, the object or the design is present in an area corresponding to the detection position information previously stored in the storage area.

2. The non-transitory computer-readable storage medium according to claim 1, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of the detection position information previously stored in the storage area on the basis of another image, from the sequentially acquired images, acquired immediately before the image.

3. The non-transitory computer-readable storage medium according to claim 1, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image by, in addition to the detection position information previously stored in the storage area, shifting a position indicated by the detection position information previously stored in the storage area.

4. The non-transitory computer-readable storage medium according to claim 1, the image recognition program further causing the computer to perform operations comprising:

on the basis of the detection position information stored in the storage area, setting a virtual camera placed in a virtual space; and

displaying on a display device a virtual space image obtained by capturing the virtual space with the virtual camera.

5. An image recognition apparatus comprising:

at least one processor configured to perform operations comprising;

sequentially acquiring images;

sequentially detecting a contour or vertices from the sequentially acquired images;

determining whether or not a contour or vertices are detected from an image of the sequentially acquired images;

when the determining determines that a contour or vertices are detected from the image, performing a pattern matching process on an area, in the image, surrounded by the contour or the vertices detected from the image, to determine whether or not an object or a design is present in the area;

when the pattern matching determines that the object or the design is present in the area, storing, in a storage area, detection position information indicating a position related to the area; and

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of detection position information previously stored in the storage area, to determine whether or not, in the image, the object or the design is present in an area corresponding to the detection position information previously stored in the storage area.

9. The image recognition apparatus according to claim 5, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of the detection position information previously stored in the storage area on the basis of another image, from the sequentially acquired images, acquired immediately before the image.

10. The image recognition apparatus according to claim 5, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image by, in addition to the detection position information previously stored in the storage area, shifting a position indicated by the detection position information previously stored in the storage area.

11. The image recognition apparatus according to claim 5, wherein the at least one processor is further configured to:

on the basis of the detection position information stored in the storage area, set a virtual camera placed in a virtual space; and

display on a display device a virtual space image obtained by capturing the virtual space with the virtual camera.

6. An image recognition method comprising:

sequentially acquiring images;

sequentially detecting a contour or vertices from the sequentially acquired images;

determining whether or not a contour or vertices are detected from an image of the sequentially detected images;

when the determining determines that a contour or vertices are detected from the image, performing a pattern matching process on an area, in the image, surrounded by the contour or the vertices detected from the image, to determine whether or not an object or a design is present in the area;

when the pattern matching determines that the object or the design is present in the area, storing, in a storage area, detection position information indicating a position related to the area; and

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of detection position information previously stored in the storage area, to determine whether or not, in the image, the object or the design is present in an area corresponding to the detection position information previously stored in the storage area.

12. The image recognition method according to claim 6, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of the detection position information previously stored in the storage area on the basis of another image, from the sequentially acquired images, acquired immediately before the image.

13. The image recognition method according to claim 6, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image by, in addition to the detection position information previously stored in the storage area, shifting a position indicated by the detection position information previously stored in the storage area.

14. The image recognition method according to claim 6, further comprising:

on the basis of the detection position information stored in the storage area, setting a virtual camera placed in a virtual space; and

displaying on a display device a virtual space image obtained by capturing the virtual space with the virtual camera.

7. An image recognition system comprising:

a camera; and

at least one processor configured to perform operations comprising;

sequentially acquiring images from the camera;

sequentially detecting a contour or vertices from the sequentially acquired images;

determining whether or not a contour or vertices are detected from an image of the sequentially acquired images;

when the determining determines that a contour or vertices are detected from the image, performing a pattern matching process on an area, in the image, surrounded by the contour or the vertices detected from the image, to determine whether or not an object or a design is present in the area;

when the pattern matching determines that the object or the design is present in the area, storing, in a storage area, detection position information indicating a position related to the area; and

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of detection position information previously stored in the storage area, to determine whether or not, in the image, the object or the design is present in an area corresponding to the detection position information previously stored in the storage area.

15. The image recognition system according to claim 7, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of the detection position information previously stored in the storage area on the basis of another image, from the sequentially acquired images, acquired immediately before the image.

16. The image recognition system according to claim 7, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image by, in addition to the detection position information previously stored in the storage area, shifting a position indicated by the detection position information previously stored in the storage area.

17. The image recognition system according to claim 7, wherein the at least one processor is further configured to:

on the basis of the detection position information stored in the storage area, set a virtual camera placed in a virtual space; and

display on a display device a virtual space image obtained by capturing the virtual space with the virtual camera.

8. An image recognition system including an image recognition apparatus and a marker in which a design is drawn, the image recognition apparatus comprising:

a capturing section for capturing the marker; and

at least one processor configured to perform operations comprising;

sequentially acquiring images from the capturing section;

sequentially detecting a contour or vertices of the marker or of the design from the sequentially acquired images;

determining whether or not the contour or the vertices are detected from an image of the sequentially acquired images;

when the determining determines that the contour or the vertices are detected from the image, performing a pattern matching process on an area, in the image, surrounded by the contour or the vertices, to determine whether or not the marker or the design is present in the area;

when the pattern matching determines that the marker or the design is present in the area, storing, in a storage area, detection position information indicating a position related to the area; and

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of detection position information previously stored in the storage area, to determine whether or not, in the image, the marker or the design is present in an area corresponding to the detection position information previously stored in the storage area.

18. The image recognition system according to claim 8, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image on the basis of the detection position information previously stored in the storage area on the basis of another image, from the sequentially acquired images, acquired immediately before the image.

19. The image recognition system according to claim 8, wherein

when the determining determines that no contour or vertices are detected from the image, performing a pattern matching process on the image by, in addition to the detection position information previously stored in the storage area, shifting a position indicated by the detection position information previously stored in the storage area.

20. The image recognition system according to claim 8, wherein the at least one processor is further configured to:

on the basis of the detection position information stored in the storage area, set a virtual camera placed in a virtual space; and

display on a display device a virtual space image obtained by capturing the virtual space with the virtual camera.