System and method for toy recognition
First Claim
1. A system for recognizing real-world toy objects from one or more images, the system comprising a model creation system and a recognition system:
- wherein the model creation system comprises one or more processors and a training database and wherein the training database is configured to store annotated digital images, each annotated digital image depicting a real-world toy object and being annotated with an object identifier identifying the depicted real-world toy object;
wherein the model creation system is configured to;
train a convolutional classification model based on at least a subset of the annotated digital images to predict a matching object identifier when the convolutional classification model is presented with a digital image of a real-world toy object, wherein the recognition system comprises an image capturing device and one or more processors and wherein the recognition system is configured to;
capture at least one image of a real-world toy object;
use the trained convolutional classification model to predict a matching object identifier from the captured image;
obtain, based on the predicted object identifier, stored information including connectivity information indicative of how the real-world toy object can be detachably connected to toy construction elements of a toy construction system; and
insert a virtual toy object corresponding to the recognized physical toy object into a virtual world as a virtual construction element attached to a virtual construction model.
1 Assignment
0 Petitions
Accused Products
Abstract
System and method for automatic computer aided optical recognition of toys, for example, construction toy elements, recognition of those elements on digital images and associating the elements with existing information is presented. The method and system may recognize toy elements of various sizes invariant of toy element distance from the image acquiring device for example camera, invariant of rotation of the toy element, invariant of angle of the camera, invariant of background, invariant of illumination and without the need of predefined region where a toy element should be placed. The system and method may detect more than one toy element on the image and identify them. The system is configured to learn to recognize and detect any number of various toy elements by training a deep convolutional neural network.
40 Citations
37 Claims
-
1. A system for recognizing real-world toy objects from one or more images, the system comprising a model creation system and a recognition system:
-
wherein the model creation system comprises one or more processors and a training database and wherein the training database is configured to store annotated digital images, each annotated digital image depicting a real-world toy object and being annotated with an object identifier identifying the depicted real-world toy object; wherein the model creation system is configured to; train a convolutional classification model based on at least a subset of the annotated digital images to predict a matching object identifier when the convolutional classification model is presented with a digital image of a real-world toy object, wherein the recognition system comprises an image capturing device and one or more processors and wherein the recognition system is configured to; capture at least one image of a real-world toy object; use the trained convolutional classification model to predict a matching object identifier from the captured image; obtain, based on the predicted object identifier, stored information including connectivity information indicative of how the real-world toy object can be detachably connected to toy construction elements of a toy construction system; and insert a virtual toy object corresponding to the recognized physical toy object into a virtual world as a virtual construction element attached to a virtual construction model. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25)
-
-
26. A computer implemented method for creating a recognition model for use in a system for detecting and recognizing real-world toy objects from one or more captured images, the method comprising:
-
receiving image data indicative of digital images depicting real-world toy objects; receiving image annotations indicative of an object identifier of each of the depicted real-world toy objects; storing annotated digital images in the training database, each annotated digital image depicting a real-world object and being annotated with an object identifier associated with the depicted real-world toy object; training a convolutional classification model based on at least a subset of the annotated digital images to predict a matching object identifier when the convolutional classification model is presented with a digital image of a real-world toy object; and storing information including connectivity information indicative of how the real-world toy object can be detachably connected to toy construction elements of a toy construction system. - View Dependent Claims (28, 29, 30)
-
-
27. A computer implemented method for recognizing real-world toy objects from one or more captured images, the method comprising:
-
receiving a captured image of a real-world toy object; using a trained convolutional classification model to predict a matching object identifier; obtaining, based on the predicted object identifier, stored information including connectivity information indicative of how the real-world toy object can be detachably connected to toy construction elements of a toy construction system; and inserting a virtual toy object corresponding to the recognised physical toy object into a virtual world as a virtual construction element attached to a virtual construction model.
-
-
31. A game system, comprising:
-
one or more real-world toy objects; a database of real-world toy objects available for inclusion as virtual objects in a virtual environment, each virtual object being associated with an object identifier of a corresponding real-world toy object and with accompanying information about one or more virtual properties of the virtual object;
an image capturing device; anda data processing system configured to; receive a captured image of a real-world toy object in a real-world scene; use a trained convolutional classification model to predict a matching object identifier; add one or more virtual objects associated with the matched object identifier to a virtual environment; and affect a gameplay experience based on the accompanying information; wherein the data processing system is further configured to; use the trained convolutional classification model to create a list of candidate object identifiers and associated likelihood scores indicative of a likelihood of the respective candidate object identifiers matching the real-world toy object; for each of the candidate object identifiers and based on the captured image, estimate a placement of the corresponding virtual object in a virtual scene corresponding to the real-world scene; compute a respective correspondence score for each candidate object identifier based on a correlation of the estimated placement with at least the captured image; and select a candidate object identifier and a corresponding placement based at least on the computed correspondence scores. - View Dependent Claims (32, 33, 34, 35, 36, 37)
-
Specification