System and method for reading graphically-encoded identifiers from physical trading cards through image-based template matching

• US 10,179,289 B2
• Filed: 06/21/2016
• Issued: 01/15/2019
• Est. Priority Date: 06/21/2016
• Status: Active Grant

• Current Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Original Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Inventors: Stirling, Spencer D.
• Primary Examiner(s): YEN, JASON TAHAI
• Application Number:
  US15/188,570
• Publication Number:
  US 20170361233A1
• Time in Patent Office: 938 Days
• Field of Search: None
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/70 Game security or game management aspects
• CPC Class Codes: A63F 2300/575 for trading virtual items
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

A system and method is provided for reading graphically-encoded identifiers from physical trading cards through image-based template matching in order to incorporate game items from the identified trading cards in a video game.

Claims

1. A computer-implemented method of identifying a physical trading card that includes a card identification region comprising a plurality of graphical encoding indicators graphically integrated within a design of the physical trading card, wherein the plurality of graphical encoding indicators encode an identity of the physical trading card, the method being implemented in a computer system having one or more physical processors programmed with computer program instructions that, when executed by the one or more physical processors, cause the computer system to perform the method, the method comprising:

obtaining, by the computer system, at least a first image frame that includes an image of the physical trading card;

determining, by the computer system, a first location in the first image frame that corresponds to at least a portion of the physical trading card imaged in the first image frame;

determining, by the computer system, a second location in the first image frame that corresponds to the card identification region based on the first location;

obtaining, by the computer system, a plurality of graphical encoding templates that each include a graphical image corresponding to an encoded value;

comparing, by the computer system, each of the plurality of graphical encoding templates to the first image frame at the second location; and

determining, by the computer system, an identification of the physical trading card based on the comparing.

2. The method of claim 1, wherein the identification of the trading card comprises an N-bit identification value comprising N binary values, N being an integer, and wherein the card identification region comprises N slots that each correspond to a graphical encoding indicator that graphically encodes either the first binary value or the second binary value and the plurality of graphical encoding templates comprise a first binary encoding template that includes a first graphical image representing a first binary value and a second binary encoding template that includes a second graphical image representing a second binary value.

3. The method of claim 2, wherein comparing each of the plurality of graphical encoding templates to the first image frame at the second location comprises:

comparing, by the computer system, the first binary encoding template to the first image frame at the second location along the N slots;

comparing, by the computer system, the second binary encoding template to the first image frame at the second location along the N slots; and

for each slot, determining, by the computer system;

(1) a first match score for the first binary encoding template based on comparing the first binary encoding template to the first image frame, (2) a second match score for the second binary encoding template based on comparing the second binary encoding template to the first image frame, and (3) the first binary value, the second binary value, or an inconclusive value for the slot based on the first match score and the second match score, wherein the identification of the physical trading card is determined based on the first binary value, the second binary value, or the inconclusive value of each of the N slots.

4. The method of claim 3, wherein comparing the first binary encoding template to the first image frame at the second location comprises:

determining, by the computer system, a window size of the first image frame with which to compare the first binary encoding template; and

performing, by the computer system, template matching based on the window size.

5. The method of claim 2, further comprising:

obtaining, by the computer system, a bias correction parameter that accounts for a bias relating to a read of the first binary value or the second binary value; and

applying, by the computer system, the bias correction parameter to adjust a read of the first binary value or the second binary value, wherein application of the bias correction parameter corrects the bias relating to the read of the first binary value or the second binary value.

6. The method of claim 2, further comprising:

obtaining, by the computer system, at least a second image frame that includes a second image of the physical trading card;

determining, by the computer system, a third location in the second image frame that corresponds to at least a portion of the physical trading card imaged in the second image frame;

determining, by the computer system, a fourth location in the second image frame that corresponds to the card identification region based on the third location;

comparing, by the computer system, each of the plurality of graphical encoding templates to the second image frame at the fourth location;

determining, by the computer system, a first set of N binary values based on comparing each of the plurality of graphical encoding templates to the first image frame at the second location; and

determining, by the computer system, a second set of N binary values based on comparing each of the plurality of graphical encoding templates to the second image frame at the fourth location, wherein the identification of the physical trading card is based further on the first set of N binary values and the second set of N binary values.

7. The method of claim 1, wherein obtaining the first image frame comprises:

obtaining, by the computer system, an image stream of the trading card;

providing, by the computer system, the image stream as an input to a computer vision engine; and

receiving, by the computer system, as an output of the computer vision engine, a plurality of frames, each frame comprising an image of the trading card that accounts for a perspective of the trading card, the plurality of frames including the first image frame.

8. The method of claim 1, wherein determining the location of the portion of the physical trading card imaged in the first image frame comprises:

obtaining, by the computer system, one or more control point templates that each include an image of a portion of a known physical trading card, wherein each control point template is associated with a corresponding location on the known physical trading card to which the image of the portion relates;

determining, by the computer system, a match between at least a first control point template and the first image frame; and

determining, by the computer system, a location of at least a portion of the physical trading card imaged in the first image frame based on the match between the first control point template and the first image frame, wherein the location of the card identification region in the first image frame is based on the location of the portion of the physical trading card.

9. The method of claim 8, the method further comprising:

obtaining, by the computer system, at least a second image frame that includes a second image of the physical trading card;

determining, by the computer system, a number of matches between the one or more control point templates and the second image frame;

determining, by the computer system, that the number of matches does not exceed a predefined threshold number; and

discarding, by the computer system, the second image frame responsive to the determination that the number of matches does not exceed a predefined threshold number.

10. The method of claim 8, wherein obtaining the one or more control point templates comprises obtaining, by the computer system, the one or more control point templates from a database of control point templates that includes excerpts of images of physical trading cards already added to the database, the method further comprising:

obtaining, by the computer system, an excerpt of an image of a new physical trading card not already added to the database;

comparing, by the computer system, the excerpt of the image of the new physical trading card with the excerpts of the images of the physical trading cards already added to the database;

determining, by the computer system, whether the excerpt of the image of the new physical trading card already exists in the database; and

adding, by the computer system, the excerpt of the image of the new physical trading card to the database responsive to a determination that the excerpt of the image of the new physical trading card does not already exist in the database.

11. A system of identifying a physical trading card that includes a card identification region comprising a plurality of graphical encoding indicators graphically integrated within a design of the physical trading card, wherein the plurality of graphical encoding indicators encode an identity of the physical trading card, the system comprising:

a computer system comprising one or more physical processors programmed by computer program instructions that, when executed, cause the computer system to;

obtain at least a first image frame that includes an image of the physical trading card;

determine a first location in the first image frame that corresponds to at least a portion of the physical trading card imaged in the first image frame;

determine a second location in the first image frame that corresponds to the card identification region based on the first location;

obtain a plurality of graphical encoding templates that each include a graphical image corresponding to an encoded value;

compare each of the plurality of graphical encoding templates to the first image frame at the second location; and

determine an identification of the physical trading card based on the comparing.

12. The system of claim 11, wherein the identification of the trading card comprises an N-bit identification value comprising N binary values, N being an integer, and wherein the card identification region comprises N slots that each correspond to a graphical encoding indicator that graphically encodes either the first binary value or the second binary value and the plurality of graphical encoding templates comprise a first binary encoding template that includes a first graphical image representing a first binary value and a second binary encoding template that includes a second graphical image representing a second binary value.

13. The system of claim 12, wherein to compare each of the plurality of graphical encoding templates to the first image frame at the second location, the computer system is caused to:

compare the first binary encoding template to the first image frame at the second location along the N slots;

compare the second binary encoding template to the first image frame at the second location along the N slots; and

for each slot, determine;

(1) a first match score for the first binary encoding template based on comparing the first binary encoding template to the first image frame, (2) a second match score for the second binary encoding template based on comparing the second binary encoding template to the first image frame, and (3) the first binary value, the second binary value, or an inconclusive value for the slot based on the first match score and the second match score, wherein the identification of the physical trading card is determined based on the first binary value, the second binary value, or the inconclusive value of each of the N slots.

14. The system of claim 13, wherein to compare the first binary encoding template to the first image frame at the second location, the computer system is caused to:

determine a window size of the first image frame with which to compare the first binary encoding template; and

perform template matching based on the window size.

15. The system of claim 13, wherein the computer system is further caused to:

obtain a bias correction parameter that accounts for a bias relating to a read of the first binary value or the second binary value; and

apply the bias correction parameter to adjust a read of the first binary value or the second binary value, wherein application of the bias correction parameter corrects the bias relating to the read of the first binary value or the second binary value.

16. The system of claim 12, the method further comprising:

obtain at least a second image frame that includes a second image of the physical trading card;

determine a third location in the second image frame that corresponds to at least a portion of the physical trading card imaged in the second image frame;

determine a fourth location in the second image frame that corresponds to the card identification region based on the third location;

compare each of the plurality of graphical encoding templates to the second image frame at the fourth location;

determine a first set of N binary values based on comparing each of the plurality of graphical encoding templates to the first image frame at the second location; and

determine a second set of N binary values based on comparing each of the plurality of graphical encoding templates to the second image frame at the fourth location, wherein the identification of the physical trading card is based further on the first set of N binary values and the second set of N binary values.

17. The system of claim 11, wherein to obtain the first image frame, the computer system is caused to:

obtain an image stream of the trading card;

provide the image stream as an input to a computer vision engine; and

receive, as an output of the computer vision engine, a plurality of frames, each frame comprising an image of the trading card that accounts for a perspective of the trading card, the plurality of frames including the first image frame.

18. The system of claim 11, wherein to determine the location of the portion of the physical trading card imaged in the first image frame, the computer system is caused to:

obtain one or more control point templates that each include an image of a portion of a known physical trading card, wherein each control point template is associated with a corresponding location on the known physical trading card to which the image of the portion relates;

determine a match between at least a first control point template and the first image frame; and

determine a location of at least a portion of the physical trading card imaged in the first image frame based on the match between the first control point template and the first image frame, wherein the location of the card identification region in the first image frame is based on the location of the portion of the physical trading card.

19. The system of claim 18, wherein the computer system is further caused to:

obtain at least a second image frame that includes a second image of the physical trading card;

determine a number of matches between the one or more control point templates and the second image frame;

determine that the number of matches does not exceed a predefined threshold number; and

discard the second image frame responsive to the determination that the number of matches does not exceed a predefined threshold number.

20. The system of claim 18, wherein to obtain the one or more control point templates, the computer system is caused to obtain the one or more control point templates from a database of control point templates that includes excerpts of images of physical trading cards already added to the database, and wherein the computer system is further caused to:

obtain an excerpt of an image of a new physical trading card not already added to the database;

compare the excerpt of the image of the new physical trading card with the excerpts of the images of the physical trading cards already added to the database;

determine whether the excerpt of the image of the new physical trading card already exists in the database; and

add the excerpt of the image of the new physical trading card to the database responsive to a determination that the excerpt of the image of the new physical trading card does not already exist in the database.

21. A computer program product for identifying a physical trading card that includes a card identification region comprising a plurality of graphical encoding indicators graphically integrated within a design of the physical trading card, wherein the plurality of graphical encoding indicators encode an identity of the trading card, the computer program product comprising:

one or more tangible, non-transitory computer-readable storage devices;

program instructions, stored on at least one of the one or more tangible, non-transitory computer-readable tangible storage devices that, when executed, cause a computer system to;

obtain at least a first image frame that includes an image of the physical trading card;

determine a first location in the first image frame that corresponds to at least a portion of the physical trading card imaged in the first image frame;

determine a second location in the first image frame that corresponds to the card identification region based on the first location;

obtain a plurality of graphical encoding templates that each include a graphical image corresponding to an encoded value;

compare each of the plurality of graphical encoding templates to the first image frame at the second location; and

determine an identification of the physical trading card based on the comparing.