Content identification and distribution

• US 8,814,048 B2
• Filed: 12/21/2012
• Issued: 08/26/2014
• Est. Priority Date: 12/23/2011
• Status: Active Grant

• Current Assignee: Zappar Ltd.
• Original Assignee: Zappar Ltd.
• Inventors: Taylor, Simon John
• Primary Examiner(s): JOHNSON, SONJI N
• Application Number:
  US13/725,086
• Publication Number:
  US 20130162676A1
• Time in Patent Office: 613 Days
• Field of Search: 235/462.09, 235/435, 345/633
• US Class Current: 235/462.09
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/75 involving models

Abstract

The invention provides an identifier system for computing identity information from image data. At least part of the image data is representative of an identifier. The identifier comprises a location element and encoded data associated with the location element. The identifier system comprises computer interpretable reference data corresponding to the identifier. The reference data is suitable for use in feature matching to determine a location and an orientation of the location element in the image data—thereby to locate the encoded data in the image data for subsequent decoding into the identity information The invention also provides a computer implemented method of presenting an augmented reality view of a physical article using the identifier system.

Claims

1. A computer implemented method of presenting an augmented reality view of a physical article, comprising:

receiving image data representative of a physical article for which augmented reality content is available;

determining an identity of the augmented reality content;

sending a request for the augmented reality content, the request including comprising an indication of the determined identity;

receiving a response to the request, the response comprising the augmented reality content; and

presenting an augmented reality view of the physical article based on the image data and the augmented reality content, wherein

the physical article comprises an identifier,

the image data comprises data representative of the identifier, and

the determining of the identity of the augmented reality content comprises computing identity information from the data representative of the identifier,

the identifier comprises a location element and encoded data associated with the location element, and

the computing of the identity information comprises;

receiving the image data,

receiving computer interpretable reference data corresponding to the identifier,

using feature matching based on the reference data to determine a location and a scale and orientation of the location element in the received image data,

using the determined location, scale and orientation of the location element to locate the encoded data for subsequent decoding into identity information, and

decoding the resulting encoded data into the identity information.

2. The method according to claim 1, wherein the computing of the identity information further comprises:

detecting candidate features in the image data; and

identifying, using local context information, one or more specific features from the detected candidate features to determine the location and orientation of the location element.

3. The method according to claim 1, wherein the reference data comprises a reference model of at least part of the location element, and the feature matching comprises obtaining a model of a part of the received image data and comparing the model with the reference model to determine whether the model matches the reference model.

4. The method according to claim 3, wherein the reference data comprises a further reference model of the at least part of the location element, each of the reference models corresponding to a different combination of scale and orientation of the location element, and the feature matching comprises comparing the model of a part of the received image data with the further reference model to determine whether model matches the further reference model.

5. The method according to claim 3, wherein the feature matching comprises:

disregarding a potential match in response to determining that the model of a part of the received image data does not match the reference model or the further reference model.

6. The method according to claim 3, further comprising:

deriving the location of the location element from the location of the part of the received image data when the model thereof was determined to match the reference model, or the further reference model, and/or when the part of the received image data was determined to match the reference image or the further reference image.

7. The method according to claim 3, further comprising:

deriving the scale and orientation of the location element from the scale and orientation associated with the reference model, or the further reference model, when the obtained model of the part of the received image data was determined to match the reference model, or the further reference model, respectively.

8. The method according to claim 7, wherein the determining of the scale and orientation of the location element comprises:

using an optimization algorithm to change the derived scale and/or orientation to fit the location element of the received image data onto a matching reference image, the matching reference image being whichever one of the reference images was determined to match the part of the received image data and/or the reference image corresponding to whichever one of the reference models was determined to match the model of the part of the received image data.

9. The method according to claim 8, wherein the reference data comprises a sparse sample of the reference image, or respective sparse samples of the reference images, each sparse sample being suitable for use with a corresponding sparse sample of the location element of the image data.

10. The method according to claim 1, wherein the reference data comprises a reference image of the location element, and the feature matching comprises comparing a part of the received image data with image data from the reference image to determine whether the part of the received image data matches the reference image.

11. The method according to claim 10, wherein the reference data comprises a further reference image of the location element, each of the reference images corresponding to a different combination of scale and orientation of the location element, and the feature matching comprises comparing the part of the received image data with image data from the further reference image to determine whether the part of the received image data matches the further reference image.

12. The method according to claim 11, wherein the further reference image corresponds to the further reference model and the part of the received image data is compared with the image data from the further reference image when the potential match between the part of the received image data and the further reference model has not already been disregarded.

13. The method according to claim 10, wherein the reference image corresponds to the reference model and the part of the received image data is compared with the image data from the reference image when the potential match between the part of the received image data and the reference model has not already been disregarded.

14. The method according to claim 10, further comprising:

deriving the scale and orientation of the location element from the scale and orientation associated with the reference image, or the further reference image, when the part of the received image data was determined to match the reference image, or the further reference image, respectively.

15. The method according to claim 1, wherein the reference data comprises decoding information that specifies a location relative to the location element from which the encoded data can be obtained.

16. The method according to claim 15, wherein the decoding information specifies one or more decoding reference values for comparison with the encoded data in order to decode the encoded data into the identity information.

17. An augmented reality content distribution system configured to present an augmented reality view of a physical article, comprising:

a client device comprising a computer processor operatively coupled to a camera and a display;

a physical article for which augmented reality content is available, the physical article comprising an identifier, the identifier comprising a location element and encoded data associated with the location element; and

a server configured to store augmented reality content associated with the physical article,

wherein the computer processor is configured to;

receive image data from the camera, the image data being representative of the physical article and comprising data representative of the identifier,

determine an identity of the augmented reality content by computing identity information from the data representative of the identifier,

send a request to the server for the augmented reality content, the request comprising an indication of the determined identity,

receive a response to the request, the response comprising the stored augmented reality content, and

present an augmented reality view of the physical article on the display, based on the image data and the augmented reality content, and

the computing of the identity information comprises;

receiving the image data,

receiving computer interpretable reference data corresponding to the identifier,

using feature matching based on the reference data to determine a location and a scale and orientation of the location element in the received image data,

using the determined location, scale and orientation of the location element to locate the encoded data for subsequent decoding into identity information, and

decoding the resulting encoded data into the identity information.