DETERMINING POSE FOR USE WITH DIGITAL WATERMARKING, FINGERPRINTING AND AUGMENTED REALITY

• US 20140119593A1
• Filed: 03/07/2013
• Published: 05/01/2014
• Est. Priority Date: 10/29/2012
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Filler, Tomas
• Granted Patent: US 9,684,941 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06T 1/0064 Geometric transfor invariant watermarking, e.g. affine transform invariant
• CPC Class Codes: G06T 2201/0065 Extraction of an embedded watermark; Reliable detection
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Image recognition and augmented reality experiences utilize auxiliary data extracted from an image or video, or image fingerprints, or a combination of both. One claim recites a method comprising: receiving a plurality of imagery frames captured by a device sensor; identifying a plurality of key points in each of the plurality of imagery frames; resolving image pose by utilizing relative relationships of the key points between different frames of the plurality of frames; modifying imagery of at least one of the plurality of frames based on resolved image pose; and deriving a fingerprint from modified imagery. Of course other claims and combinations are provided as well.

Claims

1. A method comprising:

receiving image data captured from a device sensor;

modifying received image data to compensate for distortion caused by a capture positioning of the device sensor relative to an imaged subject;

analyzing modified imagery to detect a digital watermark therefrom;

extracting a digital fingerprint from the modified imagery, the digital fingerprint corresponding to an image area where the digital watermark was detected;

determining a relative spatial position of the image area based on the digital fingerprint.

2. The method of claim 1 in which said determining compares the digital fingerprint to a plurality of digital fingerprints, with each of the plurality of digital fingerprint corresponding to a different spatial area of the received image data.

3. The method of claim 2 in which the plurality of digital fingerprints are stored in a remote server.

4. The method of claim 1 in which said extracting utilizes a linear projection.

5. The method of claim 1 further comprising:

providing the relative spatial position of the image area to an augmented reality (AR) system.

6. The method of claim 5 in which the AR system overlays graphics or video according to the relative spatial position of the image area.

7. A method comprising:

receiving a plurality of imagery frames captured by a device sensor;

identifying a plurality of key points in each of the plurality of imagery frames;

resolving image pose by utilizing relative relationships of the key points between different frames of the plurality of frames;

modifying imagery of at least one of the plurality of frames based on resolved image pose, said modifying yielding modified imagery;

detecting a digital watermark from the modified imagery.

8. The method of claim 7 in which said resolving utilizes pose homographies associated with the key points.

9. The method of claim 8 in which said resolving further minimizes a cost function associated with the pose homographies.

10. The method of claim 7 in which the digital watermark comprises orientation attributes, said method further comprising providing orientation information to an Augmented Reality (AR) system, the orientation information obtained from analysis of the orientation attributes.

11. The method of claim 10 in which the AR system overlays graphics or video on a display screen in an area corresponding to a spatial area in which the digital watermark was detected

12. The method of claim 10 in which the AR system utilizes the orientation information to transform captured imagery for display on a display screen.

13. The method of claim 7 in which the digital watermark comprises a payload, in which at least a portion of the payload is used to initiate an announcement associated with an AR system.

14. The method of claim 7 in which said resolving utilizes pose homography associated with the plurality of key points.

15. A method comprising:

receiving a plurality of imagery frames captured by a device sensor;

identifying a plurality of key points in each of the plurality of imagery frames;

resolving image pose by utilizing relative relationships of the key points between different frames of the plurality of frames;

modifying imagery of at least one of the plurality of frames based on resolved image pose, said modifying yielding modified imagery;

deriving a fingerprint from the modified imagery.

16. The method of claim 15 in which said resolving utilizes pose homographies associated with the key points.

17. The method of claim 16 in which said resolving further minimizes a cost function associated with the pose homographies.

18. The method of claim 15 further comprising providing the fingerprint to an augmented reality (AR) to help facilitate display of graphics or video on a display screen corresponding to a spatial area of modified imagery from which the fingerprint was derived from.

19. A method comprising:

receiving a plurality of imagery frames captured by a device sensor;

estimating homography between two imagery frames of the plurality of imagery frame to resolve image pose associated with the plurality of imagery frames;

modifying imagery of at least one of the plurality of frames based on resolved image pose, said modifying yielding modified imagery;

detecting a digital watermark from the modified imagery.

20. The method of claim 19 in which said estimating homography minimizes a cost function.

21. The method of claim 19 in which the digital watermark comprises orientation attributes, said method further comprising providing the orientation information to an Augmented Reality (AR) system, the orientation information obtained from analysis of the orientation attributes.

22. The method of claim 21 in which the AR system overlays graphics or video on a display screen in an area corresponding to a spatial area in which the digital watermark was detected.

23. The method of claim 21 in which the AR system utilizes the orientation information to transform captured imagery for display on a display screen.

24. A method comprising:

receiving imagery captured by a device sensor;

analyzing the imagery to decode a digital watermark therefrom, in which the digital watermark comprises orientation attributes;

determining orientation information from the orientation attributes, the orientation information being associated with a capture position of the device sensor relative to an imaged subject;

utilizing the orientation information as pose input for use by an augmented reality (AR) system; and

refining the pose input with image homography.