Salient point-based arrangements

• US 9,648,197 B2
• Filed: 04/08/2015
• Issued: 05/09/2017
• Est. Priority Date: 04/26/2011
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Aller, Joshua V.
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Potts, Ryan P
• Application Number:
  US14/681,814
• Publication Number:
  US 20160014297A1
• Time in Patent Office: 762 Days
• Field of Search: None
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06Q 30/0641 Shopping interfaces
• CPC Class Codes: G06T 1/005 Robust watermarking, e.g. average attack or collusion attack resistant
• CPC Class Codes: G06T 1/0092 Payload characteristic determination in a watermarking scheme, e.g. number of bits to be embedded
• CPC Class Codes: G06T 2201/0065 Extraction of an embedded watermark; Reliable detection
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/04 including near field communication means, e.g. RFID
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04N 1/32144 embedded in the image data, i.e. enclosed or integrated in the image, e.g. watermark, super-imposed logo or stamp
• CPC Class Codes: H04N 1/32203 Spatial or amplitude domain methods H04N1/32309 takes precedence
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document
• CPC Class Codes: H04N 2201/3269 of machine readable codes or marks, e.g. bar codes or glyphs

Abstract

A variety of methods and systems involving sensor-equipped portable devices, such as smartphones and tablet computers, are described. One particular embodiment decodes a digital watermark from imagery captured by the device and, by reference to watermark payload data, obtains salient point data corresponding to an object depicted in the imagery. Other embodiments obtain salient point data for an object through use of other technologies (e.g., NFC chips). The salient point data enables the device to interact with the object in a spatially-dependent manner. Many other features and arrangements are also detailed.

Claims

1. A system comprising:

a printed article including printed artwork, and a machine readable identifier printed with said artwork;

a computer including a processor and a memory, the memory containing instructions and data configuring the computer to receive a payload decoded from the printed article by a camera-equipped phone device, and to provide salient point data for said printed article in reply;

said salient point data comprising, for each of plural salient points on the printed article;

location information for said point on the article; and

orientation information for said point on the article, and/or information representing a feature vector associated with said location;

wherein said system enables the camera-equipped phone device to perform an augmented reality function in which (a) the device determines its spatial pose relative to the printed article, and (b) the device presents, on a screen thereof, both the camera-captured imagery, and also a graphical augmentation that is rotated, scaled or warped in accordance with said determined pose and overlaid atop imagery captured from said article, through use of said salient point data.

2. The system of claim 1 that additionally includes said camera-equipped phone device, said phone device including a processor and a memory, said phone device memory containing instructions configuring the phone device to receive said salient point data and to use the salient point data in performing said augmented reality function of (a) determining spatial pose of the phone device relative to the printed article, and (b) presenting on the screen of said device both the camera-captured imagery, and said graphical augmentation that is rotated, scaled or warped in accordance with said determined pose and overlaid atop imagery captured from said article, through use of said salient point data.

3. The system of claim 1 in which the machine readable identifier comprises a plural-bit payload steganographically conveyed in said artwork, wherein the payload is not evident to a human observer, but is decodable from analysis of imagery captured from the article by the camera-equipped phone device.

4. The system of claim 1 in which the machine readable identifier comprises a barcode.

5. A system for use with a printed article including printed artwork and a machine-readable identifier printed with said artwork, the system comprising:

a computer including a processor and a memory, the memory containing instructions and data configuring the computer to receive a payload decoded from the printed article by a camera-equipped phone device, and to provide salient point data for said printed article in reply;

said salient point data comprising, for each of plural salient points on the printed article;

location information for said point on the article; and

orientation information for said point on the article, and/or information representing a feature vector associated with said location;

wherein said system enables the camera-equipped phone device to perform an augmented reality function in which (a) the device determines its spatial pose relative to the printed article, and (b) the device presents, on a screen thereof, both the camera-captured imagery, and also a graphical augmentation that is rotated, scaled or warped in accordance with said determined pose and overlaid atop imagery captured from said article, through use of said salient point data.

6. The system of claim 5 that additionally includes said camera-equipped phone device, said phone device including a processor and a memory, said phone device memory containing instructions configuring the phone device to receive said salient point data and to use the salient point data in performing said augmented reality function of (a) determining spatial pose of the phone device relative to the printed article, and (b) presenting on the screen of said device both the camera-captured imagery, and said graphical augmentation that is rotated, scaled or warped in accordance with said determined pose and overlaid atop imagery captured from said article, through use of said salient point data.

7. The system of claim 5 in which the machine readable identifier comprises a plural-bit payload steganographically conveyed in said artwork, wherein the payload is not evident to a human observer, but is decodable from analysis of imagery captured from the article by the camera-equipped phone device.

8. The system of claim 5 in which the machine readable identifier comprises a barcode.

9. A method comprising the acts:

in a portable device, receiving identification data sensed from a barcode printed on an object; and

by reference to said identification data, obtaining a previously-established second set of data;

wherein the second set of data comprises salient points, said salient points comprising, for each of plural salient points;

location information for said point on the object; and

orientation information for said point on the object, and/or information representing a feature vector associated with said location; and

wherein either the identification data or the second set of data further comprises reference position data associated with the object, and the method includes determining position data for the portable device by reference to said reference position data and said salient points, wherein a user of the device can interact with the object in a position-dependent manner.

10. The method of claim 9 in which said determining position data comprises determining first position data and determining second position data, and the method further includes presenting on a screen of the device a first set of information when said first position data for the portable device has been determined, and presenting on the screen of the device a second, different, set of information when said second position data has been determined.

11. The method of claim 9 wherein the second set of data further comprises said reference position data associated with the object.

12. The method of claim 9 wherein the identification data further comprises said reference position data associated with the object.

13. A method comprising the acts:

in a camera-equipped portable device, capturing imagery depicting an object, and decoding identification data from a barcode depicted in said imagery; and

by reference to said identification data, obtaining a previously-established second set of data;

wherein the second set of data comprises salient points, said salient points comprising, for each of plural salient points;

location information for said point on the object; and

orientation information for said point on the object, and/or information representing a feature vector associated with said location;

the method further including;

determining a pose of the portable device relative to the object; and

presenting, on a screen of the portable device, both the camera-captured imagery, and a graphical augmentation that is rotated, scaled or warped in accordance with said pose.

14. The method of claim 13 that further includes determining the pose through use of said salient points.