Imagery and annotations

• US 9,818,150 B2
• Filed: 01/10/2014
• Issued: 11/14/2017
• Est. Priority Date: 04/05/2013
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rhoads, Geoffrey B., Conwell, William Y.
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US14/152,925
• Publication Number:
  US 20140304122A1
• Time in Patent Office: 1,404 Days
• Field of Search: 345633
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A decade from now, a visit to the supermarket will be a very different experience than the familiar experiences of decades past. Product packaging will come alive with interactivity—each object a portal into a rich tapestry of experiences, with contributions authored by the product brand, by the store selling the product, and by other shoppers. The present technology concerns arrangements for authoring and delivering such experiences. A great variety of other features and technologies are also detailed.

Claims

1. A method involving an on-screen display of differently-oriented views of a 3D product captured by a camera of a user'"'"'s mobile device, the method comprising the acts:

receiving a first camera-captured view of the 3D product, captured when the product has a first pose relative to the camera, said first camera-captured view depicting the 3D product at a first orientation; and

augmenting display of said first view on a screen of the user'"'"'s mobile device with an annotation, the annotation being associated with a first anchoring feature on the 3D product, the anchoring feature being a particular location within a face of the product depicted in the first view, the annotation being positioned to the right of the displayed 3D product on said screen;

receiving a second camera-captured view of the 3D product, captured when the product has a second pose relative to the camera, said second camera-captured view depicting the 3D product at a second orientation and again depicting said first anchoring feature; and

augmenting display of said second view on the screen of the user'"'"'s mobile device with said annotation, said annotation still being associated with said first anchoring feature, but being positioned to the left of the displayed 3D product on said screen, the annotation thus moving from the right side of the displayed 3D product to the left side; and

receiving a third camera-captured view of the 3D product, captured when the product has a third pose relative to the camera, said third camera-captured view depicting the 3D product at a third orientation and again depicting said first anchoring feature; and

displaying the third view on the screen of the user'"'"'s mobile device without said annotation, despite depiction of said first anchoring feature in said third camera-captured view;

wherein varying presentation of the annotation with variations in product-camera pose helps serve the function of drawing the user'"'"'s attention to the annotation.

2. The method of claim 1 that includes sensing said first and second orientations by analyzing image data depicting said product, using a hardware processor configured to perform such act.

3. The method of claim 1 in which the annotation is spatially associated with the first anchoring feature on the product.

4. The method of claim 1 in which the product includes a first face comprising said first anchoring feature, and the method includes:

when the product is displayed with the first face oriented within a threshold angular value of a viewpoint, augmenting the on-screen display with said annotation; and

when the product is displayed with the first face oriented outside said threshold angular value, displaying the product without said annotation;

wherein the annotation disappears when the first face is turned away from the viewpoint.

5. The method of claim 1 wherein the product comprises a retail package box.

6. A method involving an on-screen display of differently-oriented views of a 3D object that includes first and second anchoring features on a common planar face, each of said anchoring features being a particular location within said face of the 3D object, the method comprising the acts:

augmenting the on-screen display with a first annotation associated with the first anchoring feature, and with a second annotation associated with the second anchoring feature, while the orientation of the 3D object relative to the camera is varied between first, second and third orientations, wherein;

when the object is displayed at the first orientation, presenting both the first and second annotations to the right of the displayed 3D object;

when the object is displayed at the second orientation, presenting both the first and second annotations to the left of the displayed object, the annotations thus moving from the right side of the displayed 3D product to the left side; and

when the object is displayed at the third orientation, which is between the first and second orientations, presenting the first annotation to the right of the displayed object, and presenting the second annotation to the left of the displayed object;

wherein varying presentation of the first and second annotations with variations in object-camera orientation helps serve the function of drawing the user'"'"'s attention to the annotations.

7. The method of claim 6 wherein the object comprises a retail package box.

8. The method of claim 6 that includes sensing said first and second orientations by analyzing image data depicting said object, using a hardware processor configured to perform such act.

9. A method involving an on-screen display of differently-oriented views of a 3D object having a planar face, the method comprising the acts:

determining orientation information indicating an orientation at which said face of the displayed 3D object is viewed;

comparing said determined orientation information with stored reference information, using a hardware processor configured to perform such act, the stored reference information serving to specify a threshold angular value datum, said datum defining orientations at which an annotation should be rendered; and

when said comparing has a first outcome, augmenting the on-screen display by presenting the annotation, and when said comparing has a second outcome, not augmenting the on-screen display with the annotation;

wherein presentation of the annotation varies depending both on said determined orientation information, and on said threshold angular value datum, wherein such varying presentation helps serve the function of drawing a user'"'"'s attention to the annotation; and

wherein the annotation is associated with an anchoring feature on said planar face.

10. The method of claim 9 in which the object includes first and second features on a common planar face of the object, and the method includes:

determining information indicating viewing orientations of the first and second features;

comparing said determined information with stored first information for the first feature, and with stored second information for the second feature; and

at a first time, based on a result of said comparing, augmenting the on-screen display by presenting a first annotation corresponding to the first feature, but not augmenting the on-screen display by presenting a second annotation corresponding to the second feature.

11. The method of claim 10 that includes, at a different time, with a differently-oriented view of the object:

based on a result of said comparing, augmenting the on-screen display by presenting a first annotation corresponding to the first feature, and also augmenting the on-screen display by presenting a second annotation corresponding to the second feature.

12. The method of claim 9 wherein the object comprises a retail package box.

13. The method of claim 9 that includes sensing said first and second orientations by analyzing image data depicting said object, using a hardware processor configured to perform such act.

14. The method of claim 9 in which the object comprises a packaged food item, and the method further includes responding to user selection of the annotation by rendering a nutrition facts graphic on the screen, said rendering including a visual transition in which the annotation expands out from one of the corners of the display screen.

15. The method of claim 9 in which the object comprises a packaged food item that includes first and second features on a common planar face thereof, the first and second features having first and second annotations respectively associated therewith, the method further including:

when said face is viewed at a first orientation, presenting neither the first nor second annotations as augmentations on the display;

when said face is viewed at a second orientation, presenting the first but not the second annotation as an augmentation on the display; and

when said face is viewed at a third orientation, presenting both the first and second annotations on the display.

16. The method of claim 9 in which the object comprises a packaged food item that includes first and second features on a common planar face thereof, the first and second features having first and second annotations respectively associated therewith, the method further including:

when said face is viewed at a first orientation, presenting both the first and second annotations on a same side of the displayed object on the display screen; and

when said face is viewed at a second orientation, presenting the first annotation to the right side of the displayed object on the display screen, and presenting the second annotation to the left side of the displayed object on the display screen.

17. The method of claim 9 in which the object comprises a packaged food item having a planar face, and the annotation is associated with an anchoring feature on said planar face, the anchoring feature being defined using a steganographic reference system comprising plural tiles.

18. The method of claim 9 that further includes capturing imagery of said object with differently-oriented views with a head-mounted camera system worn by the user.

19. The method of claim 9 that further includes the acts:

recognizing the object from imagery thereof captured by a camera conveyed by the user;

obtaining different images of the recognized object from a remote computer system, said different images of the recognized object having been authored by a third party different than said user; and

presenting different of said images to the user in response to user input.

20. The method of claim 9 that further includes the acts:

recognizing the object from imagery thereof captured by a camera conveyed by the user;

obtaining 3D model information for the recognized product from a remote computer system, said 3D model having been authored by a third party different than said user; and

presenting a rendering of the 3D model to the user on the screen.