Methods and arrangements relating to signal rich art

• US 10,356,274 B2
• Filed: 02/06/2017
• Issued: 07/16/2019
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Haaga, Donald, Reed, Alastair M., Rodriguez, Tony F.
• Primary Examiner(s): Hong, Stephen S
• Assistant Examiner(s): Ludwig, Matthew J
• Application Number:
  US15/425,573
• Publication Number:
  US 20170214820A1
• Time in Patent Office: 890 Days
• Field of Search: 715201, 715204, 715208, 715230, 715255, 382100, 382232, 382233, 382235
• CPC Class Codes: G06F 21/64 Protecting data integrity, e.g. using checksums, certificates or signatures
• CPC Class Codes: G06F 2221/2119 Authenticating web pages, e.g. with suspicious links
• CPC Class Codes: G06F 2221/2151 Time stamp
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 11/80 Creating or modifying a manually drawn or painted image using a manual input device, e.g. mouse, light pen, direction keys on keyboard
• CPC Class Codes: H04N 1/32149 Methods relating to embedding, encoding, decoding, detection or retrieval operations

Abstract

Art can be infused with network identifiers at the time of its creation, rather than as a post-process. The identifiers may be encoded as overt elements of the art, and enable the artist to reprogram—as over time—an augmentation of the artwork experience via network-delivered components. These network components can include stimuli present when the artist created the work (e.g., music), commentary by the artist, video and augmented reality features, audience-crowdsourced content (e.g., imagery of, or feedback provided by, other viewers encountering the artwork), etc. The artwork augmentation can vary with the user'"'"'s context (e.g., location, demographics, interests, history). Physical brushes can be equipped to insert such identifiers in graphic arts; other authoring tools are disclosed as well. The network experience can be delivered via smartphones, projectors, and other devices. A great number of other features and arrangements are also detailed.

Claims

1. A method of providing augmentation data to sensor-equipped user devices, the method comprising the acts:

providing artwork for a first printed item, the artwork including a machine-readable code, the machine-readable code encoding a plural-bit identifier;

creating an augmentation to be presented on a display of a sensor-equipped user device, when the artwork is imaged by the sensor-equipped device, and storing augmentation data defining said augmentation, in association with said identifier;

in connection with imaging of the artwork by a first sensor-equipped device, (a) logging, in a database record, data associating said device and the first printed item, indicating said device has interacted with the first printed item; and

(b) sending the stored augmentation data to said device, for presentation of the augmentation on a display of said device; and

in connection with imaging of the artwork by a second sensor-equipped device, (a) logging, in a database record, data associating the second device and the first printed item, indicating the second device has interacted with the first printed item;

(b) determining, from previously-logged data, that the second device has previously interacted with different printed items, including different machine readable codes; and

(c) based on said determination, sending augmentation data to the second device that is different than the stored augmentation data sent to the first device.

2. The method of claim 1 in which the stored augmentation data comprises an augmentation overlay that is to be presented on a display of a camera-equipped user device, together with imagery captured by the camera of said device.

3. A method for authoring product packaging label artwork comprising the acts:

providing a graphical user interface that includes authoring tools enabling a user to edit layered elements of product packaging label artwork, said layered elements including first and second layered elements;

receiving a first plural-bit identifier;

steganographically-encoding the first identifier in said first layered element by digital watermarking, but not in said second layered element;

determining, by a modeling act based on input specifying rendering circumstances, how the steganographic encoding of the product packaging label artwork will be transformed by a rendering operation, said determining comprising assessing relative strength of the steganographic encoding after such rendering operation;

presenting results of said assessment for user review, in the form of a 2D strength map; and

after said analyzing, revising the steganographic encoding of said first identifier in said first product packaging label artwork layered element;

wherein at least certain of said acts are performed by a processor-controlled hardware system.

4. The method of claim 3 in which the determining act includes rendering the product packaging label artwork.

5. The method of claim 3 in which the determining act includes modeling the rendering of the product packaging label artwork.

6. The method of claim 3 in which the steganographic encoding comprises subtracting an encoding pattern from the first layered element.

7. The method of claim 3 in which the revising comprises a software tool responding to said act of assessing by automatically adjusting the artwork to enhance recognizability of the steganographic encoding.

8. The method of claim 3 wherein the product packaging label artwork also includes a second product packaging label artwork element that conveys a plural-bit identifier, wherein detection of the first product packaging label artwork element at a first region by a consumer'"'"'s mobile device leads to presentation of a first network response to the consumer, and detection of the second product packaging label artwork element at a second region by the consumer'"'"'s mobile device leads to presentation of a second, different, network response to the consumer.

9. The method of claim 8 in which the second identifier is different than the first identifier.

10. The method of claim 8 in which the second product packaging label artwork element comprises a 1D or 2D barcode.

11. The method of claim 3 that further includes:

creating an augmented reality overlay to be presented on a display of a consumer'"'"'s mobile device in response to interaction between the mobile device and the product packaging label artwork;

storing augmentation data defining said augmented reality overlay, in association with said identifier;

in connection with capture of imagery of said product packaging label artwork by the consumer'"'"'s mobile device, and decoding of the identifier from the steganographically encoded artwork element, (a) logging, in a database record, data associating the consumer or the consumer'"'"'s mobile device with the product packaging label artwork, indicating the consumer has interacted with the product packaging label artwork; and

(b) sending the stored augmentation data to the consumer'"'"'s mobile device, for presentation of the augmented reality overlay.

12. The method of claim 11 that further includes, in connection with capture of imagery of the product packaging label artwork by a second consumer'"'"'s mobile device, (a) logging, in a database record, data associating the second consumer or the second consumer'"'"'s mobile device with the product packaging label artwork, indicating the second consumer has interacted with the product packaging label artwork;

(b) determining, from previously-logged data, that the second consumer has previously interacted with different artwork, which steganographically encoded a different identifier; and

(c) based on said determination, sending data to the second consumer'"'"'s mobile device that is different than the stored augmentation data sent to the first consumer'"'"'s mobile device.

13. The method of claim 3 in which the first layered element is added to the packaging label artwork by a user employing said graphical user interface, but the steganographic encoding of said first layered element with the received identifier is not performed until after the first layered element is added.

14. The method of claim 13 in which steganographic encoding of the first layered element is simulated by texturing that is displayed on a computer screen presented to the user to represent the label artwork.

15. The method of claim 13 in which the steganographic encoding of the first layered element is performed after authoring visual appearance of the first layered element has concluded.

16. The method of claim 3 in which said determining comprises determining by a modeling act based on user input specifying rendering circumstances.

17. A method for authoring product packaging label artwork having a network dimension, the method comprising the acts:

with a graphics editing program, creating artwork having a visual appearance desired by a user of the program, the artwork including first and second layered elements;

receiving a first plural-bit identifier;

with a processor-controlled hardware system, steganographically-encoding the first identifier in the first layered element of the product packaging label artwork la digital watermarking, but not in the second layered element;

determining, by a modeling act based on input specifying rendering circumstances, whether the product packaging label artwork is deficient in terms of recognizability of its steganographically-encoded first identifier;

following a determination that the artwork is deficient in terms of said recognizability, revising the steganographic encoding to enhance recognizability of its steganographically-encoded first identifier; and

sending the adjusted artwork to another party for printing.

18. The method of claim 17 that includes presenting results of said determining act in map form, indicating strength of the steganographically-encoded first identifier at different locations across the label artwork.

19. The method of claim 17 in which the determining act includes rendering the product packaging label artwork.

20. The method of claim 17 in which the determining act includes modeling the rendering of the product packaging label artwork.

21. The method of claim 17 in which the steganographic encoding comprises subtracting an encoding pattern from the first layered element.

22. The method of claim 17 in which the adjusting act comprises a software tool responding to said determining act by automatically adjusting the artwork to enhance recognizability of the steganographic encoding.

23. The method of claim 17 wherein the product packaging label artwork also includes a second product packaging label artwork element that conveys an identifier, wherein detection of the first product packaging label artwork element at a first region in the artwork by a consumer'"'"'s mobile device leads to presentation of a first network response to the consumer, and detection of the second product packaging label artwork element at a second region in the artwork by the consumer'"'"'s mobile device leads to presentation of a second, different, network response to the consumer.

24. The method of claim 23 in which the second identifier is different than the first identifier.

25. The method of claim 24 in which the second identifier is steganographically encoded in the artwork.

26. The method of claim 23 in which the second product packaging label artwork element comprises a 1D or 2D barcode.

27. The method of claim 17 that includes steganographically-encoding the label artwork with the first identifier, and with a second identifier, wherein the first and second identifiers are encoded at different scales.

28. The method of claim 27 in which the first and second identifiers are different.

29. The method of claim 17 in which the identifier is paired with associated information, and both the identifier and the associated information are pre-existing at a time that the user creates the artwork with the graphics editing program.

30. The method of claim 17 in which said determining comprises determining by a modeling act based on user input specifying rendering circumstances.

31. A non-transitory computer readable medium containing software instructions operable to configure a system, including a processor, to perform acts including:

providing a graphical user interface that includes authoring tools enabling a user to edit elements of product packaging label artwork, said layered elements including first and second layered elements;

receiving a first plural-bit identifier;

steganographically-encoding the first identifier in said first layered element by digital watermarking, but not in said second layered element;

determining, by a modeling act based on input specifying rendering circumstances, whether the product packaging label artwork is deficient in terms of recognizability of its steganographically-encoded first identifier; and

following a determination that the artwork is deficient in terms of said recognizability, revising the steganographic encoding to enhance recognizability of its steganographically-encoded first identifier.

32. The computer-readable medium of claim 31 in which said determining comprises determining by a modeling act based on user input specifying rendering circumstances.

33. A computer system comprising:

a processor;

a memory;

a display screen;

computer instructions stored in the memory that configure the system to perform acts including (a) providing a graphical user interface that includes authoring tools enabling a user to edit layered elements of product packaging label artwork, including first and second layered elements; and

(b) receiving a first plural-bit identifier;

means for steganographically-encoding the first identifier in said first layered element by digital watermarking, but not in said second layered element;

said computer instructions further being adapted to perform acts including (c) determining, by a modeling act based on input specifying rendering circumstances, whether the product packaging label artwork is deficient in terms of recognizability of its steganographically-encoded first identifier; and

(d) in response to a determination that the artwork is deficient in terms of said recognizability, revising the steganographic encoding to enhance recognizability of its steganographically-encoded first identifier.

34. The system of claim 33 in which said determining comprises determining by a modeling act based on user input specifying rendering circumstances.