Narrowcasting From Public Displays, and Related Methods

• US 20100228632A1
• Filed: 03/03/2010
• Published: 09/09/2010
• Est. Priority Date: 03/03/2009
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F.
• Granted Patent: US 8,412,577 B2
• Time in Patent Office: Days
• US Class Current: 705/14.660
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06T 1/0064 Geometric transfor invariant watermarking, e.g. affine transform invariant
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2201/0051 Embedding of the watermark in the spatial domain
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: H04N 21/41415 involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk
• CPC Class Codes: H04N 21/8358 involving watermark protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467

Abstract

A user with a cell phone interacts, in a personalized session, with an electronic sign system. In some embodiments, the user'"'"'s location relative to the sign is discerned from camera imagery—either imagery captured by the cell phone (i.e., of the sign), or captured by the sign system (i.e., of the user). Demographic information about the user can be estimated from imagery captured acquired by the sign system, or can be accessed from stored profile data associated with the user. The sign system can transmit payoffs (e.g., digital coupons or other response data) to viewers—customized per user demographics. In some arrangements, the payoff data is represented by digital watermark data encoded in the signage content. The encoding can take into account the user'"'"'s location relative to the sign—allowing geometrical targeting of different payoffs to differently-located viewers. Other embodiments allow a user to engage an electronic sign system for interactive game play, using the cell phone as a controller.

Claims

1. A method involving an electronic sign, viewed by a first observer, the method comprising the acts:

obtaining position information about the first observer;

defining a first digital watermark signal that takes into account the position information;

encoding image data in accordance with said first digital watermark signal; and

presenting the encoded image data on the electronic sign.

2. The method of claim 1 that includes tailoring a scale of the first digital watermark signal in accordance with a viewing distance.

3. The method of claim 1 that includes tailoring a projection of the first digital watermark signal in accordance with a viewing angle.

4. The method of claim 1 that includes establishing a payload of the first digital watermark signal in accordance with contextual data relating to the first observer.

5. The method of claim 4 that includes obtaining the contextual data from a data store corresponding to the first observer.

6. The method of claim 4 that includes:

sensing the position information using a camera; and

estimating the contextual data about the first observer from image data produced by the camera.

7. The method of claim 1 that includes sensing the position information using a camera associated with the electronic sign.

8. The method of claim 1 that includes sensing the position information using a camera associated with the first observer.

9. The method of claim 8 that includes sensing the position information by reference to an apparent geometry of the electronic sign as viewed by the camera.

10. The method of claim 8 that includes sensing the position information by reference to a steganographically encoded digital watermark signal hidden in graphic data presented by the electronic sign.

11. The method of claim 1 that includes:

capturing image data corresponding to the electronic sign using a camera-equipped device conveyed by the observer; and

determining which of plural electronic signs is being observed by the first observer, by reference to the captured image data.

12. The method of claim 11 in which the determining includes discerning robust local image descriptors from captured image data.

13. The method of claim 11 that includes constraining a set of possible electronic signs that may be observed, by reference to data about the device'"'"'s location.

14. The method of claim 1 that includes:

obtaining second position information about a second observer;

defining a second digital watermark signal that takes into account the second position information; and

encoding the image data in accordance with both the second digital watermark signal and the first digital watermark signal.

15. The method of claim 14 that includes establishing a payload of the second digital watermark signal that is different than the payload of the first digital watermark signal.

16. The method of claim 15 that includes obtaining contextual data about the second observer, and establishing the payload of the second digital watermark signal in accordance with said contextual data.

17. A method involving an electronic sign system that is viewed by plural observers, each of the observers conveying a sensor-equipped device, the method comprising the acts:

establishing a first data payload for a first observer of the electronic sign;

establishing a second data payload for a second observer of the electronic sign;

steganographically encoding audio or visual content data with digital watermark data, the digital watermark data conveying the first and second data payloads; and

presenting the encoded content data using the electronic sign system;

wherein the sensor-equipped device conveyed by the first observer responds to the first data payload encoded in the presented content data but not the second data payload, and the sensor-equipped device conveyed by the second observer responds to the second data payload encoded in the presented content data but not the first data payload.

18. The method of claim 17 wherein the steganographically-encoded content data comprises visual content data, and the encoding of the first data payload is pre-distorted in accordance with a viewing geometry associated with the first observer.

19. The method of claim 17 that includes decoding both the first and second data payloads in one of said sensor-equipped devices, but selecting only one of the decoded payloads for response.

20. The method of claim 19 that includes selecting one of the decoded payloads for response based on stored profile information.

21. The method of claim 20 in which the profile information includes at least one of gender data, age data, and home zip code.

22. The method of claim 17 in which the first and second data payloads are time-multiplexed.

23. A method involving an electronic sign system including a screen viewed by different combinations of observers at different times, the method comprising the acts:

detecting a first person observing the screen;

encoding content presented by the electronic sign system with a first watermark signal corresponding to the first observer;

while the first person is still observing the screen, detecting a second person newly observing the screen;

encoding the content presented by the electronic sign system with a first watermark signal corresponding to the first observer, and also a second watermark signal corresponding to the second observer;

when one of said persons is detected as no longer observing the sign, encoding the content presented on the electronic sign system with the watermark signal corresponding to a remaining observer, but not with the watermark signal corresponding to the person who is no longer observing the sign;

wherein different combinations of watermark signals are encoded in content presented on the electronic sign system, in accordance with different combinations of persons observing the screen at different times.

24. The method of claim 23 wherein a portable device conveyed by the first person responds to the first watermark signal because the first watermark signal has been pre-distorted in accordance with a location of the first person.

25. The method of claim 23 wherein a portable device conveyed by the first person responds to the first watermark signal because the first watermark signal includes data that corresponds to profile data associated with said portable device or with said first person.

26. A method comprising the acts:

using a handheld device, capturing image data from a display;

determining a parameter of a digital watermark signal steganographically encoded in the captured image data, the parameter being other than payload data encoded by the watermark signal; and

based on a result of the determination, deciding how the device should respond to the display.

27. The method of claim 26 that includes:

comparing the determined parameter against a reference; and

based on a result of the comparison, determining how the device should respond to the display.

28. The method of claim 26 wherein the parameter is a geometrical parameter.

29. The method of claim 26 wherein the parameter is an error metric.

30. A method in which an electronic sign presents content that is viewed by plural observers, the method comprising the acts:

by use of a first camera-equipped device conveyed by a first observer, viewing the presented content and capturing first image data corresponding thereto;

processing the first image data to produce first identifying data;

by use of a second camera-equipped device conveyed by a second observer, viewing the same presented content and capturing second image data corresponding thereto, the second image data differing from the first due to different vantage points of the first and second observers;

processing the second image data to produce second identifying data;

using a sensor associated with the electronic sign, capturing third image data depicting the first and second observers;

processing the third image data to estimate contextual data associated with the first and second observers;

by reference to the estimated contextual data, determining first response data for the first observer, and second, different, response data for the second observer;

also processing the third image data to generate first location information corresponding to the first observer, and second location information corresponding to the second observer;

receiving first or second identifying data;

by reference to the generated location information, determining whether the received identifying data is based on image data captured by the first device or the second device;

if the received identifying data is determined to have been based on image data captured by the first device, responding to said received identifying data with the first response data; and

if the received identifying data is determined to have been based on image data captured by the second device, responding to said received identifying data with the second response data;

wherein the method infers from which observer the identifying data was received, and responds with contextually-determined response data corresponding to that observer.