Smartphone-Based Methods and Systems

• US 20120208592A1
• Filed: 11/17/2011
• Published: 08/16/2012
• Est. Priority Date: 11/04/2010
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Davis, Bruce L., Rodriguez, Tony F., Conwell, William Y., Alattar, Adnan M., Rogers, Eliot, Bradley, Brett A., Reed, Alastair M., Brandis, Robert Craig, Rhoads, Geoffrey G., Owen, Jerrine K.
• Granted Patent: US 8,819,172 B2
• Time in Patent Office: Days
• US Class Current: 455/556.1
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06K 7/1443 locating of the code in an image
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06T 1/0021 Image watermarking
• CPC Class Codes: G06T 1/0064 Geometric transfor invariant watermarking, e.g. affine transform invariant
• CPC Class Codes: G06T 2201/0052 Embedding of the watermark in the frequency domain
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04M 1/0264 for a camera module assembly
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals
• CPC Class Codes: H04W 4/50 Service provisioning or reconfiguring
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: H04W 8/22 Processing or transfer of terminal data, e.g. status or physical capabilities
• CPC Class Codes: H05B 47/11 by determining the brightness or colour temperature of ambient light
• CPC Class Codes: H05B 47/12 by detecting audible sound
• CPC Class Codes: Y02B 20/40 Control techniques providing energy savings, e.g. smart controller or presence detection

Abstract

Arrangements involving portable devices (e.g., smartphones and tablet computers) are disclosed. One arrangement enables a content creator to select software with which that creator'"'"'s content should be rendered—assuring continuity between artistic intention and delivery. Another utilizes a device camera to identify nearby subjects, and take actions based thereon. Others rely on near field chip (RFID) identification of objects, or on identification of audio streams (e.g., music, voice). Some technologies concern improvements to the user interfaces associated with such devices. Others involve use of these devices in connection with shopping, text entry, sign language interpretation, and vision-based discovery. Still other improvements are architectural in nature, e.g., relating to evidence-based state machines, and blackboard systems. Yet other technologies concern use of linked data in portable devices—some of which exploit GPU capabilities. Still other technologies concern computational photography. A great variety of other features and arrangements are also detailed.

Claims

1. A method practiced with a consumer electronic device, comprising:

receiving content data representing audio, image and/or video content, the content data being received in a file format of a type that the electronic device is capable of rendering to produce a content-related experience, by reason of a first software rendering program earlier installed or otherwise made available for use with the device;

by reference to the received content data, determining one or more software rendering program(s) prescribed by a proprietor of the content data for producing a content-related user experience with said received content data;

determining that none of said determined software rendering programs is presently installed or available for use with the device; and

refusing to render the received content data on the device, despite the capable first software rendering program;

wherein the consumer electronic device respects the prescription of the proprietor regarding the software rendering program(s) by which a content-related user experience is to be delivered to the user.

2. The method of claim 1 that further includes presenting a user of the device with information enabling the user to direct installation of one or more of the software rendering program(s) prescribed by the proprietor.

3. The method of claim 1 in which determining one or more software rendering program(s) prescribed by the proprietor is also based on one or more context factors.

4. The method of claim 3 in which determining one or more software rendering program(s) prescribed by the proprietor is based on a context factor comprising identification of a type identifier for the consumer electronic device.

5. A method comprising:

capturing plural frames of imagery using a portable device, the portable device having an image sensor and a second sensor;

selecting a subset of at least two of said captured frames, based on data from the second sensor; and

combining said selected frames.

6. The method of claim 5 that further includes:

discerning corresponding salient points in the selected subset of captured frames; and

transforming at least a first of said selected frames prior to combining;

said transforming including estimating a scale factor for the first frame relative to a reference frame by reference to a histogram of distances between corresponding salient points in the two frames, and thereafter estimating a translation, in x- and y-, for the first frame relative to the reference frame.

7. The method of claim 5 that includes applying a weighting factor to at least one of said selected subset of frames, wherein the weighting factor is based on data from the second sensor.

8. A smartphone device including a processor, a memory, and plural sensors including at least a camera and a microphone, the aforesaid elements cooperating to define plural recognition agents including an audio fingerprint recognition agent, a barcode recognition agent and an image watermark recognition agent, said agents being configured to read data from and write data to a blackboard data structure of the memory, the blackboard data structure also being configured to receive data from the sensors and compile queues of stored image data and audio data that said agents process, the aforesaid elements cooperating under the control of an event controller defined by instructions in said memory.

9. The smartphone device of claim 8 in which plural of the recognition agents are each configured to estimate a resource cost required to perform a function, and estimate a quality of result that may be achieved by such function, and to store such estimates in the blackboard structure, wherein the event controller is configured to assess such stored estimates from plural recognition agents and decide which recognition agent(s) to activate, based thereon.

10. The smartphone device of claim 8 which further includes an inference engine configured to monitor the blackboard data structure and infer contextual data, the event controller being configured to use said contextual data in deciding which recognition agent(s) to activate.

11. The smartphone device of claim 8 in which the device is configured to process audio and image streams in separate threads, and is configured so that each recognition agent runs in it own thread.

12. The smartphone device of claim 8 wherein the sensors further include an accelerometer, and the device is configured to capture an image, determine a luminance value related thereto, assess acceleration associated with the image, assess saturation of the image and, based thereon, assess an image quality, and thereafter decide whether to invoke a watermark reading operation on the image based on the assessed image quality.

13. A method comprising:

determining network relationships between a person and plural objects;

from the network relationships, discerning an object'"'"'s relative importance to the person; and

taking an action based on said discerned relative importance.

14. The method of claim 13 that includes processing data based on said discerned relative importance, and presenting a resulting set of processed data to the person.

15. The method of claim 14 in which the processing comprises selecting, for presentation to the person, a subset of content from among a larger set of content, based on said discerned relative importance.

16. The method of claim 13 in which determining network relationships includes analyzing imagery captured by a device operated by said person.

17. A method comprising:

in a retail store, receiving a signal indicating a product interest by a shopper in the store; and

taking an action based on said received signal, the action including at least one of;

entering the shopper in a queue for service, informing the shopper of a wait time for service, providing the shopper information about a retail store staffer who will provide service, and providing the shopper an enticement for waiting for service, wherein the enticement comprises entertainment content for which a charge would otherwise be assessed.

18. The method of claim 17 in which the action comprises entering the shopper in a queue for service ahead of one or more other shoppers who earlier entered the queue.

19. A method comprising:

by reference to data sensed from a first product displayed in a retail store by a shopper'"'"'s smartphone, determining whether a second product is compatible with the first product; and

taking an action based on said determination.

20. A method comprising:

displaying an arrangement of text characters on a screen of a portable device from which a user can select;

sensing a user'"'"'s gaze towards a part of the screen to make a text selection; and

changing the arrangement of characters on the screen from which the user can next select.

21. The method of claim 20 that further includes:

sensing the user'"'"'s gaze towards two or more reference points on the screen, to establish a geometrical reference frame; and

providing feedback on the screen to indicate where, on the screen, the smartphone senses the user'"'"'s gaze is directed.