Smartphone-Based Methods and Systems

• US 20120284012A1
• Filed: 11/02/2011
• Published: 11/08/2012
• Est. Priority Date: 11/04/2010
• Status: Active Grant

• Current Assignee: Digimarc Corporation
• Original Assignee: Digimarc Corporation
• Inventors: Rodriguez, Tony F., Davis, Bruce L.
• Granted Patent: US 9,240,021 B2
• Time in Patent Office: Days
• US Class Current: 704/1
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: H04L 2463/101 applying security measures for digital rights management data processing systems or methods, specially adapted for commerce, e.g. marketing, shopping, billing, auctions or e-commerce G06Q30/00
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 2250/04 including near field communication means, e.g. RFID
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

Technologies involving portable devices, such as smartphones and tablet computers, are disclosed. One arrangement enables a creator of content to select software with which that creator'"'"'s content should be rendered—assuring continuity between artistic intention and delivery. Another uses the camera of a smartphone to identify nearby subjects, and take actions based thereon. Others rely on near field chip (RFID) identification of objects, or on identification of audio streams (e.g., music, voice). Some technologies concern improvements to the user interfaces associated with such devices. Others involve use of these devices in shopping, text entry, sign language interpretation, and vision-based discovery. Still other improvements are architectural in nature, e.g., relating to evidence-based state machines, and blackboard systems. Yet other technologies concern use of linked data in portable devices—some of which exploit GPU capabilities. Still other technologies concern computational photography. A great variety of other features and arrangements are also detailed.

Claims

1. A portable device comprising:

a memory and a processing portion, disposed in a pocket-sized housing;

the memory including a first portion that stores code data representing plural code-based semantic triples, where each of said triples includes first, second and third codes, one of said codes being a subject code, one of said codes being a predicate code, and one of said codes being an object code; and

the memory further including a second portion that stores literal data associating different text literals with different of said codes;

wherein the processing portion including a GPU that processes code data in the first portion of the memory, to perform operations on information represented by the text literals in the second portion of the memory.

2. The device of claim 1 wherein said operations include semantic reasoning operations, by which the processing portion infers logical consequences based on facts asserted by the semantic triples in conjunction with the literal data.

3. The device of claim 1 that further includes a display screen, wherein the first portion of the memory comprises a frame buffer having storage dimensions that match pixel dimensions of the display screen, so that frame buffer technology known for use with graphics is applied for use with semantic triples.

4. The device of claim 3 wherein the processing portion is configured to store results of said semantic reasoning in said frame buffer as additional semantic triples.

5. The device of claim 1 wherein the first portion of the memory has a 2D arrangement and is organized with spatial locality, rather than having an arbitrary organization, wherein triples that include the same first code are grouped together, stored in adjoining locations in the first portion of the memory, so that the processing portion can better employ GPU functionality that relies on spatial locality.

6. The method of claim 5 wherein the first code comprises a subject code.

7. The device of claim 5 wherein triples that include the same first code are further organized by order of their second codes, to facilitate processing.

8. The method of claim 7 wherein the first code comprises a subject code, and the second codes comprise predicate codes.

9. The device of claim 1 wherein the first portion of the memory includes a first instance of a first code that is stored as a subject code, and also includes a second instance of the first code that is stored as an object code, wherein both instances of said first code are associated with the same text literal by data stored in the second portion of the memory.

10. The device of claim 1 wherein the memory further includes a third portion that stores rule data, and wherein the processing portion is configured to apply said rule data to the semantic triples, to discern information not previously expressed.

11. A portable device comprising:

a memory and a processing portion, disposed in a pocket-sized housing;

the memory including a frame buffer portion that stores code data representing plural code-based semantic triples, each triple comprising a subject code, a predicate code, and an object code; and

the memory further including a second portion that stores literal data associating different text literals with different of said codes;

wherein the processing portion including a GPU that processes code data in the first portion of the memory, to perform semantic reasoning involving concepts expressed using the text literals in the second portion of the memory.

12. A portable device comprising:

a memory and a processing portion, disposed in a pocket-sized housing;

the processing portion being configured by instructions stored in the memory, the processing portion including a GPU;

the memory storing code data representing plural code-based semantic triples, each triple comprising a subject code, a predicate code, and an object code;

the instructions configuring the processing portion to apply a series of templates to the stored code data, to perform operations based on said triples.

13. The device of claim 12 in which the instructions configure the processing portion to apply said series of templates to the stored code data, to perform semantic reasoning based on said triples.

14. A method comprising:

receiving a set of data corresponding to imagery captured by a camera-equipped portable device, said imagery depicting a subject;

determining identification data based on the received set of data;

by reference to said identification data, obtaining a second set of data, the second set of data comprising plural N-tuples, each including subject, predicate and object elements;

processing said N-tuples; and

taking an action based on said processing.

15. A portable device comprising:

a touch-screen, a memory and a processing portion, disposed in a pocket-sized housing;

the memory containing data representing plural code-based semantic triples, each triple comprising a subject code, a predicate code, and an object code;

the memory further comprising a portion containing text literals that indicate respective textual meanings for different of said codes;

the memory further containing software instructions that configure the processing portion to present, on the touch-screen, a user interface that is populated with said text literals, through which a user can define a query of the code-based semantic triples.

16. A portable device comprising:

a touch-screen, a memory and a processing portion, disposed in a pocket-sized housing;

the memory being logically organized as plural planes of storage, containing plural data N-tuples, elements of said N-tuples corresponding to text literals;

the processing portion including a GPU configured to operate on said data N-tuples as part of responding to a user query entered via a touchscreen user interface.

17. A method that includes:

receiving data corresponding to a user query;

applying a first plane-thresholding test to values stored in one plane of an N-tuple data store, the data store being logically organized to comprise N planes of data, storing plural data N-tuples, where N is at least 3, the plane-thresholding test identifying a first subset of said stored data N-tuples that meet a first aspect of the user query; and

presenting a result of the user query to the user.

18. The method of claim 17 that includes:

receiving data corresponding to a user query based on one or more text literals;

mapping said one or more text literals to one or more corresponding data codes; and

using one of said data codes in said first plane-thresholding test.

19. The method of claim 17 that further includes applying a second plane-thresholding test to values in said first identified subset of data N-tuples, to identify a further subset of data N-tuples that meet a second aspect of the user query.

20. The method of claim 19 that includes applying one or more further plane-thresholding tests to an earlier-identified subset of data N-tuples, to successively identify one or more further subsets of data N-tuples that meet further aspects of the user query.

21. The method of claim 17 in which presenting a result of the user query to the user includes populating a template with one or more text literals corresponding to one or more values from an identified subset of one or more data N-tuples.

22. A method comprising:

receiving a first set of data corresponding to imagery captured by a camera-equipped portable device, said imagery depicting a subject at a first resolution;

determining identification data based on the first set of data;

by reference to said identification data, obtaining a second set of data, the second set of data corresponding to imagery that also depicts said subject;

wherein the second set of data depicts said subject from a different perspective than the first set of data, and depicts the subject at a second resolution higher than the first resolution.

23. A method comprising:

receiving NFC identification data sensed from an object;

by reference to said identification data, obtaining a set of image data, the image data depicting said object;

presenting imagery corresponding to said image data on a touchscreen of a portable device.

24. The method of claim 23 that further includes:

receiving user input on the touchscreen; and

taking an action based on the user input;

wherein the action comprises a first action if the user touches the presented imagery at a first location, and the action comprises a second, different, action if the user touches the presented imagery at a second, different, location.