Method, device and system for user interface adaptation

• US 10,303,340 B2
• Filed: 03/16/2011
• Issued: 05/28/2019
• Est. Priority Date: 03/16/2011
• Status: Active Grant

• Current Assignee: Piece Future Pte., Ltd.
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Ronkainen, Sami, Saarinen, Jukka
• Primary Examiner(s): Trapanese, William C
• Application Number:
  US14/004,521
• Publication Number:
  US 20140157143A1
• Time in Patent Office: 2,995 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities

Abstract

The present application relates to adapting a user interface based on context information. In the method, apparatus and system according to the present application, a context vector (920) is used to select a plurality of resource availability vectors (930, 940, 950). The plurality of resource availability vectors are indicative of availability of user resources in the corresponding context. The selected resource availability vectors are then used together in adapting a user interface (960, 970, 980) according to the availability of the user resources. There is also a method and devices for forming a data structure that can be used in the selection of the resource availability vectors. The method may select from different user interface modalities (960, 970, 980) by processing sensor input data with an associative memory, such as a sparse distributed memory.

Claims

1. A method comprising:

providing a data structure, implemented as a sparse distributed memory, comprising context vectors and resource availability vectors and having an association between said context vectors and said resource availability vectors, wherein the resource availability vectors are indicative of availability of user resources related to at least speech, audio, visual, or touch and comprise an array of elements corresponding to an available resource and elements corresponding to an unavailable resource,

receiving a context vector comprising context variables,

using said context vector to select a plurality of resource availability vectors, the plurality of resource availability vectors being indicative of availability of user resources related to at least speech, audio, visual, or touch,

modifying the selected resource availability vectors using a teaching resource availability vector associated with said context vector, wherein the teaching resource availability vector is based on information measured from a user'"'"'s capabilities to use each of the user resources, wherein the user'"'"'s capabilities to use each of the user resources is derived from training data comprising reaction time or correctness to a situation given at least one limitation on the user resources caused by the context variables; and

associating the context variables with the user'"'"'s capabilities of using each of the user resources in a current situation.

2. A method according to claim 1, comprising

modifying said selected resource availability vectors by increasing a value of the elements corresponding to the available resource in said teaching resource availability vector.

3. A method according to claim 1, comprising

modifying said selected resource availability vectors by decreasing a value of the elements corresponding to the unavailable resource in said teaching resource availability vector.

4. A method according to claim 1, comprising

using a distance measure in said selecting, and

modifying said selected resource availability vectors according to said distance measure.

5. A method according to claim 1, comprising

thresholding said teaching resource availability vector, and

modifying said selected resource availability vectors using said thresholded teaching resource availability vector.

6. A method according to claim 1, wherein said resource availability vectors reside in an associative memory such as a sparse distributed memory, the method comprising

inputting said context vector as a single memory address input to said associative memory, and

selecting a plurality of data vectors from the associative memory based on said input address.

7. A method according to claim 1, wherein resource availability vectors have an associated context address, the method comprising:

carrying out said selection by comparing said context vector to said context addresses.

8. A method according to claim 1, wherein said context vector is indicative of an environmental variable such as temperature, time, location, altitude, acceleration, attitude, lighting, noise conditions and humidity.

9. A method according to claim 1, wherein said context vector is indicative of the state of the device such as an operating profile, availability of a keyboard, availability of a touch input device and power supply conditions.

10. A method according to claim 1, wherein elements of the resource availability vectors provide certainty information on the availability of user resources.

11. A method according to claim 1, wherein the context vector comprises bit representations indicative of a degree of availability of user resources.

13. A method according to claim 6, comprising

choosing said associative memory among a group of associative memories based on said received context vector, and

selecting said plurality of data vectors in said selected associative memory using said context vector.

14. A method according to claim 6, wherein the possible address space of context addresses comprises more entries than the total number of context addresses associated to availability vectors, and the addresses of the memory locations having been assigned to the resource availability vectors using a function.

15. A method according to claim 12, wherein said resource availability vectors having been formed by selecting former resource availability vectors as in claim 1, said method comprising

modifying the selected resource availability vectors using a teaching resource availability vector associated with said context vector.

12. A method comprising:

receiving a context vector comprising context variables,

using said context vector to select a plurality of resource availability vectors, the plurality of resource availability vectors being indicative of availability of user resources related to at least speech, audio, visual, or touch and comprising an array of elements corresponding to an available resource and elements corresponding to an unavailable resource,

associating the context variables with a user'"'"'s capabilities of using each of the user resources in a current situation, wherein the user'"'"'s capabilities of using each of the user resources is derived from training data comprising reaction time or correctness to a situation given at least one limitation on the user resources caused by the context variables, and

using said selected resource availability vectors together in adapting a user interface according to the availability of said user resources based on information measured from the user'"'"'s capabilities.

16. A method according to claim 12, comprising

forming an adaptation vector from said selected resource availability vectors.

17. A computer program product stored on a non-transitory computer readable medium having executable computer program code instructions stored therein, the computer program code instructions configured for:

providing a data structure, implemented as a sparse distributed memory, comprising context vectors and resource availability vectors and an association between said context vectors and said resource availability vectors,

using an input context vector comprising context variables to select a plurality of resource availability vectors from said data structure, the plurality of resource availability vectors being indicative of availability of user resources related to at least speech, audio, visual, or touch and comprising an array of elements corresponding to an available resource and elements corresponding to an unavailable resource,

associating the context variables with a user'"'"'s capabilities of using each of the user resources in a current situation, wherein the user'"'"'s capabilities of using each of the user resources is derived from training data comprising reaction time or correctness to a situation given at least one limitation on the user resources caused by the context variables, and

controlling operation of a device using said selected resource availability vectors together to adapt a user interface of the device according to the availability of said user resources based on information measured from the user'"'"'s capabilities.

18. An apparatus comprising at least one processor, memory and computer program code in said memory, said computer program code being configured to, with the at least one processor, cause the apparatus at least to:

receive a context vector comprising context variables,

use said context vector to select a plurality of resource availability vectors, the plurality of resource availability vectors being indicative of availability of user resources related to at least speech, audio, visual, or touch and comprising an array of elements corresponding to an available resource and elements corresponding to an unavailable resource,

associate the context variables with a user'"'"'s capabilities of using each of the user resources in a current situation, wherein the user'"'"'s capabilities of using each of the user resources is derived from training data comprising reaction time or correctness to a situation given at least one limitation on the user resources caused by the context variables, and

use said selected resource availability vectors together in adapting a user interface according to the availability of said user resources based on information measured from the user'"'"'s capabilities or performance.

19. An apparatus according to claim 18, further comprising an associative memory such as a sparse distributed memory, resource availability vectors residing in said associative memory, and computer program code configured to cause the apparatus at least to:

input said context vector as a single memory address input to said associative memory, and

select a plurality of data vectors from the memory base on said input address.

20. An apparatus according to claim 18, further comprising means for producing said context vector.

21. An apparatus according to claim 18 comprising different user interface modalities, and means for enabling and disabling said user interface modalities based on said availability of user resources.