Intelligent graphics interface in a handheld wireless device

• US 9,195,816 B2
• Filed: 07/02/2012
• Issued: 11/24/2015
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ferren, Bran, Jasmann, Schawn A., Buck, Morgan J.
• Primary Examiner(s): MANOHARAN, MUTHUSWAMY GANAPATHY
• Application Number:
  US13/539,833
• Publication Number:
  US 20120270601A1
• Time in Patent Office: 1,240 Days
• Field of Search: 455/556.1, 455/566, 345/619, 715/764
• US Class Current: 1/1
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G03B 3/00 Focusing arrangements of general interest for cameras, projectors or printers
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/40 by quorum, i.e. whereby two or more security principals are required
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06K 2207/1011 Aiming
• CPC Class Codes: G06K 7/10811 Focalisation
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 20/40145 : Biometric identity checks
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 : using facial parts and geometric relationships
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/45 : Detection of the body part being alive
• CPC Class Codes: H04L 63/083 : using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 2250/52 : including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

Various embodiments of the invention may allow applications and information to be clustered together in ways that simplify and automate various tasks. Other embodiments may present a graphical user interface to the user that allows the user to choose from a matrix of information categories and applications that can help provide that information. Still other embodiments may provide different user interface methods based on status awareness.

Claims

1. An apparatus for presenting information to a user, comprising:

an electronic device for operation by the user, the electronic device comprising a processor, a memory, a radio, a speaker, and a touchscreen, wherein the device is to;

obtain information for presentation to the user, wherein the information is obtained via a plurality of information sources;

determine multiple methods of presenting the information to the user;

select at least one of the multiple methods based on status awareness, wherein the status awareness includes situation awareness or context awareness, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors; and

present the information to the user with the selected method, wherein the information is presented via a matrix that is rotational at its axis to provide maximum access to a cluster of nodes representing a cluster of information points within the matrix, wherein the cluster of nodes corresponds to a plurality of information points populated by the plurality of information sources, wherein the plurality of information points includes areas of interests as labeled categories, wherein the matrix includes a matrix of choices where one axis of the matrix represents areas of interest corresponding to software applications as represented by another axis of the matrix such that a combination of an area of interest and a corresponding software application is chosen by the user and retrieved by the electronic device along with retrieval of data associated with the combination, wherein the matrix includes one or more matrixes corresponding to one or more modes associated with the user or the device, wherein the one or more modes include at least one of work mode, family mode, and play mode,

wherein the selection is based on effect of the selected method of presentation on other people, and wherein the selection includes not selecting presentation by audio through a speaker if ambient noise is higher than a reference noise, and selecting presentation by audio through a headset if ambient noise is smaller than a reference value, and

wherein each node in the cluster of nodes is selected manually by user or automatically based on the status awareness, wherein one or more nodes of the cluster of nodes are displayed as the matrix is vertically or horizontally rotated at its axis, wherein the matrix comprises one or more multi-dimensional shapes with each shape displaying one or more nodes representing one or more information points associated with one or more information sources of the plurality of information sources, wherein the matrix includes a cube or a polyhedron.

2. The apparatus of claim 1, wherein a node includes one or more of a software application, a selection of data, an internal service, an external service, an input from a sensor, a communications link, and one or more sources of useful data or functionalities, wherein the node includes a related node if it is at least deemed likely to provide the useful data or the functionalities contributing to a desired result for which the cluster of nodes is created, wherein the selected method includes one or more techniques chosen from a list of techniques comprising:

1) audio through a speaker;

2) audio through a headset;

3) displayed text;

4) a displayed image of a recorded photograph or video;

5) a displayed image of computer generated graphics; and

6) a display with brightness adjusted for ambient lighting conditions, and

wherein said selection is based at least partly on one or more factors chosen from a list of factors comprising;

1) current user activities;

2) importance of the current user activities;

3) affect of the method of presentation on other people; and

4) effectiveness of each method in the user'"'"'s current environment.

3. The apparatus of claim 1, wherein the plurality of information sources facilitate one or more primary information points represented through one or more primary nodes providing primary information relating to a subject, wherein the plurality of information sources is further to facilitate one or more secondary information points represented through one or more secondary nodes providing secondary information relating to the subject, the secondary information including additional pertinent information regarding the subject.

4. The apparatus of claim 1, wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor.

5. The apparatus of claim 1, wherein the context awareness is based on user contexts including one or more of the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user, wherein the context awareness comprises an analysis of the user contexts based on usefulness of the one or more user contexts to the status awareness.

6. The apparatus of claim 1, wherein the matrix is displayed in one or more forms including a virtual cube, wherein the virtual cube is capable of being rotated via one or more of a rotating wheel, a joystick, a trackball, a touchpad, a touch button, a human finger, a stylus, and a separate X-Y input device.

7. A method of presenting information to a user, comprising:

obtaining information for presentation to a user, wherein the information is obtained via a plurality of information sources;

determining multiple methods of presenting the information to the user with a handheld electronic device;

selecting at least one of the multiple methods based on status awareness, wherein the status awareness includes situation awareness or context awareness, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors; and

presenting the information to the user with the handheld electronic device, using said at least one method, wherein the information is presented via a matrix that is rotational at its axis to provide maximum access to a cluster of nodes representing a cluster of information points within the matrix, wherein the cluster of nodes corresponds to a plurality of information points populated by the plurality of information sources, wherein the plurality of information points includes areas of interests as labeled categories, wherein the matrix includes a matrix of choices where one axis of the matrix represents areas of interest corresponding to software applications as represented by another axis of the matrix such that a combination of an area of interest and a corresponding software application is chosen by the user and retrieved by the electronic device along with retrieval of data associated with the combination, wherein the matrix includes one or more matrixes corresponding to one or more modes associated with the user or the device, wherein the one or more modes include at least one of work mode, family mode, and play mode,

wherein the selection is based on effect of the selected method of presentation on other people, and wherein the selection includes not selecting presentation by audio through a speaker if ambient noise is higher than a referent noise, and selecting presentation by audio through a headset if ambient noise is smaller than a referent value, and

wherein each node in the cluster of nodes is selected manually by user or automatically based on the status awareness, wherein one or more nodes of the cluster of nodes are displayed as the matrix is vertically or horizontally rotated at its axis, wherein the matrix comprises one or more multi-dimensional shapes with each shape displaying one or more nodes representing one or more information points associated with one or more information sources of the plurality of information sources, wherein the matrix includes a cube or a polyhedron.

8. The method of claim 7, wherein a node includes one or more of a software application, a selection of data, an internal service, an external service, an input from a sensor, a communications link, and one or more sources of useful data or functionalities, wherein the node includes a related node if it is at least deemed likely to provide the useful data or the functionalities contributing to a desired result for which the cluster of nodes is created, wherein the selected method includes one or more techniques chosen from a list of techniques comprising:

1) audio through a speaker;

2) audio through a headset;

3) displayed text;

4) a displayed image of a recorded photograph or video;

5) a displayed image of computer generated graphics; and

6) a display with brightness adjusted for ambient lighting conditions, and

wherein said selection is based at least partly on one or more factors chosen from a list of factors comprising;

1) current user activities;

2) importance of the current user activities;

3) affect of the method of presentation on other people; and

4) effectiveness of each method in the user'"'"'s current environment.

9. The method of claim 7, wherein the plurality of information sources facilitate one or more primary information points represented through one or more primary nodes providing primary information relating to a subject, wherein the plurality of information sources is further to facilitate one or more secondary information points represented through one or more secondary nodes providing secondary information relating to the subject, the secondary information including additional pertinent information regarding the subject.

10. The method of claim 7, wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor.

11. The method of claim 7, wherein the context awareness is based on user contexts including one or more of the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user, wherein the context awareness comprises an analysis of the user contexts based on usefulness of the one or more user contexts to the status awareness.

12. The method of claim 7, wherein the matrix is displayed in one or more forms including a virtual cube, wherein the virtual cube is capable of being rotated via one or more of a rotating wheel, a joystick, a trackball, a touchpad, a touch button, a human finger, a stylus, and a separate X-Y input device.

13. A non-transitory computer-readable medium that contains instructions, which when executed by one or more processors result in performing operations comprising:

obtaining information for presentation to a user, wherein the information is obtained via a plurality of information sources;

determining multiple methods of presenting the information to the user with a handheld electronic device;

selecting at least one of the multiple methods based on status awareness, wherein the status awareness includes situation awareness or context awareness, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors; and

presenting the information to the user with the handheld electronic device, using said at least one method, wherein the information is presented via a matrix that is rotational at its axis to provide maximum access to a cluster of nodes representing a cluster of information points within the matrix, wherein the cluster of nodes corresponds to a plurality of information points populated by the plurality of information sources, wherein the plurality of information points includes areas of interests as labeled categories, wherein the matrix includes a matrix of choices where one axis of the matrix represents areas of interest corresponding to software applications as represented by another axis of the matrix such that a combination of an area of interest and a corresponding software application is chosen by the user and retrieved by the electronic device along with retrieval of data associated with the combination, wherein the matrix includes one or more matrixes corresponding to one or more modes associated with the user or the device, wherein the one or more modes include at least one of work mode, family mode, and play mode,

wherein the selection is based on effect of the selected method of presentation on other people, and wherein the selection includes not selecting presentation by audio through a speaker if ambient noise is higher than a referent noise, and selecting presentation by audio through a headset if ambient noise is smaller than a referent value, and

wherein each node in the cluster of nodes is selected manually by user or automatically based on the status awareness, wherein one or more nodes of the cluster of nodes are displayed as the matrix is vertically or horizontally rotated at its axis, wherein the matrix comprises one or more multi-dimensional shapes with each shape displaying one or more nodes representing one or more information points associated with one or more information sources of the plurality of information sources, wherein the matrix includes a cube or a polyhedron.

14. The non-transitory computer-readable medium of claim 13, wherein a node includes one or more of a software application, a selection of data, an internal service, an external service, an input from a sensor, a communications link, and one or more sources of useful data or functionalities, wherein the node includes a related node if it is at least deemed likely to provide the useful data or the functionalities contributing to a desired result for which the cluster of nodes is created,

wherein the matrix is displayed in one or more forms including a virtual cube, wherein the virtual cube is capable of being rotated via one or more of a rotating wheel, a joystick, a trackball, a touchpad, a touch button, a human finger, a stylus, and a separate X-Y input device,

wherein the selected method includes one or more techniques chosen from a list of techniques comprising:

1) audio through a speaker;

2) audio through a headset;

3) displayed text;

4) a displayed image of a recorded photograph or video;

5) a displayed image of computer generated graphics; and

6) a display with brightness adjusted for ambient lighting conditions, and

wherein said selection is based at least partly on one or more factors chosen from a list of factors comprising;

1) current user activities;

2) importance of the current user activities;

3) affect of the method of presentation on other people; and

4) effectiveness of each method in the user'"'"'s current environment.

15. The non-transitory computer-readable medium of claim 13, wherein the plurality of information sources facilitate one or more primary information points represented through one or more primary nodes providing primary information relating to a subject, and wherein the plurality of information sources is further to facilitate one or more secondary information points represented through one or more secondary nodes providing secondary information relating to the subject, the secondary information including additional pertinent information regarding the subject.

16. The non-transitory computer-readable medium of claim 13, wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor.

17. The non-transitory computer-readable medium of claim 13, wherein the context awareness is based on user contexts including one or more of the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user, wherein the context awareness comprises an analysis of the user contexts based on usefulness of the one or more user contexts to the status awareness.