Handheld electronic device using status awareness

• US 8,700,012 B2
• Filed: 07/02/2012
• Issued: 04/15/2014
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ferren, Bran, Nachman, Lama, Del Pasqua, Kieran, March, Wendy, Neumann, John Cross, Shah, Rahul, Shahabdeen, Junaith A., Healey, Jennifer A., Subramanian, Sushmita, Raffa, Giuseppe, Essaian, Alexander, Huang, Jonathan
• Primary Examiner(s): Georgewill, Opiribo
• Application Number:
  US13/539,880
• Publication Number:
  US 20120276932A1
• Time in Patent Office: 652 Days
• Field of Search: 455/436, 455/414.1, 455/450, 455/566, 455/414.2, 715/764, 715/708, 715/706, 709/204, 379/88.18, 379/88.13
• US Class Current: 455/414.2
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G03B 3/00 Focusing arrangements of general interest for cameras, projectors or printers
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/40 by quorum, i.e. whereby two or more security principals are required
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06K 2207/1011 Aiming
• CPC Class Codes: G06K 7/10811 Focalisation
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 20/40145 : Biometric identity checks
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 : using facial parts and geometric relationships
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/45 : Detection of the body part being alive
• CPC Class Codes: H04L 63/083 : using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 2250/52 : including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

Through status awareness, a handheld communications device may determine the location, activity, and/or physical or emotional state of the user. This information may in turn be used for various purposes, such as 1) determining how to alert the user of an incoming communication, 2) determining what format to use for communicating with the user, and 3) determining how to present the user'"'"'s status to another person'"'"'s communication device.

Claims

1. An apparatus comprising

a wireless communication device having a touchscreen, a processor, a memory, and a radio with a receiver and a transmitter, the device to:

monitor context awareness and situation awareness relating to a user of the device, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors;

analyze status awareness relating to the user'"'"'s status based on at least one of the context awareness and the situation awareness, wherein the device is further to analyze at least one of external priorities without regard for the status awareness, internal priorities based on the status awareness, and probable status of the user based on a combination of possibility analysis and probability analysis, and an assigned confidence factor as compared to a confidence threshold value;

create a presentation indicating the user'"'"'s status; and

transmit the presentation to an external device, wherein the presentation includes a visual presentation having at least one of a graphically-generated static or animated avatar, a picture, and text, wherein one or more potential decisions are made based on assumptions, wherein a confidence level is associated with each assumption, wherein the confidence level is compared against a predefined threshold level to determine whether a graphical image be presented, wherein each potential decision comprises a manner to contact the user or not to contact the user and instead perform a different task.

2. The apparatus of claim 1, wherein the context awareness is based on user contexts including one or more of the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user,

wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor, and

wherein said transmitting the presentation is in response to determination of at least one condition selected from a list of conditions consisting of:

1) an incoming request for communication with the user;

2) an incoming request for the user'"'"'s status;

3) a pre-determined time of day;

4) expiration of a timer,

5) manual initiation of the transmission; and

6) a pre-defined change of the user'"'"'s condition.

3. The apparatus of claim 1, wherein the presentation is in a graphics form to be displayed by the external device, wherein the presentation is to indicate at least one factor selected from a list consisting of:

1) the user'"'"'s activity;

2) the user'"'"'s location;

3) the user'"'"'s emotional state; and

4) the user'"'"'s physical state.

4. The apparatus of claim 1, wherein contents of the presentation are to be based at least partly on a condition selected from a list of conditions consisting of:

1) a requestor'"'"'s identity; and

2) a presentation capability of the external device.

11. The apparatus of claim 1, wherein the picture comprises a display of one or more of good or bad weather, the user'"'"'s surroundings, an aerial view of the user'"'"'s location, and a depiction of a physical location including one or more of a restaurant, a library, a park, and a stadium, wherein the text comprises specifying one or more of the user'"'"'s meeting status, an address of the user'"'"'s location, and an address of the user'"'"'s final destination if the user is traveling.

12. The apparatus of claim 1, wherein the avatar to indicate one or more of the user'"'"'s activity, the user'"'"'s location, the user status, and the user'"'"'s status with respect to another user or device, wherein the presentation includes a multi-layered graphical image having multiple layers depicting surrounding information relating to the user'"'"'s status, wherein the surrounding information is used by a recipient to base potential decisions

wherein the avatar further indicates the user'"'"'s activities including one or more of walking, running, sitting, standing, riding, and sleeping, and the user'"'"'s specifics including one or more of clothing, background, and current weather,

wherein the multiple layers further depict the user'"'"'s emotional state or the user'"'"'s physical state, wherein the emotional state is depicted through voice analysis which is used to determine the user'"'"'s mood including one or more of angry, happy, snoring, tired, sleeping, and hysterical, and

wherein the surrounding information further comprises the user'"'"'s past activities or the user'"'"'s know preferences, wherein the past activities and known preferences include one or more of horseback riding, jogging, and biking.

15. The apparatus of claim 1, wherein the probable status is selected if the assigned confidence factor satisfies the confidence threshold value, and wherein another probable status associated with another confidence factor is selected if the assigned confidence factor fails the confidence threshold value.

16. The apparatus of claim 1, wherein one or more recipients of the presentation at the external device are selected based on one or more factors comprising:

1) the one or more recipients are chosen by the user or an authorized representative on behalf of the user;

     2) the one or more recipients are pre-authorized;

     3) recent or historical notification reception activities relating to the one or more recipients;

     4) the one or more recipients are chosen based on one or more of time of day, location of the user, activities of the user, and a pre-programmed schedule;

     5) one or more individuals communicating with the user are deemed authorized as the one or more recipients.

5. A method comprising:

monitor context awareness and situation awareness relating to a user of the device, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors;

analyzing status awareness relating to the user'"'"'s based on at least one of the context awareness and the situation awareness, wherein the device is further to analyze at least one of external priorities without regard for the status awareness, internal priorities based on the status awareness, and probable status of the user based on a combination of possibility analysis and probability analysis, and an assigned confidence factor as compared to a confidence threshold value;

creating a presentation indicating the user'"'"'s status; and

transmitting the presentation to an external device, wherein the presentation includes a visual presentation having at least one of a graphically-generated static or animated avatar, a picture, and text, wherein one or more potential decisions are made based on assumptions, wherein a confidence level is associated with each assumption, wherein the confidence level is compared against a predefined threshold level to determine whether a graphical image be presented, wherein said analyzing, creating, and transmitting are performed by a communications device, wherein each potential decision comprises a manner to contact the user or not to contact the user and instead perform a different task.

6. The method of claim 5, wherein the context awareness is based on user contexts including one or more of the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user,

wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor, and

wherein the presentation is in a graphics format for display on the external device, wherein the graphics format displays an avatar of the user, wherein said transmitting the presentation is in response to a determination of at least one condition selected from a list of conditions consisting of:

1) an incoming request for communication with the user;

2) an incoming request for the user'"'"'s status;

3) a pre-determined time of day;

4) expiration of a timer,

5) manual initiation of the transmission; and

6) a pre-defined change of the user'"'"'s condition.

7. The method of claim 5, wherein the presentation indicates at least one factor selected from a list consisting of:

1) the user'"'"'s activity;

2) the user'"'"'s location;

3) the user'"'"'s emotional state; and

4) the user'"'"'s physical state, and

wherein contents of the presentation are based at least partly on a condition selected from a list of conditions consisting of;

1) a requestor'"'"'s identity; and

2) a presentation capability of the external device.

13. The method of claim 5, wherein the avatar to indicate one or more of the user'"'"'s activity, the user'"'"'s location, the user status, and the user'"'"'s status with respect to another user or device, wherein the presentation includes a multi-layered graphical image having multiple layers depicting surrounding information relating to the user'"'"'s status, wherein the surrounding information is used by a recipient to base potential decisions,

wherein the avatar further indicates the user'"'"'s activities including one or more of walking, running, sitting, standing, riding, and sleeping, and the user'"'"'s specifics including one or more of clothing,

wherein the multiple layers further depict the user'"'"'s emotional state or the user'"'"'s physical state, wherein the emotional state is depicted through voice analysis which is used to determine the user'"'"'s mood including one or more of angry, happy, snoring, tired, sleeping, and hysterical,

wherein the surrounding information further comprises the user'"'"'s past activities or the user'"'"'s know preferences, wherein the past activities and known preferences include one or more of horseback riding, jogging, and biking.

17. The method of claim 5, wherein the probable status is selected if the assigned confidence factor satisfies the confidence threshold value, and wherein another probable status associated with another confidence factor is selected if the assigned confidence factor fails the confidence threshold value.

18. The method of claim 5, wherein one or more recipients of the presentation at the external device are selected based on one or more factors comprising:

1) the one or more recipients are chosen by the user or an authorized representative on behalf of the user;

     2) the one or more recipients are pre-authorized;

     3) recent or historical notification reception activities relating to the one or more recipients;

     4) the one or more recipients are chosen based on one or more of time of day, location of the user, activities of the user, and a pre-programmed schedule;

     5) one or more individuals communicating with the user are deemed authorized as the one or more recipients.

8. An article comprising

a non-transitory computer-readable storage medium that contains instructions, which when executed by one or more processors result in a device performing one or more operations comprising:

monitor context awareness and situation awareness relating to a user of the device, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors;

analyzing status awareness relating to the user'"'"'s based on at least one of the context awareness and the situation awareness, wherein the device is further to analyze at least one of external priorities without regard for the status awareness, internal priorities based on the status awareness, and probable status of the user based on a combination of possibility analysis and probability analysis, and an assigned confidence factor as compared to a confidence threshold value;

creating a presentation indicating the user'"'"'s status; and

transmitting the presentation to an external device, wherein the presentation includes a visual presentation having at least one of a graphically-generated static or animated avatar, a picture, and text, wherein one or more potential decisions are made based on assumptions, wherein a confidence level is associated with each assumption, wherein the confidence level is compared against a predefined threshold level to determine whether a graphical image be presented, wherein said analyzing, creating, and transmitting are performed by a communications device, wherein each potential decision comprises a manner to contact the user or not to contact the user and instead perform a different task.

9. The article of claim 8, wherein the context awareness is based on user contexts including one or more of the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user,

wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor, and

wherein the presentation is in a graphics format for display on the external device, wherein the graphics format includes an avatar of the user, wherein the operation of transmitting the presentation is in response to a determination of at least one condition selected from a list of conditions consisting of:

1) an incoming request for communication with the user;

2) an incoming request for the user'"'"'s status;

3) a pre-determined time of day;

4) expiration of a timer,

5) manual initiation of the transmission; and

6) a pre-defined change of the user'"'"'s condition.

10. The article of claim 8, wherein the presentation indicates at least one factor selected from a list consisting of:

1) the user'"'"'s activity;

2) the user'"'"'s location;

3) the user'"'"'s emotional state; and

4) the user'"'"'s physical state, and

wherein contents of the presentation are based at least partly on a condition selected from a list of conditions consisting of;

1) a requestor'"'"'s identity; and

2) a presentation capability of the external device.

14. The article of claim 8, wherein the avatar to indicate one or more of the user'"'"'s activity, the user'"'"'s location, the user status, and the user'"'"'s status with respect to another user or device, wherein the presentation includes a multi-layered graphical image having multiple layers depicting surrounding information relating to the user'"'"'s status, wherein the surrounding information is used by a recipient to base potential decisions,

wherein the avatar further indicates the user'"'"'s activities including one or more of walking, running, sitting, standing, riding, and sleeping, and the user'"'"'s specifics including one or more of clothing,

wherein the multiple layers further depict the user'"'"'s emotional state or the user'"'"'s physical state, wherein the emotional state is depicted through voice analysis which is used to determine the user'"'"'s mood including one or more of angry, happy, snoring, tired, sleeping, and hysterical,

wherein the surrounding information further comprises the user'"'"'s past activities or the user'"'"'s know preferences, wherein the past activities and known preferences include one or more of horseback riding, jogging, and biking.

19. The article of claim 8, wherein the probable status is selected if the assigned confidence factor satisfies the confidence threshold value, and wherein another probable status associated with another confidence factor is selected if the assigned confidence factor fails the confidence threshold value.

20. The article of claim 8, wherein one or more recipients of the presentation at the external device are selected based on one or more factors comprising:

1) the one or more recipients are chosen by the user or an authorized representative on behalf of the user;

     2) the one or more recipients are pre-authorized;

     3) recent or historical notification reception activities relating to the one or more recipients;

     4) the one or more recipients are chosen based on one or more of time of day, location of the user, activities of the user, and a pre-programmed schedule;

     5) one or more individuals communicating with the user are deemed authorized as the one or more recipients.