Handheld electronic device using status awareness

• US 8,700,102 B2
• Filed: 07/02/2012
• Issued: 04/15/2014
• Est. Priority Date: 06/16/2009
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ferren, Bran, Nachman, Lama, Del Pasqua, Kieran, March, Wendy, Neumann, John Cross, Shah, Rahul, Shahabdeen, Junaith A., Healey, Jennifer A., Subramanian, Sushmita, Raffa, Giuseppe, Essaian, Alexander, Huang, Jonathan
• Primary Examiner(s): Georgewill, Opiribo
• Application Number:
  US13/539,853
• Publication Number:
  US 20120270575A1
• Time in Patent Office: 652 Days
• Field of Search: 455/414.1, 455/566, 455/432.2, 455/567, 379/88.13, 379/88.14
• US Class Current: 455/567
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G03B 3/00 Focusing arrangements of general interest for cameras, projectors or printers
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/40 by quorum, i.e. whereby two or more security principals are required
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06K 2207/1011 Aiming
• CPC Class Codes: G06K 7/10811 Focalisation
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 20/40145 : Biometric identity checks
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 : using facial parts and geometric relationships
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/45 : Detection of the body part being alive
• CPC Class Codes: H04L 63/083 : using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 2250/52 : including functional features of a camera
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

Through status awareness, a handheld communications device may determine the location, activity, and/or physical or emotional state of the user. This information may in turn be used for various purposes, such as 1) determining how to alert the user of an incoming communication, 2) determining what format to use for communicating with the user, and 3) determining how to present the user'"'"'s status to another person'"'"'s communication device.

Claims

1. An apparatus, comprising:

a handheld wireless communication device having a touchscreen, a processor, a memory, and a radio with a receiver and a transmitter, the device to;

receive an incoming communication through the radio;

monitor context awareness and situation awareness relating to a user of the device, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors; and

automatically select, based on at least one of the context awareness and the situation awareness, a format in which to present the communication to the user of the device, wherein the context awareness is based on user contexts including the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user, wherein automatic selection is performed based on at least one of

external priorities being assigned to the incoming communication without regard for the status awareness of the user,

internal priorities based on the status awareness at a time when the communication is received, and

probable status of the user based on a confidence factor assigned to the probable status and as compared to a confidence threshold value.

2. The apparatus of claim 1, wherein the internal priorities are determined based on the combination of the one or more context factors and the one or more situation factors, wherein the combination includes at least one of the user'"'"'s location at a particular time of day, the particular time of day, the user'"'"'s activities, background sounds, and movement of the device or its proximity to the user when the device is with the user or when the device is away from the user, and

wherein the probable status is selected if the assigned confidence factor satisfies the confidence threshold value, and wherein another probable status associated with another confidence factor is selected if the assigned confidence factor fails the confidence threshold value.

3. The apparatus of claim 1, wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor,

wherein the external priorities are pre-coded into the incoming communication prior to being communicated to the user if a sender is aware of a priority scheme, wherein the external priorities are assigned based on one or more factors including one or more keywords contained within the incoming communication,

wherein the probable status is determined based on a combination of possibility analysis and probability analysis using one or more factors including user activity history, and

wherein a notice comprises a communication that the incoming communication is not to be accepted in its present format and that a different format would be accepted, wherein the format is to comprise at least one technique selected from a list consisting of:

a) interactive text;

b) non-interactive text;

c) interactive voice;

d) non-interactive voice; and

e) graphics.

8. The apparatus of claim 1, wherein the one or more factors relating to the external priorities further include one or more of a sender'"'"'s relationship to the user, a recent history of communications from the sender, and a length of the incoming communication, wherein automatic selection includes deciding, based on the status awareness, on performing one or more of alerting the user immediately or a later point in time, converting the incoming communication from one format to another format, transmitting back to a sender of the incoming communication a notice relating to the incoming communication.

9. The apparatus of claim 1, wherein the format includes alerting techniques that are dynamically modified in accordance with perceived changes in the status awareness, wherein dynamic modification includes switching between two or more alerting techniques or altering a current alerting technique, wherein the alerting techniques include one or more of an audible alert, a spoken alert, a vibrate alert, a light alert, a visual alert including a video alert, and an alert via another device.

10. The apparatus of claim 9, wherein altering the current alerting technique comprises making the current alerting technique more noticeable including one or more of increased sound volume, increased frequency of sound, increased urgency in a spoken voice or ringing sound, brighter display, changing color of light, and rapid occurrence of flashing light or periodic sound.

11. The apparatus of claim 9, wherein the alert via another device comprises switching one or more alerts to a secondary device based on changes in the status awareness suggesting the device is unavailable or occupied, wherein the device is regarded as unavailable if the status awareness suggests that the user away from the device or the device is turned off, and wherein the device is regarded as occupied if the device is being used, wherein the perceived changes in the status awareness that trigger modification in alerting techniques include one or more of device location, ambient noise levels, time of day, user presence, device activity, and calendared priorities.

4. A method comprising:

receiving an incoming communication through a radio of a handheld communications device;

monitoring context awareness and situation awareness relating to a user of the device, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors;

automatically selecting, based on at least one of the context awareness and the situation awareness, a format in which to present the communication to the user of the device, wherein the context awareness is based on user contexts including the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user, wherein automatic selection is performed based on at least one of

external priorities being assigned to the incoming communication without regard for the status awareness of the user,

internal priorities based on the status awareness at a time when the communication is received, and

probable status of the user based on a confidence factor assigned to the probable status and as compared to a confidence threshold value; and

presenting the communication to the user in the selected format.

5. The method of claim 4, wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor,

wherein the external priorities are pre-coded into the incoming communication prior to being communicated to the user if a sender is aware of a priority scheme, wherein the external priorities are assigned based on one or more factors including one or more keywords contained within the incoming communication,

wherein the probable status is determined based on a combination of possibility analysis and probability analysis using one or more factors including user activity history, and

wherein a notice comprises a communication that the incoming communication is to be communicated in a different format, wherein the format comprises at least one technique selected from a list consisting of:

a) interactive text;

b) non-interactive text;

c) interactive voice;

d) non-interactive voice; and

e) graphics.

6. The method of claim 4, wherein the internal priorities are determined based on the combination of the one or more context factors and the one or more situation factors, wherein the combination includes at least one of, the user'"'"'s location at a particular time of day, the particular time of day, the user'"'"'s activities, background sounds, and movement of the device or its proximity to the user when the device is with the user or when the device is away from the user, and

wherein the probable status is selected if the assigned confidence factor satisfies the confidence threshold value, and wherein another probable status associated with another confidence factor is selected if the assigned confidence factor fails the confidence threshold value.

12. The method of claim 4, wherein the one or more factors relating to the external priorities further include one or more of a sender'"'"'s relationship to the user, a recent history of communications from the sender, and a length of the incoming communication, wherein automatic selection includes deciding, based on the status awareness, on performing one or more of alerting the user immediately or a later point in time, converting the incoming communication from one format to another format, transmitting back to a sender of the incoming communication a notice relating to the incoming communication.

13. The method of claim 4, wherein the format includes alerting techniques that are dynamically modified in accordance with perceived changes in the status awareness, wherein dynamic modification includes switching between two or more alerting techniques or altering a current alerting technique, wherein the alerting techniques include one or more of an audible alert, a spoken alert, a vibrate alert, a light alert, a visual alert including a video alert, and an alert via another device.

14. The method of claim 13, wherein altering the current alerting technique comprises making the current alerting technique more noticeable including one or more of increased sound volume, increased frequency of sound, increased urgency in a spoken voice or ringing sound, brighter display, changing color of light, and rapid occurrence of flashing light or periodic sound.

15. The method of claim 13, wherein the alert via another device comprises switching one or more alerts to a secondary device based on changes in the status awareness suggesting the device is unavailable or occupied, wherein the device is regarded as unavailable if the status awareness suggests that the user away from the device or the device is turned off, and wherein the device is regarded as occupied if the device is being used, wherein the perceived changes in the status awareness that trigger modification in alerting techniques include one or more of device location, ambient noise levels, time of day, user presence, device activity, and calendared priorities.

7. A non-transitory computer-readable storage medium that contains instructions, which when executed by one or more processors result in a device performing operations comprising:

receiving an incoming communication through a radio of a handheld communications device;

monitoring context awareness and situation awareness relating to a user of the device, wherein the situation awareness pertains to conditions external to the electronic device as determined from sensor inputs provided by one or more sensors of the electronic device, wherein the sensor inputs are based on information relating to the external conditions gathered by the one or more sensors;

automatically selecting, based on at least one of the context awareness and the situation awareness, a format in which to present the communication to the user of the device, wherein the context awareness is based on user contexts including the user'"'"'s preferences, the user'"'"'s historical activities, the user'"'"'s current activities, and the user'"'"'s speculative activities, wherein the activities include one or more of actions, reactions, and decisions associated with the user, wherein automatic selection is performed based on at least one of

external priorities being assigned to the incoming communication without regard for the status awareness of the user,

internal priorities based on the status awareness at a time when the communication is received, and

probable status of the user based on a confidence factor assigned to the probable status and as compared to a confidence threshold value; and

presenting the communication to the user in the selected format.

16. The non-transitory computer-readable storage medium of claim 7, wherein the internal priorities are determined based on the combination of the one or more context factors and the one or more situation factors, wherein the combination includes at least one of the user'"'"'s location at a particular time of day, the particular time of day, the user'"'"'s activities, background sounds, and movement of the device or its proximity to the user when the device is with the user or when the device is away from the user, and

wherein the probable status is selected if the assigned confidence factor satisfies the confidence threshold value, and wherein another probable status associated with another confidence factor is selected if the assigned confidence factor fails the confidence threshold value.

17. The non-transitory computer-readable storage medium of claim 7, wherein the sensor inputs comprise situational analysis of the user'"'"'s actions that impact the external conditions and correspondingly change the information relating to the external conditions, wherein the user'"'"'s actions are observed by the one or more sensors including one or more of a radio, a camera, a microphone, a motion sensor, an orientation sensor, a light sensor, and a temperature sensor,

wherein the external priorities are pre-coded into the incoming communication prior to being communicated to the user if a sender is aware of a priority scheme, wherein the external priorities are assigned based on one or more factors including one or more keywords contained within the incoming communication,

wherein the probable status is determined based on a combination of possibility analysis and probability analysis using one or more factors including user activity history, and

wherein the one or more factors relating to the external priorities further include one or more of a sender'"'"'s relationship to the user, a recent history of communications from the sender, and a length of the incoming communication, wherein automatic selection includes deciding, based on the status awareness, on performing one or more of alerting the user immediately or a later point in time, converting the incoming communication from one format to another format, transmitting back to a sender of the incoming communication a notice relating to the incoming communication.

18. The non-transitory computer-readable storage medium of claim 7, wherein the format includes alerting techniques that are dynamically modified in accordance with perceived changes in the status awareness, wherein dynamic modification includes switching between two or more alerting techniques or altering a current alerting technique, wherein the alerting techniques include one or more of an audible alert, a spoken alert, a vibrate alert, a light alert, a visual alert including a video alert, and an alert via another device.

19. The non-transitory computer-readable storage medium of claim 18, wherein altering the current alerting technique comprises making the current alerting technique more noticeable including one or more of increased sound volume, increased frequency of sound, increased urgency in a spoken voice or ringing sound, brighter display, changing color of light, and rapid occurrence of flashing light or periodic sound.

20. The non-transitory computer-readable storage medium of claim 18, wherein the alert via another device comprises switching one or more alerts to a secondary device based on changes in the status awareness suggesting the device is unavailable or occupied, wherein the device is regarded as unavailable if the status awareness suggests that the user away from the device or the device is turned off, and wherein the device is regarded as occupied if the device is being used, wherein the perceived changes in the status awareness that trigger modification in alerting techniques include one or more of device location, ambient noise levels, time of day, user presence, device activity, and calendared priorities.