Multi-mode handheld wireless device to provide data utilizing combined context awareness and situational awareness

• US 8,904,164 B2
• Filed: 01/30/2012
• Issued: 12/02/2014
• Est. Priority Date: 06/16/2009
• Status: Expired due to Fees

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ferren, Bran
• Primary Examiner(s): Lee, Thomas
• Assistant Examiner(s): Prifti, Aurel
• Application Number:
  US13/346,448
• Publication Number:
  US 20130019321A1
• Time in Patent Office: 1,037 Days
• Field of Search: 726/28, 455/35.1, 455/420, 455/567, 713/201, 701/207, 370/339
• US Class Current: 713/100
• CPC Class Codes: G02B 13/0065 having a beam-folding prism or mirror
• CPC Class Codes: G02B 13/009 having zoom function
• CPC Class Codes: G03B 3/00 Focusing arrangements of general interest for cameras, projectors or printers
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/40 by quorum, i.e. whereby two or more security principals are required
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0412 Digitisers structurally integrated in a display
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06K 2207/1011 Aiming
• CPC Class Codes: G06K 7/10811 Focalisation
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 20/40145 : Biometric identity checks
• CPC Class Codes: G06Q 30/02 : Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06V 40/16 : Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/165 : using facial parts and geometric relationships
• CPC Class Codes: G06V 40/19 : Sensors therefor
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/45 : Detection of the body part being alive
• CPC Class Codes: H04L 63/083 : using passwords cryptographic mechanisms or cryptographic arrangements for entity authentication using a predetermined code H04L9/3226
• CPC Class Codes: H04M 1/72448 : with means for adapting the functionality of the device according to specific conditions
• CPC Class Codes: H04M 2250/52 : including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 13/239 : using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 : using three or more 2D image sensors
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/183 : for receiving images from a single remote source

Abstract

Various embodiments of the invention relate to a multi-function handheld user device that may have multiple selectable modes of operation. Various mechanisms may be provided for switching between modes. Within each mode, context awareness and situational awareness may be used to obtain additional information or perform additional functions not directly requested by the user.

Claims

1. A handheld electronic device, the device comprising:

a touch-screen;

a radio for wireless communications;

one or more sensors to determine an environment of the device; and

a memory including a database for the storage of information, wherein the device is operable to provide context awareness based on activities performed by a user with the device, the context awareness including monitoring of a current operational mode of the device, and situational awareness based on sensing of the external environment surrounding the device, the situational awareness including monitoring the external environment using one or more sensors, the device being configured to use combined context awareness and situational awareness to;

monitor a current context of the device, wherein the current context of the device includes a current operational mode and a current activity of the device,

monitor a current surrounding external environment of the device using the one or more sensors,

determine information that is related to the current context of the device and the current external environment of the device,

receive a request by a user to perform a first operation, the request including a request for information,

access information in the database that is not requested by the user and that is not necessary to perform the first operation, the non-requested information including the information that is related to both the current context and the current external environment, and

in addition to providing requested information to the user, provide at least part of the non-requested information to the user in response to the request, wherein a selection of which non-requested information to provide to the user is based on a probability that the non-requested information will be useful to the user, the probability being based on relationships between the requested information and the non-requested information, and wherein the non-requested information that is provided to the user by the device excludes information not accessible to the user in the current operational mode of the device.

2. The device of claim 1, wherein a selection by the device of the non-requested information that is provided to the user is based at least partly on a past history of requests by the user.

3. The device of claim 1, wherein for the determination by the device of information that is related to the current context and the current external environment the device is operable to identify data of external sources, obtain the external data using the radio, and store the external data in the database.

4. The device of claim 1, wherein the one or more sensors include one or more of:

a GPS system to determine physical location;

a radio system to determine wireless networks or devices within communication range;

an image recognition system;

an audio recognition system;

an accelerometer or other sensor to determine movement, orientation, or both of the device;

or a temperature sensor.

5. The device of claim 1, wherein relationships between the requested information and the non-requested information are based on one or more of the user'"'"'s past history, the user'"'"'s recorded preferences, generally accepted behavioral norms, or common human activities.

6. A method, comprising:

monitoring a current external environment of a handheld electronic device, the device providing situational awareness based on sensing of the external environment surrounding the device, the situational awareness including monitoring the external environment using one or more sensors;

monitoring a current context of the handheld electronic device, wherein the current context of the device includes a current operational mode and a current activity of the device, the device to provide context awareness based on activities performed by a user with the device, the context awareness including monitoring of a current operational mode of the device; and

using combined context awareness and situational awareness of the handheld electronic device to provide information to a user of the handheld electronic device by;

determining by the handheld electronic device information that is related to the current context of the device and the current external environment of the device,

performing a first operation in the handheld electronic device, based on a request by a user to perform the first operation, the request including a request for information,

accessing information that is not requested by the user and that is not necessary to perform the first operation, the non-requested information including the information that is related to the current context of the device and the current external environment surrounding the device, and

in addition to providing requested information to the user, providing the non-requested information to the user in response to the request, wherein a selection of which non-requested information to provide to the user is based on a probability that the non-requested information will be useful to the user, the probability being based on relationships between the requested information and the non-requested information, and wherein the information that is provided to the user by the device excludes information not accessible to the user in the current operational mode of the device.

7. The method of claim 6, further comprising operating in each of multiple modes at different times, wherein each mode has a separate set of operational restrictions, each of said sets comprising:

a restriction on accessible Internet sites;

a restriction on available email contacts;

a restriction on data available for viewing.

8. The method of claim 6, wherein determining information that is related to the current context and the current external environment includes the handheld electronic device identifying data of external sources, obtaining the external data, and storing the external data in the database.

9. The method of claim 6, wherein relationships between the requested information and the non-requested information are based on one or more of the user'"'"'s past history, the user'"'"'s recorded preferences, generally accepted behavioral norms, or common human activities.

10. A non-transitory computer-readable storage medium that contains instructions, which when executed by one or more processors result in performing operations comprising:

monitoring a current external environment of a handheld electronic device, the device providing situational awareness based on sensing of the external environment surrounding the device, the situational awareness including monitoring the external environment using one or more sensors;

monitoring a current context of the handheld electronic device, wherein the current context of the device includes a current operational mode and a current activity of the device, the device to provide context awareness based on activities performed by a user with the device, the context awareness including monitoring of a current operational mode of the device; and

using combined context awareness and situational awareness of the handheld electronic device to provide information to a user of the handheld electronic device by;

determining by the handheld electronic device information that is related to the current context of the device the current external environment of the device,

performing a first operation in the handheld electronic device, based on a request by a user to perform the first operation, the request including a request for information,

accessing, based on the request, information not requested by the user and not necessary to perform the first operation, the non-requested information including the information that is related to the current context of the device and the current external environment surrounding the device, and

in addition to providing requested information to the user, providing the non-requested information to the user in response to the request, wherein a selection of which non-requested information to provide to the user is based on a probability that the non-requested information will be useful to the user, the probability being based on relationships between the requested information and the non-requested information, and wherein the information that is provided to the user by the device excludes information not accessible to the user in the current operational mode of the device.

11. The non-transitory computer-readable storage medium of claim 10, further comprising operating, at different times, in each of multiple modes, wherein each mode has a separate set of operational restrictions, each of said sets consisting of:

a restriction on accessible Internet sites;

a restriction on available email contacts;

a restriction on data available for viewing.

12. The non-transitory computer-readable storage medium of claim 10, wherein determining information that is related to the current context and the current external environment includes the handheld electronic device identifying data of external sources, obtaining the external data, and storing the external data in the database.

13. The non-transitory computer-readable storage medium of claim 10, wherein relationships between the requested information and the non-requested information are based on one or more of the user'"'"'s past history, the user'"'"'s recorded preferences, generally accepted behavioral norms, or common human activities.