Situation aware personal assistant

• US 10,474,946 B2
• Filed: 06/24/2016
• Issued: 11/12/2019
• Est. Priority Date: 06/24/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Joo, Tae Hong, Elabbady, Tarek Z., Habib, Mona Soliman
• Primary Examiner(s): Cassity, Robert A
• Assistant Examiner(s): Seck, Ababacar
• Application Number:
  US15/192,880
• Publication Number:
  US 20170372189A1
• Time in Patent Office: 1,236 Days
• Field of Search: 706 11
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/654 the user being the driver
• CPC Class Codes: B60W 2520/105 Longitudinal acceleration
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 40/06 Road conditions
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: H04M 1/724098 Interfacing with an on-board device of a vehicle for hands-free speaking in vehicles H04M1/6075
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72463 to restrict the functionality of the device

Abstract

Methods, systems, apparatuses, and computer program products are provided for altering the behavior of an electronic personal assistant based on a situation associated with a mobile device. A situation is sensed with a plurality of sensors to generate sensor data. A situation score is calculated based on the sensor data. Behavior of an electronic personal assistant is altered based on the calculated situation score. In one aspect, the situation is a driving situation in which a driver drives a vehicle on a roadway. In such case, a driving situation score is calculated based on the sensor data, and behavior of the electronic personal assistant is altered based on the calculated driving situation score, such as suspending interactions by the electronic personal assistant with the driver to avoid the driver being distracted.

Claims

1. A method in at least one computing device, comprising:

executing in the computing device an application that is an electronic personal assistant configured to interact with a user by speech communication including being configured to receive a spoken request and to respond with speech;

sensing a situation with a plurality of sensors to generate sensor data;

calculating a situation score based on the sensor data; and

altering behavior of the electronic personal assistant based on the calculated situation score including suspending an action from being performed by the electronic personal assistant.

2. The method of claim 1, wherein the situation is a driving situation in which a driver drives a vehicle on a roadway, said calculating comprising:

calculating a driving situation score based on the sensor data; and

said altering comprises;

altering behavior of the electronic personal assistant based on the calculated driving situation score.

3. The method of claim 2, wherein said sensing comprises:

sensing at least one aspect of the driver.

4. The method of claim 2, wherein said sensing comprises:

sensing at least one aspect of the vehicle.

5. The method of claim 2, wherein said sensing comprises

sensing at least one aspect of an environment of the roadway.

6. The method of claim 2, wherein said altering comprises:

suspending interaction by the electronic personal assistant with the driver on condition that the driving situation score has a predetermined value to avoid the driver being distracted by the electronic personal assistant.

7. The method of claim 6, wherein said suspending interaction comprises:

delaying at least one of verbal or visual communications with the driver.

8. The method of claim 2, wherein said altering comprises:

providing at least one of verbal or visual assistance to the driver.

9. At least one computing device, comprising:

a situation monitor configured to calculate a situation score based on sensor data received from a plurality of sensors sensing a situation; and

an application that is an electronic personal assistant configured to;

interact with a user by speech communication including being configured to receive a spoken request and to respond with speech; and

alter behavior based on the calculated situation score including being configured to suspend an action from being performed by the electronic personal assistant.

10. The at least one computing device of claim 9, wherein the situation is a driving situation in which a driver drives a vehicle on a roadway and the situation monitor includes a driving situation monitor configured to calculate a driving situation score based on the sensor data.

11. The at least one computing device of claim 10, wherein the sensors comprise at least one of:

a first sensor configured to sense at least one aspect of the driver;

a second sensor configured to sense at least one aspect of the vehicle;

or

a third sensor configured to sense at least one aspect of an environment of the roadway.

12. The at least one computing device of claim 10, wherein the electronic personal assistant is configured to alter behavior by suspending interaction by the electronic personal assistant with the driver on condition that the driving situation score has a predetermined value, thereby avoiding the driver being distracted by the electronic personal assistant.

13. The at least one computing device of claim 10, further comprising:

a mobile user device that implements the driving situation monitor and the electronic personal assistant.

14. The at least one computing device of claim 10, further comprising:

a mobile user device that implements the electronic personal assistant; and

the driving situation monitor is implemented by a processor circuit of the vehicle.

15. The at least one computing device of claim 10, further comprising:

a processor of the vehicle that implements the driving situation monitor and the electronic personal assistant.

16. A computing device, comprising:

at least one processor circuit; and

a memory that stores program code configured to be executed by the at least one processor circuit to perform operations, the operations including;

executing an application that is an electronic personal assistant configured to interact with a user by speech communication including being configured to receive a spoken request and to respond with speech;

calculating a situation score based on sensor data received from a plurality of sensors sensing a situation; and

causing a behavior of the electronic personal assistant to be altered based on the calculated situation score including suspending an action from being performed by the electronic personal assistant.

17. The computing device of claim 16, wherein the situation is a driving situation in which a driver drives a vehicle on a roadway, said calculating comprising:

calculating a driving situation score based on the sensor data; and

said causing comprises;

causing the behavior of the electronic personal assistant to be altered based on the calculated driving situation score.

18. The computing device of claim 17, wherein said sensing comprises at least one of:

sensing at least one aspect of the driver;

sensing at least one aspect of the vehicle;

or

sensing at least one aspect of an environment of the roadway.

19. The computing device of claim 17, wherein said causing comprises:

causing the electronic personal assistant to suspend interaction with the driver on condition that the driving situation score has a predetermined value to avoid the driver being distracted by the electronic personal assistant.

20. The computing device of claim 19, wherein said causing the electronic personal assistant to suspend interaction with the driver comprises:

causing the electronic personal assistant to delay at least one of verbal or visual communications with the driver.