Personalized situation awareness using human emotions and incident properties

• US 10,074,368 B2
• Filed: 08/17/2016
• Issued: 09/11/2018
• Est. Priority Date: 08/17/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ogunyoku, Temitope, Weldemariam, Komminist
• Primary Examiner(s): Singh, Satwant
• Application Number:
  US15/239,156
• Publication Number:
  US 20180053503A1
• Time in Patent Office: 755 Days
• Field of Search: None
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G08B 21/18 Status alarms G08B21/02 takes precedence
• CPC Class Codes: G08B 25/016 Personal emergency signalling and security systems emergency non-personal manually actuated alarm activators G08B25/12
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/72 for transmitting results of analysis
• CPC Class Codes: H04W 4/90 Services for handling of emergency or hazardous situations, e.g. earthquake and tsunami warning systems [ETWS]

Abstract

The disclosure provides systems and methods for generating and delivering situational-dependent incident responses. An incident response is generated based on incident characterizations as well as emotional data pertaining to a reporting person. The generated response may also involve an interactive component in order to obtain further situational details from the reporting person and other sources, thereby further refine the response and recommendations. Persons reporting incidents are aided by appropriate recommendations and assistance from appropriate service providers.

Claims

1. A method comprising:

receiving an incident report from a user device, the incident report pertaining to an incident involving a target;

obtaining, from at least one sensor of the user device, emotional data pertaining to a reporting person using the user device, wherein the emotional data includes at least one of heat, heartbeat, conductivity, text, speech, image, and video data;

processing the emotional data to determine an emotional status of the reporting person, wherein processing the emotional data includes at least one of face detection, voice/speech analytics, and text analytics;

processing the incident report to determine an incident type, incident category, and incident characteristic; and

transmitting an initiation dialog sequence to the user device, the initiation dialog sequence configured to initiate on the user device a cognitive dialog module that prompts a further interaction with the reporting person, wherein the further interaction is based at least in part on the emotional status of the reporting person and at least in part on the incident characteristics.

2. The method of claim 1, further comprising initiating a visible or audible alert on an emergency service provider device based at least in part on the processed incident report and at least in part on the emotional data.

3. The method of claim 1, further comprising receiving, by a system discrete from the user device, situation data pertaining to the incident, wherein the system provides the initiation dialog sequence in response to the situation data.

4. The method of claim 1, further comprising receiving, by a system discrete from the user device, situation data pertaining to the incident, wherein the situation data is from a source selected from the user device and an external source.

5. The method of claim 1, wherein the further interaction is selected from an audio interaction, a text interaction, a visual interaction, a gesture interaction, and a combination thereof.

6. The method of claim 1, wherein the further interaction is further based on identity data of the reporting person.

7. The method of claim 1, further comprising initiating the cognitive dialog module on the user device based on the initiation dialog sequence or based on an interaction pattern of the reporting person.

8. The method of claim 1, further comprising:

initiating the cognitive dialog module on the user device based on the initiation dialog sequence;

receiving, by the cognitive dialog module, additional data selected from further emotional data pertaining to the reporting person and situation data; and

customizing the further interaction based on the additional data.

9. The method of claim 1, wherein the further interaction comprises communication of data and instructions with a system discrete from the user device to further customize the further interaction.

10. A system comprising:

a processor; and

a memory coupled to the processor, the memory configured to store program instructions for instructing the processor to carry out the method of claim 1.

11. The system of claim 10, wherein the system comprises:

a reasoning module; and

an advisory module.

12. The system of claim 10, wherein the system comprises a communications module for communicating via a network with a cognitive dialog module on a user device.

13. The system of claim 10, wherein the system comprises a communications module for communicating via a network with a cognitive dialog module on a user device, the cognitive dialog module comprising natural language processing (NLP) and visual analytics software utility configured to analyze input from the reporting person and determine a response based on the input.

14. The system of claim 10, wherein the system comprises a communications module for communicating via a network with a cognitive dialog module on a user device, the cognitive dialog module comprising a linguistics analyzer component configured to process the language, dialect, or slang of the reporting person.

15. A user device comprising:

at least one sensor collecting emotional data from a reporting person, wherein the emotional data includes at least one of heat, heartbeat, text, speech, image, and video data;

a reasoning module implementing at least one of face detection, voice/speech analytics, and text analytics to infer an emotional status of the reporting person;

a communications module communicating the emotional data, the emotional status, and an incident report to a server; and

a cognitive dialog module implementing a metaphor generator to prompt a further interaction with the reporting person.

16. The user device of claim 15, wherein the cognitive dialog module implements the metaphor generator in response to an initiation dialog sequence received from the server via the communications module.

17. The user device of claim 15, wherein the further interaction is selected from an audio interaction, a text interaction, a visual interaction, a gesture interaction, and a combination thereof.

18. The user device of claim 15, wherein the further interaction is selected from a situation awareness dialog and a question and answer dialog.

19. The user device of claim 15, wherein the cognitive dialog module implements the metaphor generator in response to an interaction pattern of the reporting person.

20. The user device of claim 15, wherein the metaphor generator takes into consideration user cohort based on historical data.