Systems and methods for enhanced situation visualization

• US 10,529,028 B1
• Filed: 06/16/2016
• Issued: 01/07/2020
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Davis, Timothy Joel, Engelhorn, Amy
• Primary Examiner(s): Milef, Elda G
• Application Number:
  US15/184,916
• Time in Patent Office: 1,300 Days
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An augmented reality (AR) system for generating and displaying an enhanced situation visualization (ESV) is provided. The AR system may include an ESV computing device, a user computing device operated by a user and a reference database. The user computing device may transmit a reference request message to the ESV device, the reference request message including an image representative of a current view of a user. The ESV computing device may determine a subject of the image, retrieve reference information associated with the subject from the reference database, and determine situation information specific to the subject. The reference and situation information may be displayed on the user computing device to provide an ESV of the subject. The ESV may be used for insurance-related activities, such as handling, adjusting, and/or generating homeowner'"'"'s insurance claims, and/or for instructional guidance.

Claims

1. An enhanced situation visualization (ESV) computing device comprising a memory and at least one processor communicatively coupled to the memory, wherein the at least one processor is programmed to:

receive a reference request message from a user computing device, the reference request message including an image representative of a current view of a user of the user computing device, the image including a property in a current state;

determine a first subject of the image, wherein the first subject is the property;

retrieve, from a reference database, reference information associated with the first subject of the image, wherein the reference information includes a historical image of the property in a prior state;

determine, based upon the reference request message and the reference information, situation information specific to a situation of the first subject, wherein the situation information includes at least one difference between the prior state and the current state of the property;

generate a virtual grid of the at least one difference between the current state and the previous state;

generate a reference response message including the situation information the reference information, and the virtual grid;

transmit the reference response message to the user computing device, wherein the reference response message causes the user computing device to display at least one of the situation information and the reference information as overlaid upon the current view of the user to provide an enhanced situation visualization of the property, including displaying the virtual grid over the current view of the property as a virtual grid overlay.

2. The ESV computing device of claim 1, wherein the at least one processor is further programmed to:

identify an insurance policy associated with the first subject of the image; and

transmit the reference response message to an insurance server associated with a provider of the insurance policy.

3. The ESV computing device of claim 2, wherein the at least one processor is further programmed to:

receive a claims form from the insurance server;

populate at least one field of the claims form with at least one of the reference information and the situation information; and

transmit the populated claims form to the insurance server.

4. The ESV computing device of claim 1, wherein the reference request message further includes location information, and wherein the at least one processor is further programmed to identify, using the location information, a location of the user computing device prior to determining the subject of the first image.

5. The ESV computing device of claim 1, wherein the at least one processor is further programmed to:

identify a beacon associated with the property, wherein the beacon is coupled to an item of interest associated with the property;

transmit an interrogation signal to the beacon;

determine a location of the beacon relative to the user computing device; and

transmit the location of the beacon relative to the user computing device in the reference response message for display on the user computing device as an icon including an item tag, a direction indicator, and a distance indicator.

6. The ESV computing device of claim 1, wherein the at least one processor is further programmed to:

determine a second subject of the image, wherein the second subject is an object associated with a public utility;

determine a current state of the second subject; and

transmit an indicator of the current state of the second subject in the reference response message for display on the user computing device as an icon.

7. A computer-based method for generating and displaying an enhanced situation visualization (ESV) on a user interface of a user computing device, said method comprising:

receiving, by an ESV computing device, a reference request message from the user computing device, the reference request message including an image representative of a current view of a user of the user computing device, the image including a property in a current state;

determining a first subject of the image, wherein the first subject is the property;

retrieving, from a reference database, reference information associated with the first subject of the image, wherein the reference information includes a historical image of the property in a prior state;

determining, based upon the reference request message and the reference information, situation information specific to a situation of the first subject, wherein the situation information includes at least one difference between the prior state and the current state of the property;

generating a virtual grid of the at least one difference between the current state and the previous state;

generating a reference response message including the situation information, the reference information, and the virtual grid; and

transmitting the reference response message to the user computing device, wherein the reference response message causes the user computing device to display at least one of the situation information and the reference information as overlaid upon the current view of the user to provide an enhanced situation visualization of the property, including displaying the virtual grid over the current view of the property as a virtual grid overlay.

8. The computer-based method of claim 7 further comprising:

identifying an insurance policy associated with the first subject of the image; and

transmitting the reference response message to an insurance server associated with a provider of the insurance policy.

9. The computer-based method of claim 8 further comprising:

receiving a claims form from the insurance server;

populating at least one field of the claims form with at least one of the reference information and the situation information; and

transmitting the populated claims form to the insurance server.

10. The computer-based method of claim 7, wherein the reference request message further includes location information, and wherein determining the first subject of the image further comprises identifying, using the location information, a location of the user computing device.

11. The computer-based method of claim 7 further comprising:

identifying a beacon associated with the property, wherein the beacon is coupled to an item of interest associated with the property;

transmitting an interrogation signal to the beacon;

determining a location of the beacon relative to the user computing device; and

transmitting the location of the beacon relative to the user computing device in the reference response message for display on the user computing device as an icon including an item tag, a direction indicator, and a distance indicator.

12. The computer-based method of claim 7 further comprising:

determining a second subject of the image, wherein the second subject is an object associated with a public utility;

determining a current state of the second subject; and

transmitting an indicator of the current state of the second subject in the reference response message for display on the user computing device as an icon.

13. At least one non-transitory computer-readable storage media having computer-executable instructions embodied thereon, wherein when executed by at least one processor, the computer-executable instructions cause the processor to:

receive a reference request message from a user computing device, the reference request message including an image representative of a current view of a user of the user computing device, the image including a property in a current state;

determine a first subject of the image, wherein the first subject is the property;

retrieve, from a reference database, reference information associated with the first subject of the image, wherein the reference information includes a historical image of the property in a prior state;

determine, based upon the reference request message and the reference information, situation information specific to a situation of the first subject, wherein the situation information includes at least one difference between the prior state and the current state of the property;

generate a virtual grid of the at least one difference between the current state and the previous state;

generate a reference response message including the situation information, the reference information, and the virtual grid; and

transmit the reference response message to the user computing device, wherein the reference response message causes the user computing device to display at least one of the situation information and the reference information as overlaid upon the current view of the user to provide an enhanced situation visualization of the property, including displaying the virtual grid over the current view of the property as a virtual grid overlay.

14. The computer-readable storage media of claim 13, wherein the computer-executable instructions further cause the processor to:

identify an insurance policy associated with the first subject of the image; and

transmit the reference response message to an insurance server associated with a provider of the insurance policy.

15. The computer-readable storage media of claim 14, wherein the computer-executable instructions further cause the processor to:

receive a claims form from the insurance server;

populate at least one field of the claims form with at least one of the reference information and the situation information; and

transmit the populated claims form to the insurance server.

16. The computer-readable storage media of claim 13, wherein the reference request message further includes location information, and wherein the computer-executable instructions further cause the processor to identify, using the location information, a location of the user computing device prior to determining the subject of the first image.

17. The computer-readable storage media of claim 13, wherein the computer-executable instructions further cause the processor to:

identify a beacon associated with the property, wherein the beacon is coupled to an item of interest associated with the property;

transmit an interrogation signal to the beacon;

determine a location of the beacon relative to the user computing device; and

transmit the location of the beacon relative to the user computing device in the reference response message for display on the user computing device as an icon including an item tag, a direction indicator, and a distance indicator.