Systems and methods for enhancing and developing accident scene visualizations

• US 10,373,387 B1
• Filed: 04/07/2017
• Issued: 08/06/2019
• Est. Priority Date: 04/07/2017
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Fields, Brian Mark, Assam, Lee Marvin John
• Primary Examiner(s): McCulley, Ryan
• Application Number:
  US15/482,120
• Time in Patent Office: 851 Days
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications

Abstract

Systems and methods are disclosed for enhancing and developing a damage scene virtual reality (VR) visualization. Annotated immersive multimedia image(s) may be received from a first user, where the annotated immersive multimedia image(s) can be associated with a damage scene. A VR visualization of the annotated immersive multimedia image(s) may be rendered using a VR device associated with a second user. The VR visualization may be used to determine a damage amount, where the damage amount is determined from one or more damaged items identifiable in the annotated immersive multimedia image(s).

Claims

1. A visualization system configured for enhancing and developing a damage scene visualization, the visualization system comprising:

a damage scene visualization app configured to visualize, via a first virtual reality (VR) device of a first user, a plurality of immersive multimedia images, each of the plurality of immersive multimedia images captured at or near a location of a damage event and comprising multiple viewing angles of a vantage point defining a damage scene of the damage event,

the damage scene visualization app further configured to accept one or more augmentation commands identifying one or more damage positions associated with damaged items within the damage scene, and

the damage scene visualization app configured for transmission of each of the plurality of immersive multimedia images in a first state and a second state, the first state being a non-annotated state and the second state being an annotated state;

one or more visualization processors operable to visualize immersive multimedia images in VR; and

a second VR device of a second user communicatively coupled to the one or more visualization processors, the visualization processors and the second VR device associated with instructions for rendering immersive multimedia images in VR, the one or more visualization processors configured to;

receive, via a computer network, the plurality of immersive multimedia images from the damage scene visualization app,

determine, for each immersive multimedia image of the plurality of immersive multimedia images, a transmission state of a given multimedia image of the plurality of immersive multimedia images, such that (1) when the transmission state is of the first state, the given immersive multimedia image is augmented by the one or more visualization processors to generate a corresponding VR annotated immersive multimedia image, and (2) when the transmission state is of the second state, the given immersive multimedia image is not augmented by one or more visualization processors, the given immersive multimedia image of the second state previously augmented by the damage scene visualization app to generate the corresponding VR annotated immersive multimedia image,

wherein, for each of the first state and the second state, the corresponding VR annotated immersive multimedia image is augmented such that one or more of the multiple viewing angles of the vantage point of the plurality of immersive multimedia images includes the one or more damage positions associated with the damaged items within the damage scene,

render a visualization comprising the corresponding VR annotated immersive multimedia image and the multiple viewing angles of the vantage point defining the damage scene and the damage event, and

determine a damage amount based on the visualization, wherein the damage amount is determined from one or more damaged items identifiable by the damage positions associated with the damaged items within the damage scene.

2. The visualization system of claim 1, the one or more visualization processors further configured to generate an enhanced immersive multimedia image based on the corresponding VR annotated immersive multimedia image, wherein the corresponding VR annotated immersive multimedia image is augmented with an additional annotation via the second VR device.

3. The visualization system of claim 2, wherein the enhanced immersive multimedia image is a mixed reality immersive multimedia image, and wherein the additional annotation is a graphical annotation, and wherein the mixed reality immersive multimedia image is generated by augmenting, via the second VR device, the corresponding VR annotated immersive multimedia image with the graphical annotation.

4. The visualization system of claim 2, the one or more visualization processors further configured to simultaneously visualize the enhanced immersive multimedia image via the second VR device associated with the second user and the first VR device associated with the first user, wherein the first user is at a location remote from the second user.

5. The visualization system of claim 2, wherein the additional annotation is transmitted from the damage scene visualization app via the computer network to the second VR device.

6. The visualization system of claim 2, the one or more visualization processors further configured to receive a verification indication from the damage scene visualization app, the verification indication indicating that the enhanced immersive multimedia image represents an interpretation of the damage scene as determined by the first user.

7. The visualization system of claim 2, the one or more visualization processors further configured to provide a selectable list of one or more annotations via the damage scene visualization app, wherein selection of a particular annotation from the list of one or more annotations by the first user causes the enhanced immersive multimedia image to be augmented with the particular annotation.

8. The visualization system of claim 1, the visualization system further comprising:

an online application capable of being operated via a web browser, and,

wherein the plurality of immersive multimedia images are capable of being displayed as one or more corresponding two dimensional (2D) images via the online application.

9. The visualization system of claim 1, wherein the second user is one or more of the following:

an insurance claims adjuster or an insurance claims trainee.

10. The visualization system of claim 1, further comprising one or more computer memories, and

wherein the one or more visualization processors are further configured to store in the one or more computer memories the plurality of immersive multimedia images at an initial time and render the visualization at a later time, wherein each of the plurality of immersive multimedia images includes a timestamp value.

11. A computer-implemented visualization method for enhancing and developing a damage scene visualization, the computer-implemented visualization method comprising:

visualizing, via a damage scene visualization app communicatively coupled to a first virtual reality (VR) device of a first user, a plurality of immersive multimedia images, each of the plurality of immersive multimedia images captured at or near a location of a damage event and comprising multiple viewing angles of a vantage point defining a damage scene of the damage event;

accepting, via the damage scene visualization app, one or more augmentation commands identifying one or more damage positions associated with damaged items within the damage scene;

transmitting, via the damage scene visualization app, each of the plurality of immersive multimedia images in a first state and a second state, the first state being a non-annotated state and the second state being an annotated state;

receiving, via a computer network at one or more visualization processors operable to visualize immersive multimedia images in VR, the plurality of immersive multimedia images from the damage scene visualization app;

determining, for each immersive multimedia image of the plurality of immersive multimedia images, a transmission state of a given multimedia image of the plurality of immersive multimedia images, such that (1) when the transmission state is of the first state, the given immersive multimedia image is augmented by the one or more visualization processors to generate a corresponding VR annotated immersive multimedia image, and (2) when the transmission state is of the second state, the given immersive multimedia image is not augmented by one or more visualization processors, the given immersive multimedia image of the second state previously augmented by the damage scene visualization app to generate the corresponding VR annotated immersive multimedia image;

wherein, for each of the first state and the second state, the corresponding VR annotated immersive multimedia image is augmented such that one or more of the multiple viewing angles of the vantage point of the plurality of immersive multimedia images includes the one or more damage positions associated with the damaged items within the damage scene;

rendering, via a second VR device of a second user communicatively coupled to the one or more visualization processors, a visualization comprising the corresponding VR annotated immersive multimedia image and the multiple viewing angles of the vantage point defining the damage scene and the damage event; and

determining a damage amount based on the visualization, wherein the damage amount is determined from one or more damaged items identifiable by the damage positions associated with the damaged items within the damage scene.

12. The computer-implemented visualization method of claim 11, further comprising generating an enhanced immersive multimedia image based on the corresponding VR annotated immersive multimedia image, wherein the corresponding VR annotated immersive multimedia image is augmented with an additional annotation via the second VR device.

13. The computer-implemented visualization method of claim 12, wherein the enhanced immersive multimedia image is a mixed reality immersive multimedia image, and wherein the additional annotation is a graphical annotation, and wherein the mixed reality immersive multimedia image is generated by augmenting, via the second VR device, the corresponding VR annotated immersive multimedia image with the graphical annotation.

14. The computer-implemented visualization method of claim 12, further comprising simultaneously visualizing the enhanced immersive multimedia image via the second VR device associated with the second user and the first VR device associated with the first user, wherein the first user is at a location remote from the second user.

15. The computer-implemented visualization method of claim 12, wherein the additional annotation is transmitted from the damage scene visualization app via the computer network to the second VR device.

16. The computer-implemented visualization method of claim 12, further comprising receiving a verification indication from the damage scene visualization app, the verification indication indicating that the enhanced immersive multimedia image represents an interpretation of the damage scene as determined by the first user.

17. The computer-implemented visualization method of claim 12, further comprising providing a selectable list of one or more annotations via the damage scene visualization app, wherein selection of a particular annotation from the list of one or more annotations by the first user causes the enhanced immersive multimedia image to be augmented with the particular annotation.

18. The computer-implemented visualization method of claim 11, further comprising an online application capable of being operated via a web browser, and,

wherein the plurality of immersive multimedia images are capable of being displayed as one or more corresponding two dimensional (2D) images via the online application.

19. The computer-implemented visualization method of claim 11,

wherein the one or more visualization processors are further configured to store in one or more computer memories the plurality of immersive multimedia images at an initial time and render the visualization at a later time, wherein each of the plurality of immersive multimedia images includes a timestamp value.

20. A tangible, non-transitory computer-readable medium storing instructions for enhancing and developing a damage scene visualization that, when executed by one or more visualization processors operable to visualize immersive multimedia images in virtual reality (VR), cause the one or more visualization processors to:

visualize, via a damage scene visualization app communicatively coupled to a first virtual reality (VR) device of a first user, a plurality of immersive multimedia images, each of the plurality of immersive multimedia images captured at or near a location of a damage event and comprising multiple viewing angles of a vantage point defining a damage scene of the damage event;

accept, via the damage scene visualization app, one or more augmentation commands identifying one or more damage positions associated with damaged items within the damage scene;

transmit, via the damage scene visualization app, each of the plurality of immersive multimedia images in a first state and a second state, the first state being a non-annotated state and the second state being an annotated state;

receive, via a computer network at one or more visualization processors operable to visualize immersive multimedia images in VR, the plurality of immersive multimedia images from the damage scene visualization app;

determine, for each immersive multimedia image of the plurality of immersive multimedia images, a transmission state of a given multimedia image of the plurality of immersive multimedia images, such that (1) when the transmission state is of the first state, the given immersive multimedia image is augmented by the one or more visualization processors to generate a corresponding VR annotated immersive multimedia image, and (2) when the transmission state is of the second state, the given immersive multimedia image is not augmented by one or more visualization processors, the given immersive multimedia image of the second state previously augmented by the damage scene visualization app to generate the corresponding VR annotated immersive multimedia image;

wherein, for each of the first state and the second state, the corresponding VR annotated immersive multimedia image is augmented such that one or more of the multiple viewing angles of the vantage point of the plurality of immersive multimedia images includes the one or more damage positions associated with the damaged items within the damage scene;

render, via a second VR device of a second user communicatively coupled to the one or more visualization processors, a visualization comprising the corresponding VR annotated immersive multimedia image and the multiple viewing angles of the vantage point defining the damage scene and the damage event; and

determine a damage amount based on the visualization, wherein the damage amount is determined from one or more damaged items identifiable by the damage positions associated with the damaged items within the damage scene.