Automated assessment

• US 9,792,655 B2
• Filed: 12/23/2014
• Issued: 10/17/2017
• Est. Priority Date: 12/23/2014
• Status: Active Grant

• Current Assignee: The Travelers Indemnity Company (The Travelers Cos., Inc.)
• Original Assignee: The Travelers Indemnity Company (The Travelers Cos., Inc.)
• Inventors: Griffin, Patrick K., Baldi, Emily N.
• Primary Examiner(s): Azarian, Seyed
• Application Number:
  US14/580,742
• Publication Number:
  US 20160180467A1
• Time in Patent Office: 1,029 Days
• Field of Search: 382100, 382102-103, 382155, 382172-173, 382181, 382190, 382199, 382206, 382209, 382219, 382232, 382254, 382274, 382276, 382286-291, 382305, 382312, 382260, 707812, 707661, 4555561, 705 4
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

A system for automated assessments is provided. The system includes a processing device and a memory device. The memory device stores instructions that when executed by the processing device may result in receiving a video stream including image frames from an imaging source. Image data from one or more of the image frames can be compared with an image profile database to determine whether at least one item in one or more of the image frames is identifiable. A lookup operation can be initiated in a secondary data source to determine one or more item attributes of the one or more identified items. The video stream can be modified to add an item identifier for each of the identified items to the video stream as an annotated video. A record can be stored of the one or more item attributes of the identified items in the annotated video.

Claims

1. A system, comprising:

a processing device; and

a memory device in communication with the processing device, the memory device storing instructions that when executed by the processing device result in;

receiving a video stream comprising a plurality of image frames from an imaging source;

comparing a plurality of image data from one or more of the image frames with an image profile database to determine whether at least one item in one or more of the image frames is identifiable as one or more identified items;

locating a label on at least one of the one or more identified items, the locating comprising;

filtering the image frames to reduce a total number of the image frames analyzed and remove an outer margin of the image frames;

applying a blurring function prior to identifying at least one item after the filtering;

identifying a detailed image search region based on identifying at least one item in a filtered image frame; and

using the detailed image search region to search a non-blurred version of the filtered image frame for the label;

initiating a lookup operation in a secondary data source to determine one or more item attributes of the one or more identified items, wherein the lookup operation in the secondary data source is performed based on the label;

modifying the video stream to add an item identifier for each of the one or more identified items to the video stream as an annotated video; and

storing a record of the one or more item attributes of the one or more identified items in the annotated video.

2. The system of claim 1, further comprising instructions that when executed by the processing device result in:

buffering the video stream as the video stream is received, wherein the modifying of the video stream is performed after the buffering of the video stream and while the video stream is being received.

3. The system of claim 1, further comprising instructions that when executed by the processing device result in:

displaying a message relating to the one or more identified items based on identifying an end of the video stream.

4. The system of claim 1, further comprising instructions that when executed by the processing device result in:

prompting for a user input to add one or more values of user-defined metadata for the one or more identified items.

5. The system of claim 4, wherein initiating the lookup operation in the secondary data source to determine one or more item attributes of the one or more identified items is based on the user-defined metadata.

6. The system of claim 1, wherein the label comprises one or more of:

a barcode, a quick-response code, and an alpha-numeric value.

7. The system of claim 1, further comprising instructions that when executed by the processing device result in:

receiving a location indicator for at least one of the one or more identified items; and

storing the location indicator with the record of the one or more item attributes.

8. The system of claim 7, further comprising instructions that when executed by the processing device result in:

comparing the image data with image data from a previously saved video stream associated with the location indicator to determine whether a change in an aspect of the one or more identified items is detected; and

outputting a notification identifying the change in response to detecting the change.

9. The system of claim 1, further comprising instructions that when executed by the processing device result in:

determining a time/date value associated with one of the image frames that includes at least one of the one or more identified items; and

storing a time/date stamp in the record based on the time/date value.

10. The system of claim 1, wherein the secondary data source is a product value database, and at least one of the one or more item attributes is an estimated price of the one or more identified items.

11. The system of claim 1, further comprising instructions that when executed by the processing device result in:

accessing a list of expected items;

comparing the list of expected items with a list of the one or more identified items; and

outputting a list of missing items based on determining that at least one of the expected items is not included in the list of the one or more identified items.

12. The system of claim 1, further comprising instructions that when executed by the processing device result in:

computing a likelihood score of a predetermined condition based on an identified presence or absence of items associated with the predetermined condition in the one or more identified items;

comparing the likelihood score with a likelihood threshold for the predetermined condition; and

outputting an indication of whether the predetermined condition likely exists based on the comparing.

13. The system of claim 1, further comprising instructions that when executed by the processing device result in:

identifying an image of a purchase receipt in the video stream;

performing character recognition on the purchase receipt to determine whether the purchase receipt indicates a purchase history of at least one of the one or more identified items; and

storing a purchase price identified on the purchase receipt in the record based on the purchase history.

14. The system of claim 1, wherein the imaging source comprises one or more of:

a video camera and a thermal imaging camera.

15. The system of claim 1, further comprising instructions that when executed by the processing device result in:

analyzing an audio component of the video stream using voice recognition; and

storing in the record one or more additional features associated with the one or more identified items using a voice-based narrative extracted from the audio component of the video stream.

16. The system of claim 1, further comprising instructions that when executed by the processing device result in:

capturing the annotated video and the record of the one or more item attributes of the one or more identified items in an insurance underwriting system.

17. A computer program product comprising a non-transitory storage medium embodied with computer program instructions that when executed by a computer cause the computer to implement:

receiving a video stream comprising a plurality of image frames from an imaging source;

comparing a plurality of image data from one or more of the image frames with an image profile database to determine whether at least one item in one or more of the image frames is identifiable as one or more identified items;

locating a label on at least one of the one or more identified items, the locating comprising;

filtering the image frames to reduce a total number of the image frames analyzed and remove an outer margin of the image frames;

applying a blurring function prior to identifying at least one item after the filtering;

identifying a detailed image search region based on identifying at least one item in a filtered image frame; and

using the detailed image search region to search a non-blurred version of the filtered image frame for the label;

initiating a lookup operation in a secondary data source to determine one or more item attributes of the one or more identified items, wherein the lookup operation in the secondary data source is performed based on the label;

modifying the video stream to add an item identifier for each of the one or more identified items to the video stream as an annotated video; and

storing a record of the one or more item attributes of the one or more identified items in the annotated video.

18. The computer program product of claim 17, wherein the label comprises one or more of:

a barcode, a quick-response code, and an alpha-numeric value.

19. The computer program product of claim 17, further comprising computer program instructions that when executed by the computer cause the computer to implement:

receiving a location indicator for at least one of the one or more identified items; and

storing the location indicator with the record of the one or more item attributes.

20. The computer program product of claim 19, further comprising computer program instructions that when executed by the computer cause the computer to implement:

comparing the image data with image data from a previously saved video stream associated with the location indicator to determine whether a change in an aspect of the one or more identified items is detected; and

outputting a notification identifying the change in response to detecting the change.

21. The computer program product of claim 17, further comprising computer program instructions that when executed by the computer cause the computer to implement:

computing a likelihood score of a predetermined condition based on an identified presence or absence of items associated with the predetermined condition in the one or more identified items;

comparing the likelihood score with a likelihood threshold for the predetermined condition; and

outputting an indication of whether the predetermined condition likely exists based on the comparing.