Computer system and method for classifying temporal patterns of change in images of an area

• US 10,255,526 B2
• Filed: 06/09/2017
• Issued: 04/09/2019
• Est. Priority Date: 06/09/2017
• Status: Active Grant

• Current Assignee: Uptake Technologies, Inc.
• Original Assignee: Uptake Technologies, Inc.
• Inventors: Gandenberger, Greg
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/618,400
• Publication Number:
  US 20180357517A1
• Time in Patent Office: 669 Days
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/13 Satellite images
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal

Abstract

A set of successive images for a given area may comprise a first, second, and third images representing different times. Comparisons may be performed between the first image and the second image, the first image and the third image, and the second image and the third image. The respective outputs of the three pairwise comparisons may be evaluated to identify any sub-areas of the given area where one or more temporal patterns of change have occurred. For instance, any sub-area of the given area that exhibits a change between the first and second images, a change between the second and third images, and an absence of change between the second and third images may be identified as a persistent change. An indication that the temporal pattern of change has occurred at each identified sub-area of the given area may then be output to a client station.

Claims

1. A method comprising:

receiving, at a computing system configured to perform image-data analysis, a set of successive images for a given area from an image data source via a communication network, wherein the set of successive images comprises a first image captured at a first time, a second image captured at a second time, and third image captured at a third time, wherein the given area comprises a plurality of sectors, and wherein each image in the set of successive images is defined by a respective set of image data;

for each respective sector of the plurality of sectors of the given area;

performing, by the computing system, a first image-data analysis of the first and second images that involves (i) comparing a respective portion of the first image that corresponds to the respective sector with a respective portion of the second image that corresponds to the respective sector and (ii) based on the comparing, determining a first likelihood of change between the respective portion of the first image that corresponds to the respective sector and the respective portion of the second image that corresponds to the respective sector;

performing, by the computing system, a second image-data analysis of the first and third images that involves (i) comparing the respective portion of the first image that corresponds to the respective sector with a respective portion of the third image that corresponds to the respective sector and (ii) based on the comparing, determining a second likelihood of change between the respective portion of the first image that corresponds to the respective sector and the respective portion of the third image that corresponds to the respective sector;

performing, by the computing system, a third image-data analysis of the second and third images that involves (i) comparing the respective portion of the second image that corresponds to the respective sector with the respective portion of the third image that corresponds to the respective sector and (ii) based on the comparing, determining a third likelihood of change between the respective portion of the second image that corresponds to the respective sector and the respective portion of the third image that corresponds to the respective sector; and

evaluating the first, second, and third likelihood of change for the respective sector to classify the respective sector as exhibiting a given one of two or more different types of changes, wherein the evaluating comprises;

comparing the first likelihood of change to a first threshold, the second likelihood of change to a second threshold, and the third likelihood of change to a third threshold; and

classifying the respective sector as exhibiting a persistent type of change if the first likelihood of change exceeds the first threshold, the second likelihood of change exceeds the second threshold, and the third likelihood of change does not exceed the third threshold; and

outputting, to a client station via the communication network, data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change.

2. The method of claim 1, wherein the evaluating further comprises:

classifying the respective sector as exhibiting an ongoing type of change if the first likelihood of change exceeds the first threshold, the second likelihood of change exceeds the second threshold, and the third likelihood of change exceeds the third threshold.

3. The method of claim 1, wherein the evaluating further comprises:

classifying the respective sector as exhibiting a reverting type of change if the first likelihood of change exceeds the first threshold, the second likelihood of change falls below the second threshold, and the third likelihood of change exceeds the third threshold.

4. The method of claim 1, wherein the respective set of image data defining each of the first image, the second image, and the third image is organized into pixel blocks that each correspond to a respective sector of the given area, and wherein for each respective sector of the given area:

performing the first image-data analysis involves comparing the pixel block from the first image that corresponds to the respective sector to a complementary pixel block from the second image that corresponds to the respective sector and thereby determining the first likelihood of change for the respective sector,

performing the second image-data analysis involves comparing the pixel block from the first image that corresponds to the respective sector to a complementary pixel block from the third image that corresponds to the respective sector and thereby determining the second likelihood of change for the respective sector, and

performing the third image-data analysis involves comparing the pixel block from the second image that corresponds to the respective sector to the complementary pixel block from the third image that corresponds to the respective sector and thereby determining a first likelihood of change for the respective sector.

5. The method of claim 1, wherein the data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change comprises location data for the at least one sector.

6. The method of claim 1, wherein the data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change is included as part of an array of indicators that each correspond to a respective sector of the plurality of sectors of the given area, and wherein each indicator in the array of indicators provides an indication of whether the respective sector has been classified as exhibiting a persistent type of change.

7. The method of claim 1, further comprising:

instructing the client station to display a map of the given area that includes a visual representation of the data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change.

8. The method of claim 1, further comprising:

instructing the client station to display a map of the given area that includes a visual representation of the data indicating that a persistent type of change has occurred at the at least one sub-area of the given area.

9. A method comprising:

receiving, at a computing system configured to perform image-data analysis, a set of successive images for a given area from an image data source via a communication network, wherein the set of successive images comprises a first image captured at a first time, a second image captured at a second time, and third image captured at a third time, wherein each image in the set of successive images is defined by a respective set of image data;

performing, by the computing system, a first image-data analysis of the first image captured at the first time and the second image captured at the second time that involves (i) comparing the respective set of data defining the first image to the respective set of data defining the second image and (ii) based on the comparing, detecting any sub-area of the given area that exhibits a change between the first and second images;

performing, by the computing system, a second image-data analysis of the first image captured at the first time and the third image captured at the third time that involves (i) comparing the respective set of data defining the first image to the respective set of data defining the third image and (ii) based on the comparing, detecting any sub-area of the given area that exhibits a change between the first and third images;

performing, by the computing system, a third image-data analysis of the second image captured at the second time and the third image captured at the third time that involves (i) comparing the respective set of data defining the second image to the respective set of data defining the third image and (ii) based on the comparing, detecting any sub-area of the given area that exhibits an absence of change between the second and third images;

based on the first, second, and third image-data analyses, identifying at least one sub-area of the given area that exhibits a change between the first and second images, a change between the second and third images, and an absence of change between the second and third images;

classifying the at least one sub-area of the given area as a sub-area that exhibits a persistent type of change; and

outputting, to a client station via the communication network, data indicating that a persistent type of change has occurred at the at least one sub-area of the given area.

10. The method of claim 9, further comprising:

before receiving the set of successive images for the given area from the image data source, (a) receiving, from the client station, a request to identify any sub-area of the given area that exhibits a persistent type of change and (b) in response to receiving the request from the client station, transmitting, to the image data source, a request for the set of successive images for the given area.

11. The method of claim 9, wherein:

detecting any sub-area of the given area that exhibits a change between the first and second images comprises, for each respective sub-area of a plurality of sub-areas of the given area, (a) determining a respective first likelihood of change between a respective portion of the first image that corresponds to the respective sub-area and a respective portion of the second image that corresponds to the respective sub-area and (b) determining whether the respective first likelihood of change exceeds a first threshold;

detecting any sub-area of the given area that exhibits a change between the first and third images comprises, for each respective sub-area of the plurality of sub-areas of the given area, (a) determining a respective second likelihood of change between the respective portion of the first image that corresponds to the respective sub-area and a respective portion of the third image that corresponds to the respective sub-area and (b) determining whether the respective second likelihood of change exceeds a second threshold; and

detecting an absence of any change between the second and third images comprises, for each of the plurality of sub-areas of the given area, (a) determining a respective third likelihood of change between the respective portion of the second image that corresponds to the respective sub-area and the respective portion of the third image that corresponds to the respective sub-area and (b) determining whether the respective third likelihood of change falls below a third threshold.

12. The method of claim 11, wherein identifying the at least one sub-area of the given area that exhibits a change between the first and second images, a change between the second and third images, and an absence of change between the second and third images comprises identifying at least one sub-area having a respective first likelihood of change that exceeds the first threshold, a respective second likelihood of change that exceeds the second threshold, and a respective third likelihood of change that falls below the third threshold.

13. The method of claim 9, wherein the data indicating that a persistent type of change has occurred at the at least one sub-area of the given area comprises location data for the at least one sub-area.

14. The method of claim 9, wherein the data indicating that a persistent type of change has occurred at the at least one sub-area of the given area is included as part of an array of indicators that each correspond to a respective sub-area of the given area, and wherein each indicator in the array of indicators provides an indication of whether the respective sub-area has been classified as exhibiting a persistent type of change.

15. A computing system comprising:

a network interface configured to communicatively couple the computing system to (a) at least one image data source and (b) at least one client station;

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

receive a set of successive images for a given area from an image data source via the network interface, wherein the set of successive images comprises a first image captured at a first time, a second image captured at a second time, and third image captured at a third time, wherein the given area comprises a plurality of sectors, and wherein each image in the set of successive images is defined by a respective set of image data;

for each respective sector of the given area;

perform a first image-data analysis of the first and second images that involves (i) comparing a respective portion of the first image that corresponds to the respective sector with a respective portion of the second image that corresponds to the respective sector and (ii) based on the comparing, determining a first likelihood of change between the first image that corresponds to the respective sector and the second image that corresponds to the respective sector;

perform a second image-data analysis of the first and third images that involves (i) comparing the respective portion of the first image that corresponds to the respective sector with a respective portion of the third image that corresponds to the respective sector and (ii) based on the comparing, determining a second likelihood of change between the respective portion of the first image that corresponds to the respective sector and the respective portion of the third image that corresponds to the respective sector;

perform a third image-data analysis of the second and third images that involves (i) comparing the respective portion of the second image that corresponds to the respective sector with the respective portion of the third image that corresponds to the respective sector and (ii) based on the comparing, determining a third likelihood of change between the respective portion of the second image that corresponds to the respective sector and the respective portion of the third image that corresponds to the respective sector;

evaluate the first, second, and third likelihood of change for the respective sector of the given area to classify the respective sector as exhibiting a given one of two or more different types of changes, wherein the evaluation comprises;

comparing the first likelihood of change to a first threshold, the second likelihood of change to a second threshold, and the third likelihood of change to a third threshold; and

classifying the respective sector as exhibiting a persistent type of change if the first likelihood of change exceeds the first threshold, the second likelihood of change exceeds the second threshold, and the third likelihood of change does not exceed the third threshold; and

output, to a client station via the network interface, data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change.

16. The computer system of claim 15, wherein the evaluation further comprises:

classifying the respective sector as exhibiting an ongoing type of change if the first likelihood of change exceeds the first threshold, the second likelihood of change exceeds the second threshold, and the third likelihood of change exceeds the third threshold.

17. The computer system of claim 15, wherein the evaluation further comprises:

classifying the respective sector as exhibiting a reverting type of change if the first likelihood of change exceeds the first threshold, the second likelihood of change falls below the second threshold, and the third likelihood of change exceeds the third threshold.

18. The computer system of claim 15, wherein the data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change is included as part of an array of indicators that each correspond to a respective sector of the plurality of sectors of the given area, and wherein each indicator in the array of indicators provides an indication of whether the respective sector has been classified as exhibiting a persistent type of change.

19. The computer system of claim 15, further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

instruct the client station to display a map of the given area that includes a visual representation of the data indicating that at least one sector of the given area has been classified as exhibiting a persistent type of change.

20. A computing system comprising:

a network interface configured to communicatively couple the computing system to (a) at least one image data source and (b) at least one client station;

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

receive a set of successive images for a given area from an image data source via the network interface, wherein the set of successive images comprises a first image captured at a first time, a second image captured at a second time, and third image captured at a third time, wherein each image in the set of successive images is defined by a respective set of image data;

perform a first image-data analysis of the first image captured at the first time and the second image captured at the second time that involves (i) comparing the respective set of data defining the first image to the respective set of data defining the second image and (ii) based on the comparing, detecting any sub-area of the given area that exhibits a change between the first and second images;

perform a second image-data analysis of the first image captured at the first time and the third image captured at the third time that involves (i) comparing the respective set of data defining the first image to the respective set of data defining the third image and (ii) based on the comparing, detecting any sub-area of the given area that exhibits a change between the first and third images;

perform a third image-data analysis of the second image captured at the second time and the third image captured at the third time that involves (i) comparing the respective set of data defining the second image to the respective set of data defining the third image and (ii) based on the comparing, detecting any sub-area of the given area that exhibits an absence of change between the second and third images;

based on the first, second, and third image-data analyses, identify at least one sub-area of the given area that exhibits a change between the first and second images, a change between the second and third images, and an absence of change between the second and third images;

classify the at least one sub-area of the given area as a sub-area that exhibits a persistent type of change; and

output, to a client station via the network interface, data indicating that a persistent type of change has occurred at the at least one sub-area of the given area.