Scene change detection via multiple sensors

• US 10,169,663 B2
• Filed: 09/01/2016
• Issued: 01/01/2019
• Est. Priority Date: 09/01/2016
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Baker, Anthony W., Klein, Robert J.
• Primary Examiner(s): Baker, Charlotte M
• Application Number:
  US15/254,763
• Publication Number:
  US 20180060670A1
• Time in Patent Office: 852 Days
• Field of Search: 382103, 382236, 382224, 382159, 382207, 348699, 348143, 348154, 348169, 348152, 348156, 348168, 348170, 348171, 348172, 348164, 250330
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06V 10/75 Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

An example method includes determining first classification rule(s) based on first data that classifies pixel groups of first reference image(s) as types of objects. The first reference image(s) are captured by a first sensor. The method further includes determining second classification rule(s) based on second data that classifies pixel groups of second reference image(s) as types of objects. The second reference image(s) are captured by a second sensor. The method includes classifying, as respective first objects, pixel group(s) of a first image of a scene based on the first classification rule(s). The method includes classifying, as respective second objects, pixel group(s) of a second image of the scene based on the second classification rule(s). The method further includes determining whether a change occurred in the scene based on the object classification of the first image(s) and the second image(s).

Claims

1. A method for determining whether a change occurred within a scene, the method comprising:

determining, by a computing device, one or more first classification rules based on accessing one or more first references images and first data that classifies pixel groups of the one or more first reference images as types of objects, wherein the one or more first reference images are captured by a first sensor;

determining, by the computing device, one or more second classification rules based on accessing one or more second reference images and second data that classifies pixel groups of the one or more second reference images as types of objects, wherein the one or more second reference images are captured by a second sensor;

classifying, by the computing device, one or more pixel groups of a first image of the scene as respective first objects based on the one or more first classification rules;

classifying, by the computing device, one or more pixel groups of a second image of the scene as respective second objects based on the one or more second classification rules; and

based on the classification of the one or more pixel groups of the first image as the respective first objects and the classification of the one or more pixel groups of the second image as the respective second objects, determining, by the computing device, whether the change occurred within the scene.

2. The method of claim 1, wherein the one or more first classification rules classify the pixel groups of the one or more first reference images as types of objects in accordance with the first data.

3. The method of claim 1, wherein the one or more second classification rules classify the pixel groups of the one or more second reference images as types of objects in accordance with the second data.

4. The method of claim 1, wherein the first image of the scene is captured by the first sensor.

5. The method of claim 1, wherein the second image of the scene is captured by the second sensor.

6. The method of claim 1, wherein the first sensor comprises a first red-green-blue (RGB) sensor and the second sensor comprises a second RGB sensor.

7. The method of claim 1, wherein the first sensor comprises a first infrared sensor and the second sensor comprises a second infrared sensor.

8. The method of claim 1, wherein the first sensor comprises an infrared sensor and the second sensor comprises a red-green-blue (RGB) sensor.

9. The method of claim 1, wherein the first sensor comprises a red-green-blue (RGB) sensor and the second sensor comprises an infrared sensor.

10. The method of claim 1, wherein the first data further classifies the pixel groups of the one or more first reference images as particular respective objects.

11. The method of claim 1, wherein the second data further classifies the pixel groups of the one or more second reference images as particular respective objects.

12. The method of claim 1, further comprising:

using the one or more first classification rules and the one or more second classification rules to identify an object that is depicted in both the first image of the scene and the second image of the scene,

wherein determining whether the change occurred within the scene comprises using the identified object as a reference for comparing the first image of the scene with the second image of the scene.

13. The method of claim 1,

wherein the first image of the scene is captured before the second image of the scene, and

wherein determining whether the change occurred within the scene comprises determining that an object that is depicted in the first image of the scene is not depicted in the second image of the scene.

14. The method of claim 1,

wherein the first image of the scene is captured before the second image of the scene, and

wherein determining whether the change occurred within the scene comprises determining that an object that is depicted in the second image of the scene is not depicted in the first image of the scene.

15. The method of claim 1, wherein determining whether the change occurred within the scene comprises determining that an object that is depicted in the first image of the scene at a first location is depicted in the second image of the scene at a second location that is distinct from the first location.

16. The method of claim 1, wherein at least one of (i) the one or more first reference images, (ii) the one or more second references images, (iii) the first image of the scene, or (iv) the second image of the scene are non-spectral images.

17. A non-transitory computer readable medium storing instructions that, when executed by a computing device, cause the computing device to perform functions for determining whether a change occurred within a scene, the functions comprising:

determining one or more first classification rules based on accessing one or more first references images and first data that classifies pixel groups of the one or more first reference images as types of objects, wherein the one or more first reference images are captured by a first sensor;

determining one or more second classification rules based on accessing one or more second reference images and second data that classifies pixel groups of the one or more second reference images as types of objects, wherein the one or more second reference images are captured by a second sensor;

classifying, as respective first objects, one or more pixel groups of a first image of the scene based on the one or more first classification rules;

classifying, as respective second objects, one or more pixel groups of a second image of the scene based on the one or more second classification rules; and

based on the classification of the one or more pixel groups of the first image as the respective first objects and the classification of the one or more pixel groups of the second image as the respective second objects, determining whether the change occurred within the scene.

18. The non-transitory computer readable medium of claim 17,

wherein the first image of the scene is captured before the second image of the scene, and

wherein determining whether the change occurred within the scene comprises determining that an object that is depicted in the first image of the scene is not depicted in the second image of the scene.

19. A computing device comprising:

one or more processors; and

a non-transitory computer readable medium storing instructions that, when executed by the one or more processors, cause the computing device to perform functions for determining whether a change occurred within a scene, the functions comprising;

determining one or more first classification rules based on accessing one or more first references images and first data that classifies pixel groups of the one or more first reference images as types of objects, wherein the one or more first reference images are captured by a first sensor;

determining one or more second classification rules based on accessing one or more second reference images and second data that classifies pixel groups of the one or more second reference images as types of objects, wherein the one or more second reference images are captured by a second sensor;

classifying, as respective first objects, one or more pixel groups of a first image of the scene based on the one or more first classification rules;

classifying, as respective second objects, one or more pixel groups of a second image of the scene based on the one or more second classification rules; and

based on the classification of the one or more pixel groups of the first image as the respective first objects and the classification of the one or more pixel groups of the second image as the respective second objects, determining whether the change occurred within the scene.

20. The computing device of claim 19,

wherein the first image of the scene is captured before the second image of the scene, and

wherein determining whether the change occurred within the scene comprises determining that an object that is depicted in the second image of the scene is not depicted in the first image of the scene.