Cognitive change detection system

• US 7,571,051 B1
• Filed: 01/06/2006
• Issued: 08/04/2009
• Est. Priority Date: 01/06/2005
• Status: Active Grant

• Current Assignee: William E. Knight
• Original Assignee: Doubleshot, Inc.
• Inventors: Shulman, Alan
• Primary Examiner(s): NGUYEN, TAN QUANG
• Application Number:
  US11/326,821
• Time in Patent Office: 1,306 Days
• Field of Search: 701/213, 701/211, 701/200, 701/208, 701/205, 701/210, 340/995.22, 348/142, 348/129, 348/130, 348/135
• US Class Current: 701/213
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2219/028 Multiple view windows (top-side-front-sagittal-orthogonal)
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A method of detecting a changed condition within a geographical space from a moving vehicle. Images of that geographic space are memorialized in conjunction with GPS coordinates together with its GPS coordinates. The same geographic space is traversed from the moving vehicle while accessing the route'"'"'s GPS coordinates. The memorialized images are played back by coordinating the GPS data on a memorialized images with that of the traversed geographic space such that the memorialized images are viewed simultaneously with the geographic space being traversed. An observer traveling within the moving vehicle can compare the memorialized images with those being traversed in order to identify changed conditions.

Claims

1. A method for displaying a changed condition within a field of view at a display, the method comprising:

providing a system, wherein the system comprises a computer processor, a database, and a display;

receiving, by the computer processor, a first set of image data and geo-referenced data of the field of view at a location along a travel route, wherein the first set of image data represents a current view;

retrieving, from the database, a second set of image data based on the geo-referenced data, the second set of image data and the geo-referenced data stored within the database according to a distance traveled along the travel route and captured at a rate independent of a rate of being stored within the database;

displaying, at the display, the first set of image data and the second set of image data, synchronized by the computer processor based on the geo-referenced data; and

identifying, via the system, the changed condition within the field of view based on the first and the second set of image data.

2. The method of claim 1, wherein the current view comprises the field of view from an image acquisition device associated with a vehicle.

3. The method of claim 2, wherein the vehicle is an unmanned aerial vehicle, a manned aircraft, or a robotically-controlled vehicle.

4. The method of claim 2, further comprising:

capturing the first set of image data representing the current view from the image acquisition device at a frame rate dependent on a velocity of the image acquisition device.

5. The method of claim 2, wherein the image acquisition device is one of a video or a thermal camera.

6. The method of claim 1, wherein the first set of image data is reduced in visibility compared to the second set of image data.

7. The method of claim 1, wherein the second set of image data is stored in JPEG format.

8. The method of claim 1, wherein the second set of image data is discrete raster-based imagery based on one or more of video imagery, computer-generated imagery, multi-spectral imagery, thermal imagery, magnetic imagery, radar, ladar, or sonar.

9. The method of claim 1, wherein the geo-referenced data includes one or more of GPS coordinates, elevation, or heading information associated with the field of view provided by an image acquisition device.

10. The method of claim 1, wherein the geo-referenced data includes data received from one or more of inertial sensors, inclinometers, accelerometers, digital compasses, or clocks.

11. The method of claim 1, wherein the geo-referenced data includes information representing a velocity of an image acquisition device capturing the first set of image data.

12. The method of claim 1, wherein the system further comprises auxiliary lighting and the step of receiving the first set of image data and the geo-referenced data of the field of view at the first location further comprises:

illuminating the field of view with the auxiliary lighting; and

detecting, in response to the illumination, shadows produced by the changed condition within the field of view.

13. The method of claim 12, wherein the auxiliary lighting is constant or pulsed at a frequency associated with maintaining the second set of image data and the geo-referenced data within the database.

14. The method of claim 12, wherein the auxiliary lighting is in a visible or a non-visible range.

15. The method of claim 1, wherein first and the second set of image data are displayed simultaneously on different portions of the display or displayed as two single views on each of two displays in close proximity to each other.

16. The method of claim 15, wherein the first and the second set of image data are oriented as parallel views, one over the other, on the display.

17. The method of claim 1, wherein displaying the first set of image data and the second set of image data, synchronized based on the geo-referenced data, comprises one of:

displaying, alternatively, the first set of image data followed by the second set of image data;

superimposing either the first or the second set of image data on a remaining set of image data;

or

fusing the first and the second set of image data.

18. The method of claim 1, further comprising:

providing a warning of an identified changed condition.

19. The method of claim 18, wherein providing the warning of the identified changed condition comprises highlighting the changed condition on the display.

20. The method of claim 19, wherein highlighting the changed condition on the display comprises blinking the changed condition on and off.