Systems and methods for efficient feature based image and video analysis

• US 9,135,338 B2
• Filed: 03/01/2012
• Issued: 09/15/2015
• Est. Priority Date: 03/01/2012
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Yates, J. Harlan, Rahmes, Mark, Brown, Patricia, Faulkner, Timothy B.
• Primary Examiner(s): CONWAY, THOMAS A
• Application Number:
  US13/409,639
• Publication Number:
  US 20130230250A1
• Time in Patent Office: 1,293 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/73 Querying
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06Q 20/29 characterised by micropayments
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher
• CPC Class Codes: G06V 20/13 Satellite images

Abstract

Systems (100) and methods (300) for efficient spatial feature data analysis. The methods involve simultaneously generating chip images using image data defining at least a first image and video chips using video data defining at least a first video stream. Thereafter, an array is displayed which comprises grid cells in which at least a portion of the chip images is presented, at least a portion of the video chips is presented, or a portion of the chip images and a portion of the video chips are presented. Each chip image comprises a panned-only view, a zoomed-only view, or a panned-and-zoomed view of the first image including a visual representation of at least one first object of a particular type. Each of the video chips comprises a segment of the first video stream which include a visual representation of at least one second object of the particular type.

Claims

1. A method for efficient spatial feature data analysis, comprising:

receiving, by a computing device, a user input for selecting a feature contained in a geographical area of interest visually represented within a first image via a user-software interaction directly with the first image displayed in a display area of an application window;

in response to the user-software interaction, simultaneously and dynamically generating, by the computing device, (1) a plurality of chip images using image data defining said first image and (2) a plurality of video chips using video data defining at least a first video stream, where

the feature selected by the user input and a feature contained in at least one said chip image or video chip comprise multi-dimensional representations of different objects which have at least one attribute in common;

generating, by said computing device, at least one array comprising (1) a plurality of grid cells in which at least a portion of said plurality of chip images is presented, (2) a portion of said plurality of video chips is presented, or (3) portions of both of said plurality of chip images and said plurality of video chips are presented; and

displaying, by said computing device, said array in a Graphical User Interface (“

GUI”

) window thereof;

wherein each of said plurality of chip images comprises at least one of a panned-only view, a zoomed-only view, and a panned-and-zoomed view of said first image including a first visual representation of at least one first object of a particular type; and

wherein each of said plurality of video chips comprises a segment of said first video stream which comprises a second visual representation of at least one second object of said particular type.

2. The method according to claim 1, further comprising automatically identifying, by said computing device, visual representations of said first and second objects of said particular type which are contained in said first image and said first video stream, said first and second objects being the same object.

3. The method according to claim 1, wherein said plurality of chip images and said plurality of video chips are generated in response to a selection of a feature contained in an image or a video stream displayed in an application window.

4. The method according to claim 1, further comprising concurrently playing all video chips of said array.

5. The method according to claim 1, further comprising cycling through a plurality of screen pages comprising a plurality of arrays in which chips for a plurality of data sources are presented.

6. The method according to claim 5, wherein each of said plurality of arrays comprises chips generated using a different one of said plurality of data sources.

7. The method according to claim 5, wherein each of said plurality of arrays comprises chips generated using at least two sources of said plurality of data sources.

8. The method according to claim 7, wherein the at least two sources comprise an image and a video stream.

9. The method according to claim 1, further comprising:

generating, by said computing device, a plurality of screen pages comprising said plurality of chip images and said plurality of video chips when a selected feature class has more than a maximum number of features that can fit in an array of a pre-defined size;

wherein each of said plurality of screen pages comprises chips that are different than the chip images and the video chips of all other screen pages of said plurality of screen pages.

10. The method according to claim 9, further comprising paging forward or backward, by said computing device, through said plurality of screen pages.

11. A system, comprising:

at least one computing device configured to;

receive a user input for selecting a feature contained in a geographical area of interest visually represented within a first image via a user-software interaction directly with the first image displayed in a display area of an application window;

in response to the user-software interaction, simultaneously and dynamically generate (1) a plurality of chip images using image data defining said first image and (2) a plurality of video chips using video data defining at least a first video stream, where

the feature selected by the user input and a feature contained in at least one said chip image or video chip comprise multi-dimensional representations of different objects which have at least one attribute in common;

generate at least one array comprising a plurality of grid cells in which at least (1) a portion of said plurality of chip images is presented, (2) a portion of said plurality of video chips is presented, or (3) portions of both of said plurality of chip images and said plurality of video chips are presented; and

display said array in a Graphical User Interface (“

GUI”

) window thereof;

wherein each of said plurality of chip images comprises at least one of a panned-only view, a zoomed-only view, and a panned-and-zoomed view of said first image including a first visual representation of at least one first object of a particular type; and

wherein each of said plurality of video chips comprises a segment of said first video stream which comprises a second visual representation of at least one second object of said particular type.

12. The system according to claim 11, wherein said computing device is further configured to automatically identify visual representations of said first and second objects of said particular type which are contained in said first image and said first video stream, said first and second objects being the same object.

13. The system according to claim 11, wherein said plurality of chip images and said plurality of video chips are generated in response to a selection of a feature contained in an image or a video stream displayed in an application window.

14. The system according to claim 11, wherein said computing device is further configured to concurrently play all video chips of said array.

15. The system according to claim 11, wherein said computing device is further configured to cycle through a plurality of screen pages comprising a plurality of arrays in which chips for a plurality of data sources are presented.

16. The system according to claim 15, wherein each of said plurality of arrays comprises chips generated using a different one of said plurality of data sources.

17. The system according to claim 15, wherein each of said plurality of arrays comprises chips generated using at least two sources of said plurality of data sources.

18. The system according to claim 17, wherein the at least two sources comprise an image and a video stream.

19. The system according to claim 11, wherein said computing device is further configured to:

generate a plurality of screen pages comprising said plurality of chip images and said plurality of video chips when a selected feature class has more than a maximum number of features that can fit in an array of a pre-defined size;

wherein each of said plurality of screen pages comprises chips that are different than the chips images and the video chips of all other screen pages of said plurality of screen pages.

20. The system according to claim 19, wherein said computing device is further configured to page forward or backward through said plurality of screen pages.