Systems and methods for efficient video analysis

• US 9,152,303 B2
• Filed: 03/01/2012
• Issued: 10/06/2015
• Est. Priority Date: 03/01/2012
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Yates, J. Harlan, Faulkner, Timothy B., Rahmes, Mark, Bakir, Tariq
• Primary Examiner(s): Ng, Amy
• Assistant Examiner(s): Kim, Sang H
• Application Number:
  US13/409,660
• Publication Number:
  US 20130232419A1
• Time in Patent Office: 1,314 Days
• Field of Search: 715/723, 715/719, 715/721, 725/37, 725/41, 386/241, 386/278
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04803 Split screen, i.e. subdividing the display area or the window area into separate subareas
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Abstract

Systems (100) and methods (300) for efficient video analysis. The methods involve: automatically identifying features of at least one feature class which are contained in a first video stream; simultaneously generating a plurality of first video chips (904) using first video data defining the first video stream; displaying an array comprising the first video chips within a graphical user interface window; and concurrently playing the first video chips. Each of the first video chips comprises a segment of the first video stream which includes at least one identified feature.

Claims

1. A method for efficient video analysis, comprising:

displaying, by said computing device, a first video stream in a first Graphical User Interface (“

GUI”

) window facilitating use of first functions of a first software application;

automatically identifying, by a computing device, visual representations of at least one object which are contained in said first video stream;

generating, by said computing device, a plurality of first video chips using first video data defining said first video stream, each of said first video chips comprising a segment of said first video stream which comprises at least one of said visual representations which have been identified;

displaying, by said computing device, a first array comprising a plurality of first cells in which said first video chips are presented within a second GUI window facilitating use of second functions of a second software application, which is concurrently displayed with the first GUI window as a separate and distinct GUI window;

concurrently playing said first video chips within the second GUI window, where said first video chips each have a first temporal resolution specifying a numerical value representing a total duration of a video stream;

receiving a user-software interaction for changing said first temporal resolution of at least one selected video chip from the first video chips concurrently displayed in the second GUI window;

generating, by said computing device, a plurality of second video chips having a second temporal resolution lower than said first temporal resolution in response to said user-software interaction;

displaying, by said computing device, a second array comprising a plurality of second cells in which said plurality of second video chips can be presented within said second GUI window; and

concurrently playing said plurality of second video chips which are presented within said plurality of second cells.

2. The method according to claim 1, further comprising:

generating, by said computing device, a plurality of aggregated video chips, each comprising a segment of said first video stream including at least two chips of said plurality of first video chips, wherein said at least two chips are presented in a temporal order.

3. The method according to claim 2, further comprising:

presenting, by said computing device, a third array comprising a plurality of third cells in which said plurality of aggregated video chips can be presented within said second GUI window;

respectively presenting said plurality of aggregated video chips within said plurality of third cells; and

concurrently playing said plurality of aggregated video chips which are presented within said plurality of third cells.

4. The method according to claim 1, wherein said first array comprises a grid having a pre-defined grid size.

5. The method according to claim 4, further comprising changing, by the computing device, the grid size in response to a user input selecting a new grid size from a plurality of grid sizes.

6. The method according to claim 4, further comprising:

generating, by said computing device, a plurality of pages of video chips for said first video stream when a selected feature class has more than a maximum number of features that can fit in said grid of said pre-defined grid size;

wherein each of said plurality of pages of video chips comprises video chips that are different than said video chips of all other pages of said plurality of pages of video chips.

7. The method according to claim 6, further comprising paging forward or backward, by said computing device, through said plurality of pages of video chips in response to a user-software interaction.

8. The method according to claim 6, further comprising changing, by said computing device, visual content of said second GUI window from a first page of said plurality of pages of video chips to a second page of said plurality of pages of video chips.

9. The method according to claim 1, further comprising displaying in said GUI window information relating to an attribute of a feature contained in a selected one of said plurality of first video chips.

10. The method according to claim 1, further comprising:

sorting, by said computing device, said plurality of first video chips in an ascending or descending order based on at least one feature attribute; and

displaying, by said computing device, said plurality of first video chips in said ascending or descending order.

11. The method according to claim 1, further comprising:

filtering, by said computing device, said plurality of first video chips based on at least one feature attribute; and

displaying, by said computing device, said plurality of first video chips which have been filtered in said second GUI window.

12. The method according to claim 1, further comprising:

changing, by said computing device, a spatial zoom level of scale of images of said plurality of first video chips from a first zoom level of scale to a second zoom level of scale in response to a user-software interaction; and

wherein said second spatial zoom level of scale is a user-specified spatial zoom level of scale, a pre-defined fixed spatial zoom level of scale, or a best-fit spatial zoom level of scale.

13. The method according to claim 1, further comprising cycling, by said computing device, through a plurality of pages of video chips that were generated using said first video data of said first video stream and second video data of a second video stream.

14. The method according to claim 1, further comprising adding, by said computing device, a mark or annotation to at least one of said plurality of first video chips in response to a user-software interaction.

15. The method according to claim 1, further comprising adding, by said computing device, a mark or annotation to all of said plurality of first video chips that precede or succeed a selected one of said plurality of video chips in a temporal order or a sorted order.

16. The method according to claim 1, further comprising exporting, by said computing device, at least one of a plurality of video chips that precede or succeed a selected one of said plurality of first video chips in a temporal order or a sorted order.

17. The method according to claim 1, further comprising storing, by said computing device, various settings that a user sets for at least one of a plurality of feature classes during at least one session.

18. A system comprising:

at least one electronic circuit configured to;

display a first video stream in a first Graphical User Interface (“

GUI”

) window facilitating use of first functions of a first software application;

automatically identify visual representations of at least one object which are contained in said first video stream;

generate a plurality of first video chips using first video data defining said first video stream, each of said first video chips comprising a segment of said first video stream which comprises at least one of said visual representations which have been identified;

display a first array comprising a plurality of first cells in which said first video chips are presented within a second GUI window facilitating use of second functions of a second software application, which is concurrently displayed with the first GUI window as a separate and distinct GUI window;

concurrently play said first video chips within the second GUI window, where said first video chips each have a temporal resolution specifying a numerical value representing a total duration of a video stream;

receive a user software interaction for changing said first temporal resolution of at least one selected video chip from the first video chips concurrently displayed in the second GUI window;

generating a plurality of second video chips having a second temporal resolution lower than said first temporal resolution in response to said user-software interaction;

displaying a second array comprising a plurality of second cells in which said plurality of second video chips can be presented within said second GUI window; and

concurrently playing said plurality of second video chips which are presented within said plurality of second cells.

19. The system according to claim 18, wherein said electronic circuit is further configured to:

generate a plurality of aggregated video chips, each comprising a segment of said first video stream including at least two chips of said plurality of first video chips; and

presenting said at least two chips in a temporal order.

20. The system according to claim 19, wherein said electronic circuit is further configured to:

present a third array comprising a plurality of third cells in which said plurality of aggregated video chips can be presented within said second GUI window;

respectively present said plurality of aggregated video chips within said plurality of third cells; and

concurrently play said plurality of aggregated video chips which are presented within said plurality of third cells.

21. The system according to claim 18, wherein said first array comprises a grid having a pre-defined grid size.

22. The system according to claim 21, wherein said electronic circuit is further configured to change the grid size in response to a user input selecting a new grid size from a plurality of grid sizes.

23. The system according to claim 21, wherein said electronic circuit is further configured to:

generate a plurality of pages of video chips for said first video stream when a selected feature class has more than a maximum number of features that can fit in said grid of said pre-defined grid size;

wherein each of said plurality of pages of video chips comprises video chips that are different than said video chips of all other pages of said plurality of pages of video chips.

24. The system according to claim 23, wherein said electronic circuit is further configured to page forward or backward through said plurality of pages of video chips in response to a user-software interaction.

25. The system according to claim 23, wherein said electronic circuit is further configured to change visual content of said second GUI window from a first page of said plurality of pages of video chips to a second page of said plurality of pages of video chips.

26. The system according to claim 18, wherein said electronic circuit is further configured to display in said second GUI window information relating to an attribute of a feature contained in a selected one of said plurality of first video chips.

27. The system according to claim 18, wherein said electronic circuit is further configured to:

sort said plurality of first video chips in an ascending or descending order based on at least one feature attribute; and

display said plurality of first video chips in said ascending or descending order.

28. The system according to claim 18, wherein said electronic circuit is further configured to:

filter said plurality of first video chips based on at least one feature attribute; and

display said plurality of first video chips which have been filtered in said second GUI window.

29. The system according to claim 18, wherein said electronic circuit is further configured to:

change a spatial zoom level of scale of images of said plurality of first video chips from a first zoom level of scale to a second zoom level of scale in response to a user-software interaction; and

wherein said second spatial zoom level of scale is a user-specified spatial zoom level of scale, a pre-defined fixed spatial zoom level of scale, or a best-fit spatial zoom level of scale.

30. The system according to claim 18, wherein said electronic circuit is further configured to cycle through a plurality of pages of video chips that were generated using said first video data of said first video stream and second video data of a second video stream.

31. The system according to claim 18, wherein said electronic circuit is further configured to add a mark or annotation to at least one of said plurality of first video chips in response to a user-software interaction.

32. The system according to claim 18, wherein said electronic circuit is further configured to add a mark or annotation to all of said plurality of first video chips that precede or succeed a selected one of said plurality of video chips in a temporal order or a sorted order.

33. The system according to claim 18, wherein said electronic circuit is further configured to export at least one of said plurality of video chips that precede or succeed a selected one of said plurality of first video chips in a temporal order or a sorted order.

34. The system according to claim 18, wherein said electronic circuit is further configured to store various settings that a user sets for at least one of a plurality of feature classes during at least one session.