Automated activity detection using supervised learning

• US 7,881,537 B2
• Filed: 01/31/2006
• Issued: 02/01/2011
• Est. Priority Date: 01/31/2006
• Status: Active Grant

• Current Assignee: Honeywell International Inc.
• Original Assignee: Honeywell International Inc.
• Inventors: Bazakos, Michael E., Ma, Yunqian
• Primary Examiner(s): Werner; Brian P
• Assistant Examiner(s): DRENNAN, BARRY T
• Application Number:
  US11/343,658
• Publication Number:
  US 20070177792A1
• Time in Patent Office: 1,827 Days
• Field of Search: 382/155, 382/159, 382224-228, 706/10, 706/12, 706/13
• US Class Current: 382/190
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition

Abstract

In an embodiment, one or more sequences of learning video data is provided. The learning video sequences include an action. One or more features of the action are extracted from the one or more sequences of learning video data. Thereafter, a sequence of operational video data is received, and the one or more features of the action from the sequence of operational video data is extracted. A comparison is then made between the extracted one or more features of the action from the one or more sequences of learning video data and the one or more features of the action from the sequence of operational video data. In an embodiment, this comparison allows the determination of whether the action is present in the operational video data.

Claims

1. A method comprising:

providing to a processor one or more first sequences of learning video data for each of a plurality of actions;

identifying and tracking blobs within said learning video data;

extracting using the processor one or more features from said identified blobs for each of said plurality of actions;

determining using the processor whether at least one extracted feature of one of said plurality of actions provides a discrimination from all other of said plurality of actions;

when none of said extracted features of said one of said plurality of actions provides a discrimination from all other of said plurality of actions, providing a second sequence of learning video data for said one of said plurality of actions, said second sequence of learning video data captured under different conditions than said first sequence of learning video data;

receiving at the processor a sequence of operational video data;

identifying and tracking blobs within said operational video data;

extracting using the processor one or more features from said identified blobs in said sequence of operational video data; and

comparing using the processor said at least one extracted feature of one of said plurality of actions and said one or more features from said sequence of operational video data.

2. The method of claim 1, wherein said comparison comprises a classifier design to determine whether said sequence of operational video data includes said at least one extracted feature.

3. The method of claim 2, wherein said classifier design includes one or more of a best fit equation, a hierarchical tree structure, and a summation of data clouds.

4. The method of claim 1, further comprising storing said at least one extracted feature of one of said plurality of actions.

5. The method of claim 4, further comprising:

receiving a plurality of sequences of operational video data;

extracting one or more features of an action from said plurality of sequences of operational video data;

determining that said one or more features of said action represent an action that is not identifiable as one of said plurality of actions; and

storing said extracted features of said action.

6. The method of claim 4, further comprising:

receiving a sequence of operational video data;

comparing said one or more features extracted from said learning video data for each of said plurality of actions with said sequence of operational video data; and

determining one or more features that best differentiate said sequence of operational video data from said learning video data.

7. The method of claim 6, wherein said comparing said one or more features extracted from said learning video data for each of said plurality of actions with said sequence of operational video data and said determining one or more features that best differentiate said sequence of operational video data from said learning video data comprise a genetic algorithm.

8. A system comprising:

a processor configured to provide one or more first sequences of learning video data for each of a plurality of actions;

a processor configured to identify and track blobs within said learning video data;

a processor configured to extract one or more features from said identified blobs for each of said plurality of actions;

a processor configured to determine whether at least one extracted feature of one of said plurality of actions provides a discrimination from all other of said plurality of actions;

a processor configured to provide, when none of said extracted features of said one of said plurality of actions provides a discrimination from all other of said plurality of a second sequence of learning video data for said one of said plurality of actions, said second sequence of learning video data captured under different conditions than said first sequence of learning video data;

a processor configured to receive a sequence of operational video data;

a processor configured to identify and track blobs within said operational video data;

a processor configured to extract one or more features from said identified blobs in said sequence of operational video data; and

a processor configured to compare said at least one extracted feature of one of said plurality of actions and said one or more features from said sequence of operational video data.

9. The system of claim 8, wherein said comparison comprises a classifier design to determine whether said sequence of operational video data includes said at least one extracted feature.

10. The system of claim 9, wherein said classifier design includes one or more of a best fit equation, a hierarchical tree structure, and a summation of data clouds.

11. The system of claim 8, further comprising a processor configured to store said at least one extracted feature of one of said plurality of actions.

12. The system of claim 11, further comprising:

a processor configured to receive a plurality of sequences of operational video data;

a processor configured to extract one or more features of an action from said plurality of sequences of operational video data;

a processor configured to determine that said one or more features of said action represent an action that is not identifiable as one of said plurality of actions; and

a processor configured to store said extracted features of said action.

13. The system of claim 11, further comprising:

a processor configured to receive a sequence of operational video data;

a processor configured to compare said one or more features extracted from said learning video data for each of said plurality of actions with said sequence of operational video data; and

a processor configured to determine one or more features that best differentiate said sequence of operational video data from said learning video data.

14. The system of claim 13, wherein said comparison of said one or more features extracted from said learning video data for each of said plurality of actions with said sequence of operational video data and said determination of one or more features that best differentiate said sequence of operational video data from said learning video data comprise a genetic algorithm.

15. A non-transitory machine-readable medium having instructions that when executed by a processor execute a method comprising:

providing one or more first sequences of learning video data for each of a plurality of actions;

identifying and tracking blobs within said learning video data;

extracting one or more features from said identified blobs for each of said plurality of actions;

determining whether at least one extracted feature of one of said plurality of actions provides a discrimination from all other of said plurality of actions;

when none of said extracted features of said one of said plurality of actions provides a discrimination from all other of said plurality of actions, providing a second sequence of learning video data for said one of said plurality of actions, said second sequence of learning video data captured under different conditions than said first sequence of learning video data, the different conditions including one or more of a different range and a different angle;

receiving a sequence of operational video data;

identifying and tracking blobs within said operational video data;

extracting one or more features from said identified blobs in said sequence of operational video data; and

comparing said at least one extracted feature of one of said plurality of actions and said one or more features from said sequence of operational video data.

16. The machine-readable medium of claim 15, wherein said comparison comprises a classifier design to determine whether said sequence of operational video data includes said at least one extracted feature.

17. The machine-readable medium of claim 16, wherein said classifier design includes one or more of a best fit equation, a hierarchical tree structure, and a summation of data clouds.

18. The machine-readable medium of claim 15, further comprising instructions for storing said at least one extracted feature of one of said plurality of actions.

19. The machine-readable medium of claim 18, further comprising instructions for:

receiving a plurality of sequences of operational video data;

extracting one or more features of an action from said plurality of sequences of operational video data;

determining that said one or more features of said action represent an action that is not identifiable as one of said plurality of actions; and

storing said extracted features of said action.

20. The machine-readable medium of claim 18, further comprising instructions for:

receiving a sequence of operational video data;

comparing said one or more features extracted from said learning video data for each of said plurality of actions with said sequence of operational video data; and

determining one or more features that best differentiate said sequence of operational video data from said learning video data.