SYSTEMS AND METHODS FOR OBJECT TRACKING AND CLASSIFICATION

• US 20170132468A1
• Filed: 11/06/2015
• Published: 05/11/2017
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Mosher, Aaron Y., Mefford, David Keith
• Granted Patent: US 9,959,468 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

A method for classifying at least one object of interest in a video is provided. The method includes accessing, using at least one processing device, a frame of the video, the frame including at least one object of interest to be classified, performing, using the at least one processing device, object detection on the frame to detect the object of interest, tracking, using the at least one processing device, the object of interest over a plurality of frames in the video over time using a persistent tracking capability, isolating, using the at least one processing device, a segment of the frame that includes the object of interest, classifying, using the at least one processing device, the object of interest by processing the segment using deep learning, and generating an output that indicates the classification of the object of interest.

Claims

1. A method for classifying at least one object of interest in a video, the method comprising:

accessing, using at least one processing device, a frame of the video, the frame including at least one object of interest to be classified;

performing, using the at least one processing device, object detection on the frame to detect the object of interest;

tracking, using the at least one processing device, the object of interest over a plurality of frames in the video over time using a persistent tracking capability;

isolating, using the at least one processing device, a segment of the frame that includes the object of interest;

classifying, using the at least one processing device, the object of interest by processing the segment using deep learning; and

generating an output that indicates the classification of the object of interest.

2. A method in accordance with claim 1, wherein tracking the object of interest comprising tracking the object of interest using a tracking component, and wherein classifying the object of interest comprises classifying the object of interest using a classification component, wherein the classification component is located remotely from the tracking component.

3. A method in accordance with claim 2, wherein classifying the object of interest using a classification component comprises classifying the object of interest using a classification component of at least one processing device installed on a vehicle.

4. A method in accordance with claim 2, wherein classifying the object of interest using a classification component comprises classifying the object of interest using a classification component that satisfies small size, weight, and power requirements.

5. A method in accordance with claim 1, wherein classifying the object of interest by processing the segment using deep learning comprises classifying the object of interest by processing the segment using at least one of a convolutional neural network and a deep neural network.

6. A method in accordance with claim 1, wherein isolating a segment of the frame that includes the object of interest comprises isolating the segment such that the object of interest is substantially centered within the segment.

7. A method in accordance with claim 1, further comprising continuing to track, using the classification of the object of interest, the object of interest after the object of interest has left and returned to a field of view of the video.

8. A method in accordance with claim 1, wherein generating an output comprises:

receiving, from a user at an input device coupled to the at least one processor, a predetermined object type; and

generating, using a presentation interface coupled to the at least one processor, an alert when the classification of the object of interest matches the predetermined object type.

9. A computer-implemented system for classifying at least one object of interest in a video, the system comprising:

a tracking component implemented using at least one processing device and configured to;

access a frame of the video, the frame including at least one object of interest to be classified;

perform object detection on the frame to detect the object of interest;

track the object of interest over a plurality of frames in the video over time using a persistent tracking capability; and

isolate a segment of the frame that includes the object of interest; and

a classification component communicatively coupled to said tracking component, said classification component implemented using the at least one processing device and configured to;

classify the object of interest by processing the segment using deep learning; and

generate an output that indicates the classification of the object of interest.

10. A system in accordance with claim 9, wherein said classification component and said tracking component are implemented using a single processing device.

11. A system in accordance with claim 9, wherein said classification component is located remotely from said tracking component.

12. A system in accordance with claim 11, wherein said classification component is implemented using at least one processing device installed on a vehicle.

13. A system in accordance with claim 11, wherein said classification component satisfies small size, weight, and power requirements.

14. A system in accordance with claim 9, wherein to classify the object of interest by processing the segment using deep learning, said classification component is configured to classify the object of interest using at least one of a convolutional neural network and a deep neural network.

15. A system in accordance with claim 9, wherein to isolate a segment of the frame that includes the object of interest, said tracking component is configured to isolate the segment such that the object of interest is substantially centered within the segment.

16. A system in accordance with claim 9, wherein said tracking component is further configured to continue to track, using the classification of the object of interest, the object of interest after the object of interest has left and returned to a field of view of the video.

17. An object classification computing device for classifying at least one object of interest in a video, the object classification computing device comprising:

a memory device; and

a processor communicatively coupled to said memory device, said processor configured to;

access a frame of the video, the frame including at least one object of interest to be classified;

perform object detection on the frame to detect the object of interest;

track the object of interest over a plurality of frames in the video over time using a persistent tracking capability;

isolate a segment of the frame that includes the object of interest;

classify the object of interest by processing the segment using deep learning; and

generate an output that indicates the classification of the object of interest.

18. An object classification computing device in accordance with claim 17, wherein said processor is installed on a vehicle.

19. An object classification computing device in accordance with claim 17, wherein said processor satisfies small size, weight, and power requirements.

20. An object classification computing device in accordance with claim 17, wherein to classify the object of interest by processing the segment using deep learning, said processor is configured to classify the object of interest using a convolutional neural network.