Object detection and classification

• US 9,922,271 B2
• Filed: 05/05/2016
• Issued: 03/20/2018
• Est. Priority Date: 03/20/2015
• Status: Active Grant

• Current Assignee: Netra, Inc.
• Original Assignee: Netra, Inc.
• Inventors: Kant, Shashi
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US15/147,598
• Publication Number:
  US 20160275376A1
• Time in Patent Office: 684 Days
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06F 18/2431 Multiple classes
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Object detection and across disparate fields of view are provided. A first image generated by a first recording device with a first field of view, and a second image generated by a second recording device with a second field of view can be obtained. An object detection component can detect a first object within the first field of view, and a second object within the second field of view. An object classification component can determine first and second level classification categories of the first object. Object components can correlate the first object with the second object based on the descriptor of the first object or a descriptor of the second object, and can determine a characteristic or the first object or the second object based on the correlation.

Claims

1. A system of object detection across disparate fields of view, comprising:

a data processing system having an object detection component, an object classification component, an object forecast component, and an object matching component, the data processing system obtains a first image generated by a first recording device, the first recording device having a first field of view;

the object detection component of the data processing system detects, from the first image, a first object present within the first field of view;

the object classification component of the data processing system determines a first level classification category of the first object and determines a second level classification category of the first object;

the data processing system generates a descriptor of the first object based on at least one of the first level classification category of the first object and the second level classification category of the first object;

the data processing system obtains a second image generated by a second recording device, the second recording device having a second field of view different than the first field of view;

the object detection component of the data processing system detects, from the second image, a second object present within the second field of view;

the data processing system generates a descriptor of the second object based on at least one of a first level classification category of the second object and a second level classification category of the second object;

the object matching component of the data processing system identifies a correlation of the first object with the second object based on the descriptor of the first object and the descriptor of the second object; and

the object forecast component of the data processing system determines a characteristic of at least one of the first object and the second object based at least in part on the correlation of the first object with the second object.

2. The system of claim 1, comprising:

the data processing system configured to determine the characteristic, the characteristic indicating that the first object and the second object are different objects that have an association with each other.

3. The system of claim 1, wherein the first image includes a third object and a fourth object, comprising:

the object matching component of the data processing system configured to identify an association between the first object, the third object, and the fourth object.

4. The system of claim 1, wherein the first image includes a third object, comprising:

the data processing system configured to generate a descriptor of the third object based on at least one of a first level classification category of the third object and a second level classification category of the third object; and

the object matching component of the data processing system configured to identify an association between the first object and the third object based on the descriptor of the first object and the descriptor of the third object.

5. The system of claim 1, comprising:

the object matching component configured to determine a likelihood that the first object and the second object are a same object.

6. The system of claim 5, wherein the first image includes a third object and, wherein the first object and the second object are a same object comprising:

the object matching component of the data processing system configured to determine that the same object and the third object are at least part of a family unit.

7. The system of claim 6, wherein the characteristic indicates predicted behavioral activity of at least one of the same object and the third object.

8. The system of claim 1, wherein the characteristic indicates predicted behavioral activity of at least part of a family unit.

9. The system of claim 1, wherein the characteristic indicates a predicted future location of at least one of the first object and the second object.

10. The system of claim 1, comprising:

the object forecast component configured to identify the characteristic, the characteristic indicating that the first object and the second object are different but related objects.

11. The system of claim 1, comprising:

the first recording device and the second recording device each located in a respective identified geographic location.

12. The system of claim 1, comprising:

the first recording device and the second recording device located in a respective unidentified geographic location.

13. The system of claim 1, wherein the first image and the second image are obtained from the internet.

14. The system of claim 1, wherein the characteristic of at least one of the first object and the second object includes a predictive characteristic.

15. The system of claim 1, wherein the first object and the second object are a same object present in both the first image and the second image.

16. The system of claim 1, wherein the first object and the second object are different objects.

17. The system of claim 1, comprising:

the data processing system operational to create, for the first object, a data structure indicating a probability identifier for the descriptor of the first object, and to create, for the second object, a data structure indicating a probability identifier for the descriptor of the second object; and

the object matching component operational to identify the correlation of the first object with the second object based on the probability identifier for the descriptor of the first object and the probability identifier for the descriptor of the second object.

18. The system of claim 1, comprising:

the data processing system configured to provide, to an end user computing device via a computer network, an indication that the characteristic is satisfied.

19. A method of digital image object analysis across disparate fields of view, comprising:

obtaining, by a data processing system having at least one of an object detection component, an object classification component, an object forecast component, and an object matching component, a first image generated by a first recording device, the first recording device having a first field of view;

detecting, by the object detection component of the data processing system, from the first image, a first object present within the first field of view;

determining, by the object classification component of the data processing system, a first level classification category of the first object and a second level classification category of the first object;

generating, by the data processing system, a descriptor of the first object based on at least one of the first level classification category of the first object and the second level classification category of the first object;

obtaining, by the data processing system, a second image generated by a second recording device, the second recording device having a second field of view different than the first field of view;

detecting, by the object detection component of the data processing system, from the second image, a second object present within the second field of view;

generating, by the data processing system, a descriptor of the second object based on at least one of a first level classification category of the second object and a second level classification category of the second object;

identifying, by the object matching component of the data processing system, a correlation between the first object and the second object based on the descriptor of the first object and the descriptor of the second object; and

determining, by the object forecast component of the data processing system, a characteristic of at least one of the first object and the second object based on the correlation between the first object and the second object.

20. The method of claim 19, comprising:

providing, by the data processing system via a computer network, for display by an end user computing device, a first electronic document that includes an indication of the characteristic.