Method of searching data to identify images of an object captured by a camera system

• US 9,830,511 B2
• Filed: 03/11/2016
• Issued: 11/28/2017
• Est. Priority Date: 03/03/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Analytics Corporation
• Inventors: Saptharishi, Mahesh, Lisin, Dimitri A.
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US15/068,433
• Publication Number:
  US 20160259975A1
• Time in Patent Office: 627 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A camera system comprises an image capturing device, object detection module, object tracking module, and match classifier. The object detection module receives image data and detects objects appearing in one or more of the images. The object tracking module temporally associates instances of detected objects, each of which has a signature representing features of the detected object. The match classifier matches object instances by analyzing data derived from the signatures. The match classifier determines whether the signatures match.

Claims

1. A camera system, comprising:

an object detection module operable to receive image data produced by at least one image capturing device, the image data representing multiple images of a field of view of the at least one image capturing device, and the object detection module operable to detect objects appearing in one or more of the multiple images;

an object tracking module connected to the object detection module and operable to temporally associate instances of objects detected by the object detection module, wherein each detected object has a signature representing features of the detected object from the multiple images; and

a match classifier connected to the object tracking module and operable to match object instances by analyzing data derived from the signature of each detected object based on matching parameters and non-matching parameters to indicate if two or more of the detected objects correspond to a same physical object.

2. The camera system of claim 1, further comprising

a user interface including a display for displaying objects detected by the object detection module and an input device operable to select objects detected by the object detection module in response to user commands.

3. The camera system of claim 2, further comprising

a search module connected to the match classifier and the user interface, the search module being operable to retrieve stored images of objects selected by the input device, the search module being operable to call on the match classifier to locate a stored image of a selected object.

4. The camera system of claim 1, wherein the object tracking module temporally associates instances of objects using a motion modeling module and the match classifier.

5. The camera system of claim 1, further comprising:

a rule engine operable to determine a rule violation based at least in part on the image data; and

an object search module operable to identify images of an object in the image data captured prior to the rule violation.

6. The camera system of claim 5, wherein a video analytics module includes the object detection module, the object tracking module, the match classifier, and the object search module, and wherein the rule engine uses the video analytics module to determine the rule violation.

7. The camera system of claim 1, wherein:

the at least one capturing device includes a first image capturing device and a second image capturing device;

the image data includes first image data produced by the first image capturing device and second image data produced by the second image capturing device; and

the match classifier is further operable to determine that a first object detected in the first image data is the same as a second object detected in the second image data.

8. The camera system of claim 7, wherein the match classifier is operable to make the determination by matching a first signature representing features of the first object with a second signature representing features of the second object.

9. The camera system of claim 8, wherein the first signature is an average signature and the second signature is an average signature.

10. The camera system of claim 1, wherein the camera system is operable to send a first signature of a detected object to a second camera system.

11. The camera system of claim 10, further comprising:

a routing table including a probability that a detected object will also appear in a field of view of an image capturing device of the second camera system; and

a match notification responder operable to receive, from the second camera system, match notification information indicating whether a match to the first signature was detected at the second camera system, and is further operable to update the probability in the routing table based on the match notification information.

12. The camera system of claim 11, wherein the match notification information includes a second signature of a matching object at the second camera system, and the match notification responder is operable to update the first signature with the second signature.

13. The camera system of claim 12, wherein the first signature is updated with the second signature by averaging the first signature with the second signature to produce an updated first signature.

14. The camera system of claim 1, further comprising:

a match database including one or more signatures received from a second camera system, the one or more signatures representing features of a remote object detected at the second camera system; and

wherein the match classifier is operable to determine whether the one or more signatures match any of the signatures of the objects detected the camera system.

15. The camera system of claim 14, wherein, in response to the one or more signatures matching any of the signatures of the objects detected by the camera system, the match classifier is operable to update the signature of the object detected using the one or more signatures and send match notification information to the second camera system.

16. The camera system of claim 15, wherein the signatures of the objects detected by the camera system are updated by averaging the signature of the object detected by the camera system with the one or more signatures.

17. A camera system, comprising:

means for object detection operable to receive image data produced by at least one image capturing device, the image data representing multiple images of a field of view of the at least one image capturing device, and the means for object detection operable to detect objects appearing in one or more of the multiple images;

means for object tracking connected to the means for object detection and operable to temporally associate instances of objects detected by the means for object detection, wherein each detected object has a signature representing features of the detected object from the multiple images; and

means for match classifying connected to the means for object tracking and operable to match object instances by analyzing data derived from the signature of each detected object based on matching parameters and non-matching parameters to indicate if two or more of the detected objects correspond to a same physical object.

18. The camera system of claim 17, further comprising

a user interface including a display for displaying objects detected by the means for object detection and an input device operable to select objects detected by the means for object detection in response to user commands.

19. The camera system of claim 18, further comprising

means for searching connected to the means for match classifying and the user interface, the means for searching being operable to retrieve stored images of objects selected by the input device, the means for searching being operable to call on the means for match classifying to locate a stored image of a selected object.

20. The camera system of claim 17, wherein the means for object tracking temporally associates instances of objects using a motion modeling module and the means for match classifying.

21. The camera system of claim 17, further comprising:

a rule engine means operable to determine a rule violation based at least in part on the image data; and

means for object searching operable to identify images of an object in the image data captured prior to the rule violation.

22. The camera system of claim 21, wherein means for video analytics includes the means for object detection, the means for object tracking, the means for match classifying, and the means for object searching, and wherein the rule engine means uses the means for video analytics to determine the rule violation.

23. The camera system of claim 17, wherein:

the at least one capturing device includes a first image capturing device and a second image capturing device;

the image data includes first image data produced by the first image capturing device and second image data produced by the second image capturing device; and

the means for match classifying is further operable to determine that a first object detected in the first image data is the same as a second object detected in the second image data.

24. The camera system of claim 23, wherein the means for match classifying is operable to make the determination by matching a first signature representing features of the first object with a second signature representing features of the second object.

25. The camera system of claim 24, wherein the first signature is an average signature and the second signature is an average signature.

26. The camera system of claim 17, wherein the camera system is operable to send a first signature of a detected object to a second camera system.

27. The camera system of claim 26, further comprising:

a routing table including a probability that a detected object will also appear in a field of view of an image capturing device of the second camera system; and

a match notification responder means operable to receive, from the second camera system, match notification information indicating whether a match to the first signature was detected at the second camera system, and is further operable to update the probability in the routing table based on the match notification information.

28. The camera system of claim 27, wherein the match notification information includes a second signature of a matching object at the second camera system, and the match notification responder means is operable to update the first signature with the second signature.

29. The camera system of claim 28, wherein the first signature is updated with the second signature by averaging the first signature with the second signature to produce an updated first signature.

30. The camera system of claim 17, further comprising:

a match database including one or more signatures received from a second camera system, the one or more signatures representing features of a remote object detected at the second camera system; and

wherein the means for match classifying is operable to determine whether the one or more signatures match any of the signatures of the objects detected the camera system.

31. The camera system of claim 30, wherein, in response to the one or more signatures matching any of the signatures of the objects detected by the camera system, means for the match classifying is operable to update the signature of the object detected using the one or more signatures and send match notification information to the second camera system.

32. The camera system of claim 31, wherein the signatures of the objects detected by the camera system are updated by averaging the signature of the object detected by the camera system with the one or more signatures.