Video analysis

• US 8,983,131 B2
• Filed: 04/15/2013
• Issued: 03/17/2015
• Est. Priority Date: 05/08/2012
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Hugosson, Fredrik
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US13/863,055
• Publication Number:
  US 20130301876A1
• Time in Patent Office: 701 Days
• Field of Search: 382/103, 382/284
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method (200) and an object analyzer (104) for analyzing objects in images captured by a monitoring camera (100) uses a first and a second sequence of image frames, wherein the first sequence of image frames covers a first image area (300) and has a first image resolution, and the second sequence of image frames covers a second image area (302) located within the first image area (300) and has a second image resolution higher than the first image resolution. A common set of object masks is provided wherein object masks of objects (304) that are identified as being present in both image areas are merged.

Claims

1. A method of analyzing objects in images captured by a monitoring camera, comprising the steps of:

receiving, by an electronic device, a first sequence of image frames having a first image resolution and covering a first image area;

receiving, by the electronic device, a second sequence of image frames having a second image resolution higher than the first image resolution and covering a second image area being a portion of the first image area;

detecting, by the electronic device, objects present in the first sequence of image frames;

detecting, by the electronic device, objects present in the second sequence of image frames;

providing, by the electronic device, a first set of object masks for objects detected in the first sequence of image frames;

providing, by the electronic device, a second set of object masks for objects detected in the second sequence of image frames;

identifying, by the electronic device, an object present in the first and the second sequence of image frames by detecting a first object mask in the first set of object masks at least partly overlapping a second object mask in the second set of object masks;

merging, by the electronic device, the first and the second object mask into a third object mask by including data from the first object mask for parts present only in the first image area, and data from the second object mask for parts present in the second image area; and

providing, by the electronic device, a third set of object masks including

the first set of object masks excluding the first object mask;

the second set of object masks excluding the second object mask; and

the third object mask.

2. The method of claim 1, wherein the steps of detecting objects comprise comparing the first and the second sequence of image frames with image data representing a background model.

3. The method of claim 1, wherein the steps of detecting objects comprise comparing an image frame with a previously captured image frame in the respective sequence of image frames.

4. The method of claim 1, wherein the steps of detecting objects comprise performing pattern recognition, such as face recognition.

5. The method of claim 1, further comprising composing an object description of the identified object by including image data from the first sequence of image frames for parts of the third object mask that are only present in the first sequence of image frames, and image data from the second sequence of image frames for parts of the third object mask present in the second sequence of image frames.

6. The method of claim 5, wherein the step of composing an object description comprises providing a first bitmap representation of parts of the identified object, present in the first sequence of image frames, from the first sequence of image frames, and providing a second bitmap representation of parts of the identified object, present in the second sequence of image frames, from the second sequence of image frames, and providing a third bitmap representation by combining the first and the second bitmap representation by scaling the first bitmap representation to the second image resolution.

7. The method of claim 6, wherein the step of composing an object description comprises providing a vector representation of the identified object based on the third bitmap representation.

8. An object analyzer for analyzing objects in images captured by a monitoring camera, the object analyzer comprising

an image data input arranged to receive a first sequence of image frames having a first image resolution and covering a first image area, and a second sequence of image frames having a second image resolution higher than the first image resolution and covering a second image area being a portion of the first image area,

an object detector arranged to detect objects present in the first sequence of image frames and objects present in the second sequence of image frames,

a first object mask set provider arranged to provide a first set of object masks for objects detected in the first sequence of image frames, and a second set of object masks for objects detected in the second sequence of image frames,

an object identifier arranged to identify an object present in the first and the second sequence of image frames by identifying a first object mask in the first set of object masks at least partly overlapping a second object mask in the second set of object masks,

an object mask merger arranged to merge the first and the second object mask into a third object mask by including data from the first object mask for parts present only in the first image area, and data from the second object mask for parts present in the second image area,

a second object mask set provider arranged to provide a third set of object masks including

the first set of object masks excluding the first object mask,

the second set of object masks excluding the second object mask, and

the third object mask.

9. The object analyzer of claim 8, wherein the object detector is arranged to compare an image frame with a previously captured image frame in the respective sequence of image frames.

10. The object analyzer of claim 8, wherein the object detector is arranged to compare the first and the second sequence of image frames with image data representing a background model.

11. The object analyzer of claim 8, wherein the object detector is arranged to perform pattern recognition, such as face recognition.

12. The object analyzer of claim 8, further comprising an object description composer arranged to provide an object description of the identified object by including image data from the first sequence of image frames for parts of the third object mask that are only present in the first sequence of image frames, and image data from the second sequence of image frames for parts of the third object mask present in the second sequence of image frames.

13. The object analyzer of claim 12, wherein the object description composer is arranged to

provide a first bitmap representation of parts of the identified object, present in the first sequence of image frames, from the first sequence of image frames, and a second bitmap representation of parts of the identified object, present in the second sequence of image frames, from the second sequence of image frames, and provide a third bitmap representation by combining the first and the second bitmap representation by scaling the first bitmap representation to the second image resolution.

14. The object analyzer of claim 13, wherein the object description composer is arranged to

provide a vector representation of the identified object based on the third bitmap representation.

15. A non-transitory computer-readable medium including computer-program instructions, which when executed by an information processing system, cause the information processing system to:

receive a first sequence of image frames having a first image resolution and covering a first image area;

receive a second sequence of image frames having a second image resolution higher than the first image resolution and covering a second image area being a portion of the first image area;

detect objects present in the first sequence of image frames;

detect objects present in the second sequence of image frames;

provide a first set of object masks for objects detected in the first sequence of image frames;

provide a second set of object masks for objects detected in the second sequence of image frames;

identify an object present in the first and the second sequence of image frames by detecting a first object mask in the first set of object masks at least partly overlapping a second object mask in the second set of object masks;

merge the first and the second object mask into a third object mask by including data from the first object mask for parts present only in the first image area, and data from the second object mask for parts present in the second image area; and

provide a third set of object masks including

the first set of object masks excluding the first object mask;

the second set of object masks excluding the second object mask; and

the third object mask.

16. The non-transitory computer-readable medium of claim 15, wherein the system includes one or more of an imaging device, a video encoding device, a video server and a video processing device.

17. An information processing system comprising:

circuitry configured to

receive a first sequence of image frames having a first image resolution and covering a first image area;

receive a second sequence of image frames having a second image resolution higher than the first image resolution and covering a second image area being a portion of the first image area;

detect objects present in the first sequence of image frames;

detect objects present in the second sequence of image frames;

provide a first set of object masks for objects detected in the first sequence of image frames;

provide a second set of object masks for objects detected in the second sequence of image frames;

identify an object present in the first and the second sequence of image frames by detecting a first object mask in the first set of object masks at least partly overlapping a second object mask in the second set of object masks;

merge the first and the second object mask into a third object mask by including data from the first object mask for parts present only in the first image area, and data from the second object mask for parts present in the second image area; and

provide a third set of object masks including

the first set of object masks excluding the first object mask;

the second set of object masks excluding the second object mask; and

the third object mask.

18. The information processing system of claim 17, wherein the information processing system is one or more of an imaging device, a video encoding device, a video server and a video processing device.