Method and system for categorization of a scene

• US 10,127,458 B2
• Filed: 12/21/2015
• Issued: 11/13/2018
• Est. Priority Date: 01/07/2015
• Status: Active Grant

• Current Assignee: Viscando AB
• Original Assignee: Viscando AB
• Inventors: Singh, Amritpal, Erlandsson, Ulf
• Primary Examiner(s): Johns, Andrew W
• Application Number:
  US15/538,498
• Publication Number:
  US 20170344835A1
• Time in Patent Office: 1,058 Days
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

The present invention relates to a method for categorizing a moving object within a scene In particular, the present invention relates to a method specifically taking into account the three dimensional data within a scene for determining the location of and type of objects present within the scene. The method involves determining a first probability level for the type of object based on its size and shape, determining a second probability level for the type of object based on its relative speed, and defining the type of the object based on a combination of the first and the second probability level. The invention also relates to a corresponding system and a computer program product.

Claims

1. A computer-implemented method for classifying a type of a moving object within a scene, the moving object being one of a plurality of different types of predetermined objects, the method comprising:

receiving an image stream comprising a plurality of images, each of the plurality of images being a captured representation of the scene and comprising depth information as to content within the scene, wherein the content of the scene comprises at least one physical foreground object and a physical background;

processing the plurality of images to produce an equal plurality of depth maps of the scene;

forming height maps for each of the plurality of images based on a predetermined model of the background and the corresponding depth map;

extracting the at least one physical foreground object from each of the plurality of images based on the corresponding height map;

determining, for each of the plurality of images, a relative position of the at least one physical foreground object within the scene based on the height maps;

determining a first probability level for the type of object within each of the images by matching the extracted physical foreground object with a predetermined set of different object types, wherein each of the predetermined set of different object types are defined to have at least a length, a height and a width being within a predetermined range;

determining a second probability level for the type of the object by determining a difference in the relative position for the object in at least two images of the plurality of images, converting the difference in relative position to a relative speed for the object, and matching the relative speed with speed profiles for the predetermined set of different object types; and

defining the type of the foreground object based on a combination of the first and the second probability level.

2. The method according to claim 1, wherein the different types of predetermined objects comprise a pedestrian, a bicycle and a vehicle.

3. The method according to claim 1, further defining the type of foreground object based on a third probability level for the type of the object, wherein the third probability is determined by:

matching the extracted foreground object with a set of predefined images for the plurality of different types of objects, and

determining a probability level, being a representation for the third probability, based on a resulting matching level.

4. The method according to claim 3, wherein the determination of the third probability level is defined to be formed by an image based classifier.

5. The method according to claim 1, wherein the determination of the first and the second probability level is defined to be formed by a geometrical classifier.

6. The method according to claim 1, further comprising:

determining a three dimensional boundary box surrounding the extracted foreground object.

7. The method according to claim 1, further comprising:

tracking the extracted foreground object within subsequent images of the image stream.

8. The method according to claim 1, further comprising:

storing, for each of the plurality of images, data representing the type of the extracted object and its relative position within a database.

9. The method according to claim 1, further comprising:

controlling at least two cameras for acquiring the image stream, wherein each of the at least to cameras are arranged to capturing a predetermined spatially separated corresponding image of the scene, and

determining the depth information based on the corresponding images captured by the at least two cameras.

10. The method according to claim 1, further comprising:

determining an absolute position for the extracted foreground object based on an absolute position for at least one location within the scene.

11. An image processing system for classifying a type of a moving object within a scene, the moving object being one of a plurality of different types of predetermined objects, the image processing system comprising a control unit configured to:

receiving an image stream comprising a plurality of images, each of the plurality of images being a captured representation of the scene and comprising depth information as to content within the scene, wherein the content of the scene comprises at least one physical foreground object and a physical background;

processing the plurality of images to produce an equal plurality of depth maps of the scene;

forming height maps for each of the plurality of images based on a predetermined model of the background and the corresponding depth map;

extracting the at least one physical foreground object from each of the plurality of images based on the corresponding height map;

determining, for each of the plurality of images, a relative position of the at least one physical foreground object within the scene based on the height maps;

determining a first probability level for the type of object within each of the images by matching the extracted physical foreground object with a predetermined set of different object types, wherein each of the predetermined set of different object types are defined to have at least a length, a height and a width being within a predetermined range;

determining a second probability level for the type of the object by determining a difference in the relative position for the object in at least two images of the plurality of images, converting the difference in relative position to a relative speed for the object, and matching the relative speed with speed profiles for the predetermined set of different object types; and

defining the type of the foreground object based on a combination of the first and the second probability level.

12. The system according to claim 11, further comprising:

at least two cameras wherein each of the at least to cameras are arranged to capturing a predetermined spatially separated corresponding image of the scene, wherein the control unit is further configured to;

control at least two cameras for acquiring the image stream, and

determine the depth information based on the corresponding images captured by the at least two cameras.

13. The system according to claim 12, further comprising:

a position interface for receiving an absolute position for at least one location within the scene,

wherein the control unit is further configured to;

determine an absolute position for the extracted foreground object based on the absolute position.

14. The system according to claim 13, further comprising a GPS receiver connected to the position interface and providing the absolute position of the system.

15. A computer program product comprising a non-transitory computer readable medium having stored thereon computer program means for operating an image processing system comprising a control unit, the image processing system configured for classifying a type of a moving object within a scene, the moving object being one of a plurality of different types of predetermined objects, the computer program product comprising:

code for receiving an image stream comprising a plurality of images, each of the plurality of images being a captured representation of the scene and comprising depth information as to content within the scene, wherein the content of the scene comprises at least one physical foreground object and a physical background;

code for processing the plurality of images to produce an equal plurality of depth maps of the scene;

code for forming height maps for each of the plurality of images based on a predetermined model of the background and the corresponding depth map;

code for extracting the at least one physical foreground object from each of the plurality of images based on the corresponding height map;

code for determining, for each of the plurality of images, a relative position of the at least one physical foreground object within the scene based on the height maps;

code for determining a first probability level for the type of object within each of the images by matching the extracted physical foreground object with a predetermined set of different object types, wherein each of the predetermined set of different object types are defined to have at least a length, a height and a width being within a predetermined range;

code for determining a second probability level for the type of the object by determining a difference in the relative position for the object in at least two images of the plurality of images, converting the difference in relative position to a relative speed for the object, and matching the relative speed with speed profiles for the predetermined set of different object types; and

code for defining the type of the foreground object based on a combination of the first and the second probability level.