Sensing system and method for detecting moving objects

• US 9,516,274 B2
• Filed: 08/15/2013
• Issued: 12/06/2016
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Original Assignee: Honda Research Institute Europe GmbH (Honda Motor Company)
• Inventors: Deigmoeller, Joerg, Eggert, Julian, Einecke, Nils
• Primary Examiner(s): Pontius, James
• Application Number:
  US13/967,772
• Publication Number:
  US 20140049644A1
• Time in Patent Office: 1,209 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The present invention presents a sensing system 1 and a corresponding method for detecting moving objects 11 in the surroundings of a vehicle 10. The sensing system 1 comprises an imaging unit 2 for obtaining an image stream 2a, a computing unit 3 for analyzing the image stream 2a, and a control unit 4 for controlling the vehicle 10 based on the analysis result of the computing unit 3. The sensing system 1 particularly employs a background-model-free estimation. The sensing system 1 is configured to perform a local analysis of two neighboring motion vectors 6, which are computed from points 7a in images 5a, 5b of the image stream 2a, and to determine, whether the points 7a corresponding to these motion vectors 6 belong to a particularly moving object.

Claims

1. A sensing system for detecting moving objects in the surroundings of a vehicle, comprising

an imaging unit for acquiring images and generating an image stream consisting of a plurality of images;

and

a computing unit configured to

receive the image stream from the imaging unit,

select a plurality of points in a first image of the image stream,

compute for each selected point in the first image a matching point in a second image of the image stream to obtain 2D motion vectors,

compute a connectivity graph based on the selected points, wherein the connectivity graph represents local relations between the selected points,

select a plurality of point pairs based on the connectivity graph, wherein each point pair represents two endpoints of an edge in the connectivity graph,

compute for each selected point pair first transformation parameters by using the computed 2D motion vectors and 2D coordinates of the selected point pairs,

group each selected point pair into a transformation class based on the computed first transformation parameters to obtain first class information; and

a control unit for controlling the behavior of the vehicle or for issuing a warning indication based on the first class information received from the computing unit.

2. A sensing system according to claim 1, wherein

the first transformation parameters are scaling factors of the selected point pair with the first image in respect to a matching point pair in the second image, and

the computing unit is configured to group each selected point pair into a transformation class based on the scaling factors.

3. A sensing system according to claim 1, wherein

the computing unit is configured to obtain the connectivity graph by performing a triangulation method.

4. A sensing system according to claim 3, wherein

the computing unit is configured to

calculate for at least one triangle, which is obtained by the triangulation method, second transformation parameters by combining the first transformation parameters for three point pairs that correspond to the three edges of the triangle, and

group each point of the at least one triangle into a transformation class based on the second transformation parameters to obtain second class information, and

the control unit is configured to control the behavior of the vehicle or issue a warning indication based on the second class information received from the computing unit.

5. A sensing system according to claim 3, wherein

the computing unit is further configured to post-process the second class information by integrating the second class information over time in a separate integration map by using the positions of triangles obtained by the triangulation method for spatial accumulation, and

the control unit is configured to control the behavior of the vehicle or issue a warning indication by using the post-processed second class information.

6. A sensing system according to claim 1, wherein

the computing unit is further configured to post-process the first class information by integrating the first class information over time in a separate integration map by using the positions of the point pairs for spatial accumulation, and

the control unit is configured to control the behavior of the vehicle or issue a warning indication by using the post-processed first class information.

7. A sensing system according to claim 1, wherein

the imaging unit is mounted on the vehicle, and

the control unit is configured to control the behavior of the vehicle or issue the warning indication, if one or more objects approach or retreat from the vehicle.

8. A sensing method for detecting moving objects in the surroundings of a vehicle, comprising the steps of:

acquiring images and generating an image stream of a plurality of images;

selecting a plurality of points in a first image of the image stream;

computing for each selected point in the first image a matching point in a second image of the image stream to obtain 2D motion vectors;

computing a connectivity graph based on the selected points, wherein the connectivity graph represents local relations between the selected points,

selecting a plurality of point pairs based on the connectivity graph, wherein each point pair represents two endpoints of an edge in the connectivity graph;

computing for each selected point pair first transformation parameters by using the computed 2D motion vectors and 2D coordinates of the selected point pairs;

grouping each selected point pair into a transformation class based on the computed first transformation parameters to obtain first class information; and

controlling the behavior of the vehicle or issuing a warning indication based on the first class information.

9. A computer software program, embodied on a non-transitory computer readable medium, the computer program, when executed by a processor, causes the processor to:

acquire images and generate an image stream of a plurality of images;

select a plurality of points in a first image of the image stream;

compute for each selected point in the first image a matching point in a second image of the image stream to obtain 2D motion vectors;

compute a connectivity graph based on the selected points, wherein the connectivity graph represents local relations between the selected points,

select a plurality of point pairs based on the connectivity graph, wherein each point pair represents two endpoints of an edge in the connectivity graph;

compute for each selected point pair first transformation parameters by using the computed 2D motion vectors and 2D coordinates of the selected point pairs;

group each selected point pair into a transformation class based on the computed first transformation parameters to obtain first class information; and

control the behavior of the vehicle or issuing a warning indication based on the first class information.

10. A land, sea or air vehicle provided with a sensing system for detecting moving objects in the surroundings of the vehicle, comprising:

an imaging unit for acquiring images and generating an image stream consisting of a plurality of images; and

a computing unit configured to

receive the image stream from the imaging unit,

select a plurality of points in a first image of the image stream,

compute for each selected point in the first image a matching point in a second image of the image stream to obtain 2D motion vectors,

compute a connectivity graph based on the selected points, wherein the connectivity graph represents local relations between the selected points,

select a plurality of point pairs based on the connectivity graph, wherein each point pair represents two endpoints of an edge in the connectivity graph,

compute for each selected point pair first transformation parameters by using the computed 2D motion vectors and 2D coordinates of the selected point pairs,

group each selected point pair into a transformation class based on the computed first transformation parameters to obtain first class information; and

a control unit for controlling the behavior of the vehicle or for issuing a warning indication based on the first class information received from the computing unit.