Method for advanced and low cost cross traffic alert, related processing system, cross traffic alert system and vehicle

• US 10,242,272 B2
• Filed: 03/28/2017
• Issued: 03/26/2019
• Est. Priority Date: 09/21/2016
• Status: Active Grant

• Current Assignee: STMicroelectronics SRL (STMicroelectronics NV)
• Original Assignee: STMicroelectronics SRL (STMicroelectronics NV)
• Inventors: Spampinato, Giuseppe, Curti, Salvatore, Guarneri, Nunziata Ivana, Bruna, Arcangelo Ranieri
• Primary Examiner(s): Osinski, Michael S
• Application Number:
  US15/471,138
• Publication Number:
  US 20180082132A1
• Time in Patent Office: 728 Days
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08G 1/09626 where the origin of the information is within the own vehicle, e.g. a local storage device, digital map

Abstract

A sequence of images obtained by a camera mounted on a vehicle is processed in order to generate Optical Flow data including a list of Motion Vectors being associated with respective features in the sequence of images. The Optical Flow data is analyzed to calculate a Vanishing Point by calculating the mean point of all intersections of straight lines passing through motion vectors lying in a road. An Horizontal Filter subset is determined taking into account the Vanishing Point and a Bound Box list from a previous frame in order to filter from the Optical Flow the horizontal motion vectors. The subset of Optical Flow is clustered to generate the Bound Box list retrieving the moving objects in a scene. The Bound Box list is sent to an Alert Generation device and an output video shows the input scene where the detected moving objects are surrounded by a Bounding Box.

Claims

1. A method for Cross Traffic Alert, comprising:

receiving a sequence of image frames from a camera mounted on a vehicle;

processing said sequence of image frames in order to generate Optical Flow data for each image frame comprising a list of Motion Vectors which indicate motion of features in said sequence of image frames;

filtering said Optical Flow data according to orientation and selecting only those Motion Vectors for features which are laying in a road in order to form a reduced set of Motion Vectors for each image frame;

calculating an image frame Vanishing Point as an average position of all intersections of straight lines generated by directions of Motion Vectors belonging to said reduced set of Motion Vectors;

checking that the calculated image frame Vanishing Point lies within a certain area of the image frame;

averaging the calculated image frame Vanishing Points which lie within said certain area over a plurality of image frames in order to calculate a Vanishing Point;

filtering said Optical Flow data taking into account the calculated Vanishing Point and a Bound Box list from a previous image frame to generate for a current image frame a Horizontal Filter subset which includes only horizontal Motion Vectors;

clustering the Horizontal Filter subset to generate a Bound Box list for a current image frame which includes transversally approaching moving objects; and

outputting said Bound Box list for the current image frame to an Alert Generation device.

2. The method for Cross Traffic Alert according to claim 1, wherein checking comprises overlying a grid over the image to determine when a position of the image frame Vanishing Point belongs to a central area of the grid.

3. The method for Cross Traffic Alert according to claim 1, wherein filtering to generate the Horizontal Filter subset comprises:

checking the Bound Box list from the previous image frame and maintaining in said Horizontal Filter subset the Motion Vectors inside the previous calculated Bound Box List from the previous image frame;

performing a Horizontal Check wherein the Motion Vectors outside the Bound Box List from the previous image frame are checked and the horizontal Motion Vectors are filtered and if the Motion Vector is considered Horizontal it is maintained in the Horizontal Filter subset, and otherwise it is discarded.

4. The method for Cross Traffic Alert according to claim 3, wherein a Motion Vector is considered horizontal, when the following two conditions are both satisfied:

an orientation of the Motion Vector lies within a certain degree range of a horizontal orientation; and

an orientation difference between the orientation of the Motion Vector and an orientation of the Motion Vector translated on the calculated Vanishing Point exceeds a threshold.

5. The method for Cross Traffic Alert according to claim 4, wherein said threshold is a percentage of a difference between a maximum orientation and a minimum orientation of the Motion Vector.

6. The method for Cross Traffic Alert according to claim 1, further comprising generating an output video showing a scene imaged by the camera where each moving object from said Bound Box list for the current image frame is surrounded by a Bounding Box.

7. A method for Cross Traffic Alert, comprising:

receiving a sequence of image frames from a camera mounted on a vehicle;

processing said sequence of image frames in order to generate Optical Flow data for each image frame comprising Motion Vectors indicating the motion of features in a current image frame compared with same features in a previous image frame;

filtering said Optical Flow data, as a function of an orientation of each Motion Vector, to generate a reduced set of Optical Flow data comprising Motion Vectors for only the features which are lying in a road;

calculating a Vanishing Point which is a mean point of intersections of straight lines passing through the Motion Vectors of the reduced set of Optical Flow data;

calculating a Horizontal Filter subset by;

receiving a Bound Box list of a previous image frame;

checking the Bound Box list of the previous image frame and maintaining in an Optical Flow data subset the Motion Vectors of the Bound Box list of the previous image frame; and

performing a Horizontal Check wherein the Motion Vectors outside the Bound Box list of the previous image frame are checked to identify horizontal Motion Vectors to be maintained in the Optical Flow data subset, wherein the Motion Vector is considered horizontal, when the following two conditions are both satisfied;

an orientation of the Motion Vector lies within a certain degree range of a horizontal orientation; and

an orientation difference between the orientation of the Motion Vector and an orientation of the Motion Vector translated on the calculated Vanishing Point exceeds a threshold;

clustering the Horizontal Filter subset to generate the Bound Box list for a current image frame which includes transversally approaching moving objects; and

outputting said Bound Box list for the current image frame to an Alert Generation device.

8. The method for Cross Traffic Alert according to claim 7, further comprising generating an output video showing a scene imaged by the camera where each moving object from said Bound Box list is surrounded by a Bounding Box.

9. The method for Cross Traffic Alert according to claim 7, wherein calculating the Vanishing Point comprises:

calculating an image frame Vanishing Point for each image frame;

checking that that calculated image frame Vanishing Point lies within a certain area of the image frame; and

averaging the calculated image frame Vanishing Points which lie within said certain area over a plurality of image frames in order to calculate the Vanishing Point.

10. The method for Cross Traffic Alert according to claim 9, wherein checking comprises overlying a grid over the image to determine when a position of the image frame Vanishing Point belongs to a central area of the grid.

11. The method for Cross Traffic Alert according to claim 7, wherein said threshold is a percentage of a difference between a maximum orientation and a minimum orientation of the considered Motion Vector.