Automatic detection of moving object by using stereo vision technique

• US 10,187,617 B2
• Filed: 06/30/2011
• Issued: 01/22/2019
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Original Assignee: TATA Consultancy Services Limited (Tata Sons Pvt Ltd.)
• Inventors: Bhowmick, Brojeshwar, Sinharay, Arijit, Pal, Arpan, Bhadra, Sambit
• Primary Examiner(s): Mahmud, Farhan
• Application Number:
  US13/807,755
• Publication Number:
  US 20130194419A1
• Time in Patent Office: 2,763 Days
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

The present invention discloses a system and method for identifying a moving object in front of a vehicle by using stereo vision based technique. The invention also discloses a method and a system for determining an instantaneous distance between the moving object and the vehicle by processing the image of the moving object through a processor and by filtering and refining the image by using a Kalman filter to remove noise.

Claims

1. A method for identifying a moving object in front of a vehicle and determining an instantaneous distance of the moving object from the said vehicle, the method comprises the steps of:

capturing real time images of the moving objects, falling in range of an image capturing device in the form of a plurality of frames;

synchronizing each frame of the captured plurality of frames to obtain a synchronized frame or an identical frame, wherein each frame of the captured plurality of frames is assigned a time stamp such that the time stamp identifies a synchronized frame corresponding to each frame of the captured plurality of frames; and

processing the captured plurality of frames in the synchronized frame by executing processor implemented steps of;

identifying the moving object by a feature collected from the image of the moving object, wherein the feature includes pixel;

matching the features collected from the image of the moving object by using a standard correlation, wherein the matching is a Harris corner matching;

refining and tracking the moving object by color correlation and a kernel based correlation tracking if the moving object is segmented in a previous frame and not segmented in a current frame by;

creating a candidate model using a histogram model corresponding to the previous frame to obtain a target candidate histogram, wherein the target candidate histogram is a color histogram and wherein the color histogram of the segmented object is computed by a quantized color scheme;

calculating distance between the target candidate histogram and a target model, wherein the target model corresponds to the current frame having identical position as the previous frame; and

obtaining a tracked moving object by calculating displacement of the target model using the calculated distance to obtain a new position for the target model;

stereo-vision based triangulation of the tracked moving object to obtain an instantaneous distance between the moving object and the vehicle;

determining direction of the vehicle by calculating speed of the vehicle using an average rate of change of instantaneous distance between the moving object at each time frame for the assigned time stamp and steering angle by using an optical flow of each frame; and

refining the instantaneous distance corresponding to the direction of the vehicle using a Kalman filter layer output to remove noise developed in the measurement of the instantaneous distance.

2. The method of claim 1, wherein identifying the moving object further comprises of collecting plurality of feature obtained by subtracting sum of pixels in white region from the sum of pixels in black region.

3. The method of claim 1, wherein the method further comprises of measuring and fixing steering angles and velocity of the vehicle based on the filtering and refinement of the instantaneous distance by using the Kalman filter layer output.

4. The method of claim 1, wherein the method further comprises of alerting an operator of the vehicle by generating an alarm.

5. A system for identifying a moving object in front of a vehicle and determining the instantaneous distance between the moving object and the said vehicle, the system comprises of:

an image capturing device adapted to capture images of the moving object falling in the range of the image capturing device in the form of a plurality of frames;

a system clock coupled with the image capturing device for synchronizing the plurality of frames by;

synchronizing each frame of the captured plurality of frames to obtain a synchronized frame or an identical frame, wherein each frame of the captured plurality of frames is assigned a time stamp such that the time stamp identifies a synchronized frame corresponding to each frame of the captured plurality of frames; and

identifying the moving object by a feature collected from the image of the moving object, wherein the feature includes pixel;

matching the features collected from the image of the moving object by using a standard correlation, wherein the matching is a Harris corner matching;

refining and tracking the moving object by color correlation and a kernel based correlation tracking if the moving object is segmented in a previous frame and not segmented in a current frame by;

creating a candidate model using a histogram model corresponding to the previous frame to obtain a target candidate histogram, wherein the target candidate histogram is a color histogram, and wherein the color histogram of the segmented object is computed by a quantized color scheme;

calculating distance between the target candidate histogram and a target model, wherein the target model corresponds to the current frame having identical position as the previous frame; and

obtaining a tracked moving object by calculating displacement of the target model using the calculated distance to obtain a new position for the target model;

stereo-vision based triangulation of the tracked moving object to obtain an instantaneous distance between the moving object and the vehicle;

determining direction of the vehicle by calculating speed of the vehicle using an average rate of change of instantaneous distance between the moving object at each time frame for the assigned time stamp and steering angle by using an optical flow of each frame;

a processor for processing the captured image in the synchronized frame and capable of executing programmed instructions for filtering and refining the instantaneous distance measured between the moving object and the vehicle corresponding to the direction of the vehicle by using a Kalman filter layer; and

a calculating unit for determining the instantaneous distance between the moving object and the vehicle.

6. The system of claim 5, wherein the system further comprises of a plurality of image capturing device.

7. The system of claim 5, wherein the image capturing device includes a camera.

8. The system of claim 5, wherein the camera is mounted on the vehicle and is in motion.

9. The system of claim 5, wherein the system further comprises of a control unit for measuring and fixing steering angles and velocity of the vehicle based on the filtering and refinement of the instantaneous distance by using the Kalman filter layer after determining the instantaneous distance through the stereo-vision based triangulation.

10. The system of claim 5, wherein the system further comprises of an alarm generating device for alerting an operator of the vehicle by generating an alarm.