System and method for use of a vehicle back-up camera as a dead-reckoning sensor

• US 8,855,917 B2
• Filed: 10/16/2008
• Issued: 10/07/2014
• Est. Priority Date: 10/16/2008
• Status: Active Grant

• Current Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Original Assignee: CSR Technology Incorporated (Qualcomm, Inc.)
• Inventors: Koenig, Michael, Brodie, Keith
• Primary Examiner(s): Garg, Yogesh C
• Application Number:
  US12/252,701
• Publication Number:
  US 20100100321A1
• Time in Patent Office: 2,182 Days
• Field of Search: 701/200, 701/207, 701/208, 701/209, 701/214, 701/409, 701/400, 701/408, 701/417, 701/532, 701/410, 701/425, 701/450, 701/469, 701/472, 701/500
• US Class Current: 701/409
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/04 by terrestrial means G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/10 by using measurements of speed or acceleration G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G01C 21/3461 Preferred or disfavoured areas, e.g. dangerous zones, toll or emission zones, intersections, manoeuvre types, segments such as motorways, toll roads, ferries
• CPC Class Codes: G01C 21/3658 Lane guidance
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Systems and methods are disclosed herein to use a vehicle back-up camera as a cost-effective dead-reckoning sensor in satellite-based vehicle navigation systems. Since the back-up camera may already use a display of the navigation system for display, the data from the back-up camera may be easily obtained and integrated into the navigation system. The data from the camera is received by a navigation receiver wirelessly or through a wired connection. The image is processed to determine the speed, heading, turn-rate of the vehicle to aid the satellite-based navigation system if the satellite signals are inadequate. Thus, enhanced vehicle navigation- performance can be obtained without adding new sensors and/or connecting to a vehicle data bus.

Claims

1. A vehicle navigation system comprising:

a back-up camera mounted on a rear of a vehicle to provide video data of ground features to the rear of the vehicle to provide back-up camera images to a display in the vehicle and to provide dead reckoning images, wherein a motion of the camera is forward along a line bisecting a horizontal field of view of the camera;

a video dead-reckoning subsystem to correlate the dead reckoning images from the back-up camera for generating an estimated motion of the vehicle; and

a satellite-based navigation system to use the estimated motion of the vehicle to aid in generating a navigation solution for the vehicle.

2. The vehicle navigation system of claim 1, wherein the video dead-reckoning subsystem receives the dead reckoning images from the back-up camera through a wired connection.

3. The vehicle navigation system of claim 1, wherein the video dead-reckoning subsystem receives the dead reckoning images from the back-up camera through a wireless connection.

4. The vehicle navigation system of claim 1, wherein the video dead-reckoning subsystem further comprises:

a frame capture module to sample the dead reckoning images in a frame to provide sampled video data;

a memory to store one or more frames of the sampled dead reckoning images;

one or more correlators to perform image correlation between two frames of the sampled dead reckoning images to search for one or more displacement vectors; and

a video dead-reckoning state estimator to estimate motion of the vehicle and spatial parameters of the back-up camera from the one or more displacement vectors.

5. The vehicle navigation system of claim 4, wherein each of the one or more correlators is assigned a corresponding region within each frame of the sampled dead reckoning images.

6. The vehicle navigation system of claim 5, wherein each of the one or more correlators is configured to perform the image correlation through the acts of:

taking pixel values of the sampled dead reckoning images within the corresponding region of the correlator from a first frame of the sampled video data;

shifting the corresponding region of the correlator by a displacement to form a shifted region;

taking pixel values of the sampled dead reckoning images within the shifted region of the correlator from a second frame of the sampled dead reckoning images;

multiplying a pixel value of each pixel of the sampled dead reckoning images within the corresponding region of the correlator from the first frame of the sampled video data with a pixel value of a corresponding pixel of the sampled dead reckoning images video data within the shifted region of the correlator from the second frame of the sampled dead reckoning images to form a product; and

summing the product over all pixels taken.

7. The vehicle navigation system of claim 6, wherein a plurality of image correlations is performed for a plurality of displacements of the corresponding region for each correlator.

8. The vehicle navigation system of claim 7, wherein the displacement for shifting the corresponding region of each correlator is restricted to a search space.

9. The vehicle navigation system of claim 8, wherein the displacement vector between two frames of the sampled dead reckoning images for each correlator is the displacement of the corresponding region corresponding to a largest of the plurality of image correlations.

10. The vehicle navigation system of claim 9, wherein for each correlator the displacement vector found for the image correlation between the first frame and the second frame of the sampled dead reckoning images video data is used to modify the search space for a next image correlation.

11. The vehicle navigation system of claim 5, wherein if there is more than one correlator the region corresponding to each correlator does not overlap with the regions corresponding to the remaining correlators.

12. The vehicle navigation system of claim 5, wherein if there is more than one correlator the region corresponding to one correlator overlaps with the region corresponding to another correlator.

13. A method of providing dead-reckoning data for a vehicle from a back-up camera without a dead-reckoning sensor to aid a satellite-based navigation system, comprising steps executed by a dead-reckoning device:

receiving video data of ground features to the rear of the vehicle from the back-up camera, wherein a motion of the camera is forward along a line bisecting a horizontal field of view of the camera;

sampling the video data in a first frame;

sampling the video data in a second frame;

correlating the video data in the first frame with the video data in the second frame to search for a displacement vector;

estimating motion data of the vehicle from the displacement vector; and

providing the motion data to the satellite-based navigation system.

14. The method of claim 13, wherein correlating the video data in the first frame with the video data in the second frame comprises:

taking pixel values of the video data from a region of the first frame;

shifting the region by a displacement to form a shifted region in the second frame;

taking pixel values of the video data from the shifted region of the second frame;

multiplying the pixel values for each pixel of the video data from the region of the first frame with the pixel values of a corresponding pixel of the video data from the shifted region of the second frame to form a product; and

summing the product over all pixels from the region of the first frame and the shifted region of the second frame to form a correlation.

15. The method of claim 14, wherein correlating the video data in the first frame with the video data in the second frame is performed for a plurality of displacements to generate a plurality of correlations.

16. The method of claim 15, wherein the displacement vector is the displacement corresponding to a largest of the plurality of correlations.