Rear view mirror simulation

• US 10,800,329 B2
• Filed: 08/30/2017
• Issued: 10/13/2020
• Est. Priority Date: 04/19/2010
• Status: Active Grant

• Current Assignee: SMR Patents Sarl (SMR Patents)
• Original Assignee: SMR Patents Sarl (SMR Patents)
• Inventors: Herrmann, Andreas, Schwalb, Martin, Linsenmaier, Frank, Eder, Oliver, Mualla, Firas
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Edwards, Tyler B
• Application Number:
  US15/691,332
• Publication Number:
  US 20180126903A1
• Time in Patent Office: 1,140 Days
• CPC Class Codes: B60Q 1/2665 on rear-view mirrors lighting devices for vehicle interior on or for rear-view mirrors B60Q3/258
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8046 for replacing a rear-view mirror system
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/76 based on eigen-space representations, e.g. from pose or different illumination conditions; Shape manifolds
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/0104 : Measuring and analyzing of parameters relative to traffic conditions
• CPC Class Codes: G08G 1/0175 : by photographing vehicles, e.g. when violating traffic rules
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/61 : the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"
• CPC Class Codes: H04N 25/615 : involving a transfer function modelling the optical system, e.g. optical transfer function [OTF], phase transfer function [PhTF] or modulation transfer function [MTF]

Abstract

The invention relates to an exterior mirror simulation with image data recording and a display of the recorded and improved data for the driver of a motor vehicle. The display on a display device shows the data in a way chosen by the driver or the vehicle manufacturer.

Claims

1. A method for producing a single image from data collected in a plurality of images captured by an optical system to simulate a mirror used as a rear-view mirror of a vehicle, the method comprising the steps of:

identifying primary edges of a first image of the plurality of images;

locating a first border within the first image that extends to part of a first edge of the first image;

identifying secondary edges of a second image of the plurality of images;

selecting a second edge from the secondary edges closest in position and orientation to the part of the first edge of the first image;

locating a second border within the second image that extends to the second edge; and

adjusting the second image such that a second part of the second border of the second image aligns with the first border of the first image,

wherein the optical system simultaneously simulates both a plane mirror and a spotter mirror in form of an aspheric mirror in the single image to provide a plurality of optical subsystems.

2. A method as set forth in claim 1, wherein the plurality of optical subsystems includes a system designed to detect electromagnetic radiation outside the visible spectrum.

3. A method as set forth in claim 1, wherein the step of identifying secondary edges includes identifying secondary edges of a second image of the plurality of images closest in position and orientation to at least part of the first edge of the first image.

4. A method as set forth in claim 1, wherein the step of adjusting includes a step of generating an overall image using the first and second images.

5. A method as set forth in claim 4, further comprising a step of rectifying visual distortion in the single image.

6. A method as set forth in claim 1, wherein a portion of the secondary edges and a portion of the first edge overlap.

7. A method as set forth in claim 6, wherein the portion of the first image includes a part of the secondary edges and the first edge.

8. A method as set forth in claim 1, further comprising a step of iteratively selecting first and second images until all of the plurality of images have been adjusted to produce the single image.

9. A method as set forth in claim 8, wherein the step of adjusting includes a step of generating an overall image using the plurality of images.

10. A method as set forth in claim 9, further comprising a step of rectifying visual distortion in the single image.

11. A method as set forth in claim 1, further comprising a step of classifying an object by sampling a periodic descriptor function for a query point in the image from the environment.

12. A method for producing a single image from data collected in a plurality of images captured by an optical system to simulate a mirror used as a rear-view mirror of a vehicle, the method comprising the steps of:

identifying primary edges of a first image of the plurality of images;

locating a first border within the first image that extends to part of a first edge of the first image;

identifying secondary edges of a second image of the plurality of images;

selecting a second edge from the secondary edges closest in position and orientation to the part of the first edge of the first image;

locating a second border within the second image that extends to the second edge;

adjusting the second image such that a second part of the second border of the second image aligns with the first border of the first image;

classifying an object by sampling a periodic descriptor function for a query point in the image from the environment; and

extracting at least one feature value from at least one of one or more keypoints in an image from an environment to assist in the step of classifying the object,

wherein the optical system simultaneously simulates both a plane mirror and a spotter mirror in form of an aspheric mirror in the single image to provide a plurality of optical subsystems.

13. A method for producing a single image from data collected in a plurality of images captured by an optical system to simulate a mirror used as a rear-view mirror of a vehicle, the method comprising the steps of:

identifying primary edges of a first image of the plurality of images;

locating a first border within the first image that extends to part of a first edge of the first image;

identifying secondary edges of a second image of the plurality of images;

selecting a second edge from the secondary edges closest in position and orientation to the part of the first edge of the first image;

locating a second border within the second image that extends to the second edge;

adjusting the second image such that a second part of the second border of the second image aligns with the first border of the first image; and

stitching the first image and the second image together to form a single image,

wherein the optical system simultaneously simulates both a plane mirror and a spotter mirror in form of an aspheric mirror in the single image to provide a plurality of optical subsystems.