Image processing method of vehicle camera and image processing apparatus using the same

• US 9,619,895 B2
• Filed: 08/21/2014
• Issued: 04/11/2017
• Est. Priority Date: 08/22/2013
• Status: Active Grant

• Current Assignee: HL Mando Corporation (Halla Holdings Corporation)
• Original Assignee: Mando Corporation
• Inventors: Lim, Jun Taek
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Duley, Janese
• Application Number:
  US14/465,814
• Publication Number:
  US 20150054955A1
• Time in Patent Office: 964 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

Disclosed herein is an image processing apparatus of a vehicle camera, including a light source extraction unit configured to extract light sources from a first frame exposed to light sources during a first time and to extract light sources including motion blurs from a second frame exposed to the light sources during a second time longer than the first time, a target light source classification unit configured to classify a first target light source that belongs to the light sources of the first frame and that does not move and a second target light source that belongs to the light sources of the second frame and that does not move, and a motion determination unit configured to estimate a first location that is the central point of the first target light source and a second location that is the central line including the central point of the second target light source, compare the first location with the second location, and determine whether or not a pitch motion of the vehicle has occurred.

Claims

1. An image processing apparatus of a vehicle camera, comprising:

a light source extraction unit configured to extract light sources from a first frame exposed to light sources during a first time and to extract light sources generating motion blurs from a second frame exposed to the light sources during a second time longer than the first time;

a target light source classification unit configured to classify a first target light source that belongs to the light sources of the first frame and that does not move and a second target light source that belongs to the light sources of the second frame and that does not move;

a motion determination unit configured to estimate a first location that is a central point of the first target light source and a second location that is a central line comprising a central point of the second target light source, compare the first location with the second location, determine whether or not a pitch motion of the vehicle has occurred, perform binarization on an image of the first frame or the second frame, and extract coordinates of the center of gravity of pixels having a specific value as the first location or the second location; and

a correction unit configured to correct an image of the vehicle camera based on a result of the comparison between the first location and the second location.

2. The image processing apparatus according to claim 1, wherein the light source that does not move comprises at least one of a streetlamp and a visual guidance reflector.

3. The image processing apparatus according to claim 1, wherein the motion determination unit is further configured to overlap the first frame and the second frame and to determine a direction or degree of the pitch motion of the vehicle based on a location where the first location and the second location are overlapped.

4. The image processing apparatus according to claim 1, wherein a digital sensor of the image processing apparatus is a rolling shutter type CMOS sensor.

5. An image processing method of a vehicle camera, comprising:

extracting light sources from a first frame that has been obtained by the vehicle camera and that has been exposed to light sources during a first time;

classifying a first target light source that belongs to the extracted light sources and that does not move;

computing a first location that is a central point of the first target light source;

extracting light sources generating motion blurs from a second frame that has been obtained by the vehicle camera and that has been exposed to light sources during a second time longer than the first time;

classifying a second target light source that belongs to the light sources extracted from the second frame and that does not move;

computing a second location that is a central line comprising a central point of the second target light source;

comparing the first location with the second location; and

determining whether or not a pitch motion of the vehicle has occurred based on a result of the comparison between the first location and the second location,

wherein the computing the first location or the computing the second location comprises;

performing binarization on an image of the first frame or the second frame, and

extracting coordinates of the center of gravity of pixels having a specific value as the first location or the second location.

6. The image processing method according to claim 5, further comprising correcting an image of the vehicle camera based on the result of the comparison between the first location and the second location.

7. The image processing method according to claim 5, wherein classifying the first target light source or classifying the second target light source comprises classifying a light source present in a predetermined region of the first frame or the second frame as a target light source.

8. The image processing method according to claim 5, wherein classifying the first target light source or classifying the second target light source comprises classifying a plurality of streetlamps or visual guidance reflectors arranged along a road as target light sources in accordance with a training method using a support vector machine.

9. The image processing method according to claim 5, wherein determining whether or not a pitch motion of the vehicle has occurred based on a result of the comparison between the first location and the second location comprises:

overlapping the first frame and the second frame, and

determining a direction or degree of the pitch motion of the vehicle based on a position where the first location and the second location are overlapped.

10. The image processing method according to claim 5, wherein the second frame is a frame that belongs to a plurality of frames included in the image and that is placed right after the first frame.

11. The image processing method according to claim 6, wherein correcting an image of the vehicle camera comprises:

extracting an optimized correction direction and distance using a RANdom SAmple Consensus (RANSAC) algorithm, and

correcting the image.