Image processing apparatus, image processing method, computer program and computer readable recording medium

• US 10,197,413 B2
• Filed: 11/27/2017
• Issued: 02/05/2019
• Est. Priority Date: 11/26/2016
• Status: Active Grant

• Current Assignee: Thinkware Corporation
• Original Assignee: Thinkware Corporation
• Inventors: Lee, Ki Wook, Byun, Hye Kyung, Han, Tae Kyu, Kim, Shin Hyoung
• Primary Examiner(s): Nguyen, Tan Q
• Application Number:
  US15/822,705
• Publication Number:
  US 20180149487A1
• Time in Patent Office: 435 Days
• Field of Search: 701450
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3811 Point data, e.g. Point of Interest [POI]
• CPC Class Codes: G01C 21/3841 Data obtained from two or more sources, e.g. probe vehicles
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G06F 16/23 Updating
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

Provided is an image processing method. The image processing method includes receiving images acquired from a plurality of vehicles positioned on a road; storing the received images according to acquisition information of the received images; determining a reference image and a target image based on images having the same acquisition information among the stored images; performing an image registration using a plurality of feature points extracted from each of the determined reference image and target image; performing a transparency process for each of the reference image and the target image which are image-registered; extracting static objects from the transparency-processed image; and comparing the extracted static objects with objects on map data which is previously stored and updating the map data when the objects on the map data which is previously stored and the extracted static objects are different from each other.

Claims

1. An image processing method comprising:

receiving images acquired from a plurality of vehicles positioned on a road;

storing the received images according to acquisition information of the received images;

determining a reference image and a target image based on images having the same acquisition information among the stored images;

performing an image registration using a plurality of feature points extracted from each of the determined reference image and target image;

performing a transparency process for each of the reference image and the target image which are image-registered;

extracting static objects from the transparency-processed image; and

comparing the extracted static objects with objects on map data which is previously stored and updating the map data when the objects on the map data which is previously stored and the extracted static objects are different from each other.

2. The image processing method of claim 1, wherein the performing of the image registration includes

extracting the plurality of feature points from each of the determined reference image and target image;

and

performing the image registration for the determined images using the plurality of extracted feature points.

3. The image processing method of claim 1, wherein the plurality of feature points are points at which image brightness value suddenly changes in the reference image or the target image and are edges of pixels or corner points.

4. The image processing method of claim 2, wherein the transparency process multiplies R, G, and B pixel values of respective pixels included in the images for which the transparency process is to be performed by a predetermined value smaller than 1, and

the predetermined value is a reciprocal number of N, which is a total number of the images for which the transparency process is to be performed.

5. The image processing method of claim 1, wherein the acquisition information includes at least one of information on positions at which the images are photographed, information on angles at which the images are photographed, and information on directions in which the images are photographed.

6. The image processing method of claim 2, wherein the extracting of the plurality of feature points includes

extracting a plurality of first feature points from the reference image;

and

extracting a plurality of second feature points from the target image.

7. The image processing method of claim 6, wherein the performing of the image registration includes

performing a matching operation matching a first feature point group in which the plurality of first feature points are grouped and a second feature point group in which the plurality of second feature points are grouped;

calculating a homography using information of matched pairs between the first feature point group and the second feature point group through the matching operation;

converting the target image using the calculated homography; and

registering the reference image and the converted target image.

8. The image processing method of claim 1, wherein the updating of the map data includes

confirming position information of the extracted static objects;

examining whether or not objects different from the extracted static objects exist at a position corresponding to the confirmed position information of the static objects in the map data which is previously stored; and

updating the map data by reflecting the extracted static objects to the map data which is previously stored corresponding to the position information, when the different objects exist as a result of the examination.

9. The image processing method of claim 1, further comprising transmitting the updated map data to the plurality of vehicles positioned on the road.

10. The image processing method of claim 1, wherein the extracting of the static objects includes

calculating standard deviations of a pixel value of the registered image, a pixel value of the reference image, and pixel values of target images for each of pixels;

and

determining pixels of which the calculated standard deviation is a predetermined value or less as pixels for the static objects and determining pixels of which the calculated standard deviation exceeds the predetermined value as pixels for the dynamic objects.

11. The image processing method of claim 10, further comprising excluding the dynamic objects from the transparency-processed image,

wherein in the updating of the map data, the map data is updated based on position information of the static objects in the registered image from which the dynamic objects are excluded.

12. An image processing apparatus comprising:

a receiving unit receiving images acquired from a plurality of vehicles positioned on a road;

a storing unit storing the received images according to acquisition information of the received images;

a controlling unit determining a reference image and a target image based on images having the same acquisition information among the stored images; and

an image processing unit performing an image registration using a plurality of feature points extracted from each of the determined reference image and target image and performing a transparency process for each of the reference image and the target image which are image-registered,

wherein the controlling unit extracts static objects from the transparency-processed image, compares the extracted static objects with objects on map data which is previously stored, and updates the map data when the objects on the map data which is previously stored and the extracted static objects are different from each other.

13. The image processing apparatus of claim 12, wherein the image processing unit generates one synthesized image by extracting the plurality of feature points from each of the determined reference image and target image and performing the image registration for the determined images using the plurality of extracted feature points.

14. The image processing apparatus of claim 12, wherein the plurality of feature points are points at which image brightness value suddenly changes in the reference image or the target image and are edges of pixels or corner points.

15. The image processing apparatus of claim 13, wherein the transparency process multiplies R, G, and B pixel values of respective pixels included in the images for which the transparency process is to be performed by a predetermined value smaller than 1, and

the predetermined value is a reciprocal number of N, which is a total number of the images for which the transparency process is to be performed.

16. The image processing apparatus of claim 12, wherein the acquisition information includes at least one of information on positions at which the images are photographed, information on angles at which the images are photographed, and information on directions in which the images are photographed.

17. The image processing apparatus of claim 13, wherein the image processing unit includes

a feature point extracting unit extracting a plurality of first feature points from the reference image and extracting a plurality of second feature points from the target image;

a feature point matching unit performing a matching operation matching a first feature point group in which the plurality of first feature points are grouped and a second feature point group in which the plurality of second feature points are grouped;

a homography calculating unit calculating a homography using information of matched pairs between the first feature point group and the second feature point group through the matching operation;

an image registration unit converting the target image using the calculated homography and registering the reference image and the converted target image; and

a transparency processing unit performing a transparency process for the registered images.

18. The image processing apparatus of claim 12, wherein

the controlling unit confirms position information of the extracted static objects, examines whether or not objects different from the extracted static objects exist at a position corresponding to the confirmed position information of the static objects in the map data which is previously stored, and updates the map data by reflecting the extracted static objects to the map data which is previously stored corresponding to the position information, when the different objects exists as a result of the examination.

19. The image processing apparatus of claim 12, further comprising a transmitting unit transmitting the updated map data to the plurality of vehicles positioned on the road.

20. A computer readable recording medium in which a program for executing an image processing method is recorded, wherein

the image processing method includes

receiving images acquired from a plurality of vehicles positioned on a road;

storing the received images according to acquisition information of the received images;

determining a reference image and a target image based on images having the same acquisition information among the stored images;

performing an image registration using a plurality of feature points extracted from each of the determined reference image and target image;

performing a transparency process for each of the reference image and the target image which are image-registered;

extracting static objects from the transparency-processed image; and

comparing the extracted static objects with objects on map data which is previously stored and updating the map data when the objects on the map data which is previously stored and the extracted static objects are different from each other.