IMAGE STITCHING PROCESSING METHOD AND SYSTEM THEREOF

• US 20200134782A1
• Filed: 02/28/2019
• Published: 04/30/2020
• Est. Priority Date: 10/24/2018
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: Wistron Corporation
• Inventors: LAI, Chang-Hsin
• Granted Patent: US 10,789,677 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/304 using merged images, e.g. merging camera image with stored images
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

An image stitching processing method includes: obtaining a first image taken by a first camera module, wherein the first camera module has a first distance from a reference plane; obtaining a second image taken by a second camera module, wherein the second camera module has a second distance from the reference plane; scaling the second image according to a predetermined ratio and stitching the scaled second image with the first image to obtain a third image, wherein the third image has a stitching line between the first image and the scaled second image; determining whether an object image corresponding to an object in the third image is close to or located on the stitching line; identifying the object image to generate an identification confidence value; covering a standard object image on the object image to generate a modified third image; and outputting and displaying the modified third image.

Claims

1. An image stitching processing method, comprising:

obtaining a first image taken by a first camera module, wherein the first camera module is disposed a first distance from a reference plane;

obtaining a second image taken by a second camera module, wherein the second camera module is disposed a second distance from the reference plane, and the second distance is different from the first distance;

scaling the second image according to a predetermined ratio and stitching the scaled second image with the first image to obtain a third image, wherein the third image has a stitching line between the first image and the scaled second image;

determining whether an object image corresponding to an object in the third image is close to or located on the stitching line;

identifying the object image to generate an identification confidence value;

covering a standard object image on the object image to generate a modified third image in response to the object image being close to or located on the stitching line and the identification confidence value of the object image being less than a first predetermined value; and

outputting and displaying the modified third image.

2. The image stitching processing method as claimed in claim 1, wherein the step of determining whether the object image in the third image is close to the stitching line comprises:

defining an observation area by extending an equal distance from both sides of the stitching line; and

determining whether the object image enters the observation area.

3. The image stitching processing method as claimed in claim 1, wherein the step of scaling the second image according to the predetermined ratio and stitching the scaled second image with the first image to obtain the third image comprises:

defining a target area that does not overlap the first image in the second image, and cropping the second image according to the target area to obtain a sub-image;

enlarging the sub-image according to the predetermined ratio to obtain the scaled second image; and

stitching the first image with the scaled second image to obtain the third image.

4. The image stitching processing method as claimed in claim 1, wherein the predetermined ratio is a ratio of the second distance to the first distance.

5. The image stitching processing method as claimed in claim 1, further comprising:

calculating the identification confidence value of the object image in response to the object image being close to or located on the stitching line; and

storing a current object image as the standard object image in response to the identification confidence value being greater than a second predetermined value.

6. The image stitching processing method as claimed in claim 1, wherein the step of covering the standard object image on the object image in response to the object image being close to or located on the stitching line and the identification confidence value of the object image being less than the first predetermined value comprises:

adjusting a height of the standard object image according to a height of the object image corresponding to a part of the first image to obtain an adjusted standard object image, wherein the height of the adjusted standard object image is equal to the height corresponding to the part of the first image; and

covering the adjusted standard object image on the object image.

7. The image stitching processing method as claimed in claim 1, wherein the first camera module and the second camera module are configured on a vehicle, and the image stitching processing method further comprises:

stopping calculating the identification confidence value corresponding to the object image and/or stopping covering the standard object image on the object image in response to a current speed of the vehicle being greater than a predetermined speed.

8. The image stitching processing method as claimed in claim 1, wherein the second image is correspondingly enlarged according to the predetermined ratio and the enlarged second image is stitched with the first image to obtain the third image, when the second distance is greater than the first distance.

9. An image stitching processing system, comprising:

a first camera module, configured to obtain a first image, wherein the first camera module is disposed a first distance from a reference plane;

a second camera module, configured to obtain a second image, wherein the second camera module is disposed a second distance from the reference plane, and the second distance is different from the first distance;

a processing unit, configured to;

receive the first image and the second image from the first camera module and the second camera module;

scale the second image according to a predetermined ratio and stitch the scaled second image with the first image to obtain a third image, wherein the third image has a stitching line between the first image and the scaled second image;

determine whether an object image corresponding to an object in the third image is close to or located on the stitching line;

identify the object image to generate an identification confidence value; and

cover a standard object image on the object image to generate a modified third image in response to the object image being close to or located on the stitching line and an identification confidence value of the object image being less than the first predetermined value;

a storage unit, configured to store the standard object image; and

a display unit, configured to display the modified third image.

10. The image stitching processing system as claimed in claim 9, wherein the processing unit defines an observation area by extending an equal distance from both sides of the stitching line, and determines whether the object image enters the observation area.

11. The image stitching processing system as claimed in claim 9, wherein the processing unit further defines a target area that does not overlap the first image in the second image, and crops the second image according to the target area to obtain a sub-image;

enlarges the sub-image according to the predetermined ratio to obtain the scaled second image; and

stitches the first image with the scaled second image to obtain the third image.

12. The image stitching processing system as claimed in claim 9, wherein the predetermined ratio is the ratio of the second distance to the first distance.

13. The image stitching processing system as claimed in claim 9, wherein:

the processing unit further calculates the identification confidence value of the object image in response to the object image being close to or located on the stitching line; and

the processing unit further stores the current object image as the standard object image in response to the identification confidence value being greater than a second predetermined value.

14. The image stitching processing system as claimed in claim 9, wherein the processing unit further adjusts the height of the standard object image according to the height of the object image corresponding to a part of the first image to obtain an adjusted standard object image, and covers the adjusted standard object image on the object image, wherein the height of the adjusted standard object image is equal to the height corresponding to the part of the first image.

15. The image stitching processing system as claimed in claim 9, wherein the first camera module and the second camera module are configured on a vehicle, and the processing unit further stops calculating the identification confidence value corresponding to the object image and/or stops covering the standard object image on the object image in response to the current speed of the vehicle being greater than a predetermined speed.

16. The image stitching processing system as claimed in claim 9, wherein the processing unit correspondingly enlarges the second image according to the predetermined ratio and the enlarged second image is stitched with the first image to obtain the third image, when the second distance is greater than the first distance.

17. An image stitching processing method, comprising:

obtaining a first image and a second image, wherein the first image is adjacent to the second image or the first image overlaps with at least a portion of the image content of the second image;

scaling at least one of the first image and the second image according to a ratio, and stitching the scaled first image and the scaled second image or stitching the scaled one of the first image and the second images with the other to obtain a third image, wherein the third image has a stitching line between the scaled first image and the scaled second image or stitching the scaled one of the first image and the second images with the other;

determining whether an object image in the third image is located on the stitching line;

identifying the object image to generate an identification confidence value;

covering a standard object image on the object image to generate a modified third image in response to the object image being located on the stitching line and the identification confidence value of the object image being less than the first predetermined value; and

outputting and displaying the modified third image.