IMAGE REGISTRATION METHOD, IMAGE REGISTRATION DEVICE AND STORAGE MEDIUM

• US 20200134845A1
• Filed: 07/05/2019
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: Beijing BOE Optoelectronics Technology Co Ltd. (BOE Technology Group Co., Ltd.), BOE Technology Group Co., Ltd.
• Original Assignee: Beijing BOE Optoelectronics Technology Co Ltd. (BOE Technology Group Co., Ltd.), BOE Technology Group Co., Ltd.
• Inventors: WANG, Yunqi, HE, Huidong, CHU, Minglei, CHEN, Lili, ZHANG, Hao
• Granted Patent: US 11,120,565 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 2207/30121 CRT, LCD or plasma display
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G09G 2360/145 the light originating from the display screen
• CPC Class Codes: G09G 3/006 Electronic inspection or testing of displays and display drivers, e.g. of LED or LCD displays testing individual LED's G01R31/2635; testing lamps G01R31/44; testing of optical features of LCD displays G02F1/1309

Abstract

An image registration method, an image registration device, and a storage medium. The image registration method includes: causing a display device to display at least one spot array; obtaining a feature image, and performing a feature-based image registration operation on the feature image to obtain at least one transformed image; and obtaining a mapping model based on the at least one transformed image. The feature image is an image which is shown on the display device and displays the at least one spot array.

Claims

1. An image registration method, comprising:

causing a display device to display at least one spot array;

obtaining a feature image and performing a feature-based image registration operation on the feature image to obtain at least one transformed image; and

obtaining a mapping model based on the at least one transformed image,

wherein the feature image is an image which is shown on the display device and displays the at least one spot array.

2. The image registration method according to claim 1, wherein the causing the display device to display the at least one spot array comprises:

causing the display device to display a plurality of spot arrays which have different sizes and are located in different display areas.

3. The image registration method according to claim 2, wherein the obtaining the feature image and performing the feature-based image registration operation on the feature image to obtain the at least one transformed image comprises:

obtaining the feature image, and performing, on the feature image, spot array feature extracting, spot array feature matching, and transformation model estimating to obtain a transformation model; and

performing image re-sampling and transforming on the feature image based on the transformation model to obtain at least one transformed image.

4. The image registration method according to claim 3, wherein the performing, on the feature image, the spot array feature extracting, the spot array feature matching, and the transformation model estimating to obtain the transformation model comprises:

performing the spot array feature extracting on the feature image to obtain a feature of the feature image;

performing the spot array feature matching based on the feature of the feature image; and

performing the transformation model estimating based on a result of the spot feature matching, to obtain the transformation model.

5. The image registration method according to claim 3, wherein the and performing, on the feature image, the spot array feature extracting, the spot array feature matching, and the transformation model estimating to obtain the transformation model comprises:

capturing an image displayed by the display device by using an image capturing device to obtain the feature image; and

performing the spot array feature extracting, the spot array feature matching, and the transformation model estimating on each spot array in the feature image.

6. The image registration method according to claim 5, wherein the spot array feature matching comprises:

matching a coordinate of a spot of the spot array in the feature image with a coordinate of a pixel or subpixel which emits light in the display device.

7. The image registration method according to claim 5, wherein an imaging resolution of the image capturing device is greater than a display resolution of the display device.

8. The image registration method according to claim 3, wherein

the at least one spot array comprises a plurality of spot arrays, and

the performing the image re-sampling and transforming on the feature image based on the transformation model to obtain the at least one transformed image comprises:

obtaining a plurality of transformation models corresponding to the spot arrays in the feature image respectively based on the transformation model estimating, and performing the image re-sampling and transforming on the feature image by using the plurality of transformation models respectively to obtain a plurality of transformed images.

9. The image registration method according to claim 8, wherein the obtaining the mapping model based on the at least one transformed image comprises:

performing a confidence level assessment on the plurality of transformed images, and selecting the transformation model corresponding to one of the plurality of transformed images based on a result of the confidence level assessment, as the mapping model.

10. The image registration method according to claim 9, wherein the selecting the transformation model corresponding to one of the plurality of transformed images based on the result of the confidence level assessment, as the mapping model comprises:

calculating difference values between coordinates of all spots of spot arrays in each transformed image and coordinates of corresponding spots in the reference image, wherein the reference image is an original image which is displayed by the display device and comprises spot arrays;

calculating a variance of the difference values corresponding to each transformed image to obtain a plurality of variances corresponding to the plurality of transformed images respectively; and

using the plurality of variances as an initial variance and performing a first operation based on the initial variance,

wherein the first operation comprises;

comparing the initial variance with a preset value;

discarding the transformed image corresponding to at least one first variance of the initial variances, discarding, in a remaining transformed image of the plurality of transformed images, the spot array corresponding to the transformation model corresponding to the discarded transformed image, calculating difference values between coordinates of spots of a remaining spot array in the remaining transformed image and coordinates of corresponding spots in the reference image, calculating a variance of the difference values corresponding to each of the remaining transformed image, taking the calculated variance as the initial variance and performing the first operation, in a case where the at least one first variance of the initial variance is greater than a preset value; and

selecting the transformation model corresponding to the transformed image corresponding to a minimal variance in the initial variance as the mapping model, in a case where each variance of the initial variance is all less than or equal to the preset value.

11. The image registration method according to claim 1, wherein in a case where the display device displays one spot array and one transformed image is obtained correspondingly, the obtaining the mapping model based on the at least one transformed image comprises:

taking the transformation model corresponding to the one transformed image as the mapping model.

12. The image registration method according to claim 1, wherein each spot in the spot array is generated by a pixel or subpixel which emits light in the display device.

13. The image registration method according to claim 1, wherein the spot array is arranged in a rectangle.

14. The image registration method according to claim 1, wherein a size of the spot array in column×

row is 2×

2, 2×

3, 3×

3 or 4×

4.

15. The image registration method according to claim 1, further comprising:

aligning a detection image shown on the display device with a reference image based on the mapping model.

17. An image registration device, comprising:

a processor;

a storage; and

one or more computer program modules, the one or more computer program modules being stored in the storage and configured to be executed by the processor, and the one or more computer program modules comprising instructions configured for implementing the image registration method according to claim 1.

18. A non-transient storage medium, configured for storing computer readable instructions, wherein the computer readable instructions, when executed by a computer, cause the computer to execute the image registration method according to claim 1.

16. An image registration device, comprising:

a display control unit, configured to cause the display device to display at least one spot array;

an image registration operation executing unit, configured to obtain a feature image and perform a feature-based image registration operation on the feature image to obtain at least one transformed image; and

a mapping model obtaining unit, configured to obtain a mapping model based on the at least one transformed image,

wherein the feature image is an image which is shown on the display device and displays the at least one spot array.