REALIZATION METHOD AND DEVICE FOR TWO-DIMENSIONAL CODE AUGMENTED REALITY

• US 20140210857A1
• Filed: 12/16/2013
• Published: 07/31/2014
• Est. Priority Date: 01/28/2013
• Status: Abandoned Application

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: CHEN, Bo, LIU, Xiao, LIU, Hailong
• Application Number:
  US14/108,214
• Publication Number:
  US 20140210857A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A computer-implemented method for two-dimensional code augmented reality includes: detecting an image capture of a two-dimensional code through a camera video frame; identifying the contour of the two-dimensional code captured in the camera video frame; decoding the information embedded in the detected two-dimensional code; obtaining content information corresponding to the decoded two-dimensional code; tracking the identified contour of the two-dimensional code within the camera video frame to obtain the position information of the two-dimensional code in the camera video frame; performing augmented reality processing on the two-dimensional code based on the content information and the position information; and generating the augmented reality on the device while simultaneously displaying real-world imagery on the display of the device, wherein any visual augmented reality is displayed in accordance with the location of the two-dimensional code in the video frame.

Claims

1. A method of generating augmented reality at an electronic device with a camera and a display, comprising:

detecting an image capture of a two-dimensional code through a camera video frame;

identifying the contour of the two-dimensional code captured in the camera video frame;

decoding the information embedded in the detected two-dimensional code;

obtaining content information corresponding to the decoded two-dimensional code;

tracking the identified contour of the two-dimensional code within the camera video frame to obtain the position information of the two-dimensional code in the camera video frame;

performing augmented reality processing on the two-dimensional code based on the content information and the position information; and

generating the augmented reality on the device while simultaneously displaying real-world imagery on the display of the device, wherein any visual augmented reality is displayed in accordance with the location of the two-dimensional code in the video frame.

2. The method of claim 1, wherein the two-dimensional code is a quick-response (QR) code.

3. The method of claim 2, wherein detecting image capture of a two-dimensional code through a camera imaging frame includes converting the image capture to grayscale and converting the grayscale image capture to a binary image capture;

and

identifying the contour of the two-dimensional code further includes;

executing the horizontal anchor point characteristic scanning and vertical anchor point characteristic scanning against this binary image;

obtaining a horizontal anchor point characteristic line and a vertical anchor point characteristic line;

calculating the intersection point between the horizontal anchor point characteristic line and vertical anchor point characteristic line;

obtaining the position of an anchor point of the QR two-dimensional code, corresponding to the calculated intersection point;

obtaining the contour of QR two-dimensional code in accordance with the calculated position of the anchor point.

4. The method of claim 3, wherein tracking the identified contour of the two-dimensional code within the imaging frame to obtain the position information of the two-dimensional code in the imaging frame further includes:

obtaining an initial camera video grayscale frame and calculating an initial tracking point aggregation within the contour of the two-dimensional code;

obtaining a current camera video grayscale frame, a previous tracking point aggregation and previous camera video grayscale frame, in accordance with a determination that the initial tracking point aggregation number is greater than a predetermined threshold value;

calculating a current tracking point aggregation, tracked by the current camera video frame image, by applying the current camera video grayscale frame, previous tracking point aggregation and previous camera video grayscale frame in optic flow tracking modes;

calculating a homography matrix in accordance with corresponding dotted pairs of the initial tracking point aggregation and current tracking point aggregation.

5. The method of claim 4, wherein calculating a homography matrix further includes:

determining that the current tracking point aggregation does not exceed the predetermined threshold value of the initial tracking point aggregation; and

calculating the homography matrix in accordance with a determination that the current tracked number of camera video frame images is less than the preset threshold value.

6. The method of claim 1, further comprising:

performing down sampling processing against the camera video frame image and reattempting to detect the two-dimensional code in the camera video frame image, in accordance with a determination that no two-dimensional code is detected in a camera video frame image of the camera video frame.

7. The method of claim 1, further comprising:

terminating the presentation of the augmented reality on the device, in accordance with a determination that no two-dimensional code is detected in a camera video frame image of the camera video frame.

8. The method of claim 1, wherein the generating the augmented reality on the device further comprises:

displaying the augmented reality on the display of the device and in the area occupied by the two-dimensional code in the camera video frame.

9. The method of claim 8, wherein the displaying the augmented reality on the display of the device and only in the area occupied by the two-dimensional code in the camera video frame further comprises:

converting the size of the augmented reality into the size of the captured two-dimensional code image in the camera video frame.

10. The method of claim 8, wherein the displayed augmented reality is a three-dimensional representation, based on the content and position information of the two-dimensional code.

11. The method of claim 10, wherein the displaying the augmented reality on the display of the device and only in the area occupied by the two-dimensional code in the camera video frame further comprises:

calculating a transformation matrix of polar coordinates of a three-dimensional model to the two-dimensional coordinates of the display screen; and

using the transformation matrix to overlay the three-dimensional model in the camera video frame image according to the position information of the two-dimensional code in camera video frame image.

12. An electronic device, comprising:

a display;

a camera;

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for;

detecting an image capture of a two-dimensional code through a camera video frame;

identifying the contour of the two-dimensional code captured in the camera video frame;

decoding the information embedded in the detected two-dimensional code;

obtaining content information corresponding to the decoded two-dimensional code;

tracking the identified contour of the two-dimensional code within the camera video frame to obtain the position information of the two-dimensional code in the camera video frame;

performing augmented reality processing on the two-dimensional code based on the content information and the position information; and

generating the augmented reality on the device while simultaneously displaying real-world imagery on the display of the device, wherein any visual augmented reality is displayed in accordance with the location of the two-dimensional code in the video frame.

13. The device of claim 12, wherein the two-dimensional code is a quick-response (QR) code.

14. The device of claim 13, wherein detecting image capture of a two-dimensional code through a camera imaging frame includes converting the image capture to grayscale and converting the grayscale image capture to a binary image capture;

and

identifying the contour of the two-dimensional code further includes;

executing the horizontal anchor point characteristic scanning and vertical anchor point characteristic scanning against this binary image;

obtaining a horizontal anchor point characteristic line and a vertical anchor point characteristic line;

calculating the intersection point between the horizontal anchor point characteristic line and vertical anchor point characteristic line;

obtaining the position of an anchor point of the QR two-dimensional code, corresponding to the calculated intersection point;

obtaining the contour of QR two-dimensional code in accordance with the calculated position of the anchor point.

15. The device of claim 12, further including instructions for:

performing down sampling processing against the camera video frame image and attempting to detect the two-dimensional code in the camera video frame image, in accordance with a determination that no two-dimensional code is detected in a camera video frame image of the camera video frame.

16. The device of claim 12, further including instructions for:

terminating the presentation of the augmented reality on the device, in accordance with a determination that no two-dimensional code is detected in a camera video frame image of the camera video frame.

17. The device of claim 12, wherein the generating the augmented reality on the device further comprises instructions for:

displaying the augmented reality on the display of the device and in the area occupied by the two-dimensional code in the camera video frame.

18. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display and a camera, cause the device to:

detect an image capture of a two-dimensional code through a camera video frame;

identify the contour of the two-dimensional code captured in the camera video frame;

decode the information embedded in the detected two-dimensional code;

obtain content information corresponding to the decoded two-dimensional code;

track the identified contour of the two-dimensional code within the camera video frame to obtain the position information of the two-dimensional code in the camera video frame;

perform augmented reality processing on the two-dimensional code based on the content information and the position information; and

generate the augmented reality on the device while simultaneously displaying real-world imagery on the display of the device, wherein any visual augmented reality is displayed in accordance with the location of the two-dimensional code in the video frame.

19. The non-transitory computer readable storage medium of claim 18, wherein the two-dimensional code is a quick-response (QR) code.

20. The non-transitory computer readable storage medium of claim 18, further comprising instructions that cause the device to:

perform down sampling processing against the camera video frame image and attempting to detect the two-dimensional code in the camera video frame image, in accordance with a determination that no two-dimensional code is detected in a camera video frame image of the camera video frame.

21. The non-transitory computer readable storage medium of claim 18, further comprising instructions that cause the device to:

terminate the presentation of the augmented reality on the device, in accordance with a determination that no two-dimensional code is detected in a camera video frame image of the camera video frame.

22. The non-transitory computer readable storage medium of claim 18, wherein the generating the augmented reality on the device further comprises instructions that cause the device to:

display the augmented reality on the display of the device and in the area occupied by the two-dimensional code in the camera video frame.