Augmented Reality Method and System, and User Mobile Device Applicable Thereto

• US 20160163107A1
• Filed: 06/18/2015
• Published: 06/09/2016
• Est. Priority Date: 12/09/2014
• Status: Abandoned Application

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chen, Szu-Wei, Chen, Yi-Cheng, CHANG, Teng-Wen, Liang, Che-Wei
• Application Number:
  US14/743,810
• Publication Number:
  US 20160163107A1
• Time in Patent Office: Days
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An augmented reality method includes: capturing physical objects in a physical space, to obtain respective depth information of the physical objects; generating respective physical coordinates of the physical objects; generating a 3D map of the physical space; searching an AR deposition corresponding to the physical space; converting respective AR virtual coordinates of a plurality AR objects in the AR deposition into respective physical coordinates of the AR objects in the physical space; judging whether an AR alignment error occurs; if yes, adjusting a first AR virtual coordinate of a first AR object, among the AR objects, which causes the AR alignment error; and performing AR demonstration.

Claims

1. An augmented reality (AR) method, comprising:

capturing a plurality of physical objects in a physical space by a user mobile device to obtain respective depth information of the physical objects;

generating respective physical coordinates of the physical objects by the user mobile device to send to an AR server system;

generating a three-dimension (3D) map of the physical space by the AR server system;

searching an AR deposition corresponding to the physical space by the AR server system;

converting respective AR virtual coordinates of a plurality AR objects in the AR deposition into respective physical coordinates of the AR objects in the physical space;

judging, by the AR server system, whether an AR alignment error occurs;

if yes, adjusting a first AR virtual coordinate of a first AR object, among the AR objects, which causes the AR alignment error, by the AR server system; and

performing AR demonstration by the user mobile device.

2. The AR method according to claim 1, further comprising:

placing a physical marker and the user mobile device having a 3D depth camera in a location of the physical space; and

obtaining respective distances between walls and the location of the physical space by the user mobile device to calculate an area of the physical space.

3. The AR method according to claim 1, wherein:

the AR server system stores the 3D map of the physical space into a physical space database; and

the AR server system searches the AR deposition corresponding to the physical space from an AR deposition database.

4. The AR method according to claim 1, wherein:

the AR server system calculates an area of the AR deposition;

the AR server system finds an area of an AR virtual marker in the AR deposition; and

the AR server system calculates an area of the AR virtual marker in the physical space.

5. The AR method according to claim 1, wherein the step of judging whether the AR alignment error occurs includes:

judging whether the AR alignment error occurs based on the respective physical coordinates of the physical objects and the respective physical coordinates of the AR objects.

6. The AR method according to claim 5, wherein

judging whether the AR alignment error occurs based on judging whether the AR objects block or overlap at least one of the physical objects in the physical space;

or

judging whether the AR alignment error occurs based on judging whether the AR objects pass through at least one wall of the physical space.

7. The AR method according to claim 1, wherein if the first AR object and a second AR object are in pair, then the first AR virtual coordinate of the first AR object and a second AR virtual coordinate of the second AR object are adjusted together.

8. An augmented reality (AR) system, comprising:

a user mobile device, capturing a plurality of physical objects in a physical space to obtain respective depth information of the physical objects; and

an AR server system, coupled to the user mobile device, the AR server system including;

a physical space generation module, receiving the respective physical coordinates of the physical objects generated by the user mobile device to generate a three-dimension (3D) map of the physical space;

an AR object intelligent suggestion module, searching an AR deposition corresponding to the physical space;

an AR space conversion module, converting respective AR virtual coordinates of a plurality AR objects in the AR deposition into respective physical coordinates of the AR objects in the physical space; and

an AR space correction module, judging whether an AR alignment error occurs, if yes, the AR space correction module adjusting a first AR virtual coordinate of a first AR object, among the AR objects, which causes the AR alignment error;

wherein the AR object intelligent suggestion module sends back the adjusted AR deposition to the user mobile device and the user mobile device performs AR demonstration.

9. The AR system according to claim 8, wherein the user mobile device comprises:

a 3D depth camera, for capturing; and

a 3D space generating module,

wherein when a physical marker and the user mobile device are placed in a location of the physical space, the 3D space generating module obtains respective distances between walls and the location of the physical space to calculate an area of the physical space.

10. The AR system according to claim 8, wherein the AR server system further includes a physical space database and an AR deposition database,

the physical space generation module stores the 3D map of the physical space into the physical space database;

and

the AR object intelligent suggestion module searches the AR deposition corresponding to the physical space from the AR deposition database.

11. The AR system according to claim 8, wherein:

the AR space conversion module calculates an area of the AR deposition;

the AR space conversion module finds an area of an AR virtual marker in the AR deposition; and

the AR space conversion module calculates an area of the AR virtual marker in the physical space.

12. The AR system according to claim 8, wherein the AR space correction module judges whether the AR alignment error occurs based on the respective physical coordinates of the physical objects and the respective physical coordinates of the AR objects.

13. The AR system according to claim 12, wherein the AR space correction module judges whether the AR alignment error occurs based on judging whether the AR objects block or overlap at least one of the physical objects in the physical space;

or

the AR space correction module judges whether the AR alignment error occurs based on judging whether the AR objects pass through at least one wall of the physical space.

14. The AR system according to claim 8, wherein if the first AR object and a second AR object are in pair, then the AR space correction module adjusts the first AR virtual coordinate of the first AR object and a second AR virtual coordinate of the second AR object together.

15. A user mobile device, comprising:

a 3D depth camera, capturing a plurality of physical objects in a physical space to obtain respective depth information of the physical objects; and

a 3D space generating module,

wherein when a physical marker and the user mobile device are placed in a location of the physical space, the 3D space generating module obtains respective distances between walls and the location of the physical space to calculate an area of the physical space;

the 3D space generating module generates respective physical coordinates of the physical objects to send to an AR server system;

after receiving the respective physical coordinates of the physical objects from the user mobile device, the AR server system generates a three-dimension (3D) map of the physical space, converts coordinates of a plurality of AR objects, searches and adjusts an AR deposition corresponding to the physical space, sends back the AR deposition to the user mobile device; and

the user mobile device performs AR demonstration.

16. The user mobile device according to claim 15, further comprising:

an AR basic module, receiving the AR deposition from the AR server system;

a screen, coupled to the 3D depth camera and the AR basic module, displaying photos captured by the 3D depth camera and displaying the AR deposition from the AR basic module.