METHODS AND APPARATUS FOR AUGMENTED REALITY APPLICATIONS

• US 20160291922A1
• Filed: 04/05/2016
• Published: 10/06/2016
• Est. Priority Date: 04/06/2015
• Status: Active Grant

• Current Assignee: Scope Technologies US Inc.
• Original Assignee: Scope Technologies US Inc.
• Inventors: MONTGOMERIE, Scott, NEDOHIN, David, MELLEY, Graham
• Granted Patent: US 10,360,729 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G09B 5/06 with both visual and audible presentation of the material to be studied
• CPC Class Codes: G09B 5/125 the stations being mobile
• CPC Class Codes: G09B 5/14 with provision for individual teacher-student communication
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2380/14 Electronic books and readers
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04W 4/18 Information format or content conversion, e.g. adaptation by the network of the transmitted or received information for the purpose of wireless delivery to users or terminals

Abstract

A method of interaction using augmented reality includes capturing a first video image using a camera, generating first augmented reality (AR) coordinates corresponding to the first video image, transmitting AR coordinates and first video image to remote user, receiving first video image and annotations from remote user, capturing a second video image using a camera, generating second AR coordinates corresponding to the second video image, and viewing annotations registered to second video image.

Claims

1. A method of interaction using augmented reality, the method comprising:

capturing a first video image using an imaging device;

generating first augmented reality (AR) coordinates corresponding to the first video image;

transmitting AR coordinates and first video image to a remote user;

receiving annotations from the remote user;

capturing a second video image using the camera;

generating second AR coordinates corresponding to the second video image; and

viewing the received annotations registered to the second video image.

2. The method of claim 1, wherein each step of the method is repeated for each task in a predefined workflow.

3. The method of claim 1, further comprising recording metadata of the interaction.

4. The method of claim 3, wherein the recorded metadata includes one or more of identities of the remote user and a local user, an elapsed time for completion of a task in the AR interaction, measurements of values related to the task in the AR interaction, check list items associated with the task in the AR interaction, and screenshots of the AR interaction.

5. The method of claim 1, wherein the receiving and viewing operations are performed in real time or in near-real time.

6. The method of claim 1, further comprising transmitting between a local user and the remote user audio, video, or other data.

7. The method of claim 1, wherein the viewing is performed on a device chosen from a group including a tablet, a mobile phone, a laptop computer, a head-mounted display and a virtual-reality display.

8. A method of interaction using augmented reality, the method comprising:

capturing a video image using an imaging device;

generating augmented reality (AR) coordinates corresponding to the video image;

updating a scene view according to the generated AR coordinates;

encoding the video image;

loading 3D object information from external storage;

combining the AR coordinates, 3D object information and encoded video image; and

transmitting the combined data to a remote user.

9. The method of claim 8 wherein each step of the method is repeated for each task in a predefined workflow.

10. The method of claim 8, further comprising recording metadata of the interaction.

11. The method of claim 10, wherein the recorded metadata includes one or more of identities of the remote user and a local user, an elapsed time for completion of a task in the AR interaction, measurements of values related to the task in the AR interaction, check list items associated with the task in the AR interaction, and screenshots of the AR interaction.

12. The method of claim 8, further comprising transmitting between a local user and the remote user audio, video, or other data.

13. The method of claim 8, further comprising viewing the combined data on a device chosen from a group including a tablet, a mobile phone, a laptop computer, a head-mounted display and a virtual-reality display.

14. A method of interaction using augmented reality, the method comprising:

receiving combined augmented reality (AR) coordinates, 3D object information and encoded video frame from a remote user;

updating a background image based on the encoded video frame;

updating a scene view according to the AR coordinates;

updating positions and existence of the 3D objects;

loading 3D content from external storage;

creating an updated rendered view by combining the loaded 3D content and the background image; and

returning the updated rendered view to the remote user.

15. The method of claim 14, wherein each step of the method is repeated for each task in a predefined workflow.

16. The method of claim 14, further comprising recording metadata of the interaction.

17. The method of claim 16, wherein the recorded metadata includes one or more of identities of the remote user and a local user, an elapsed time for completion of a task in the AR interaction, measurements of values related to the task in the AR interaction, check list items associated with the task in the AR interaction, and screenshots of the AR interaction.

18. The method of claim 14, wherein the operations of the method are performed in real time or in near-real time.

19. The method of claim 14, further comprising transmitting between a local user and the remote user audio, video, or other data.

20. The method of claim 14, further comprising viewing the updated rendered view on a device chosen from a group including a tablet, a mobile phone, a laptop computer, a head-mounted display and a virtual-reality display.