Collaborative augmented reality system

• US 10,565,764 B2
• Filed: 04/09/2018
• Issued: 02/18/2020
• Est. Priority Date: 04/09/2018
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.), Hong Kong University of Science and Technology
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Han, Bo, Gopalakrishnan, Vijay, Zavesky, Eric, Zhang, Wenxiao, Hui, Pan
• Primary Examiner(s): Mushambo, Martin
• Application Number:
  US15/948,697
• Publication Number:
  US 20190311513A1
• Time in Patent Office: 680 Days
• Field of Search: None
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/30 Profiles
• CPC Class Codes: H04L 67/306 User profiles

Abstract

An augmented reality device computationally processes an image frame against a first augmented reality profile stored in a local database. The first augmented reality profile includes first annotation content associated with a first object and is obtained from a second user device. In response to an object computationally processed from the first image frame satisfying a predetermined threshold of similarity with the first object in the first augmented reality profile, the first annotation content is rendered, on a display of the first user device, relative to the first object according to rendering instructions for the first annotation content.

Claims

1. A collaborative, multi-device augmented reality (AR) method comprising:

at a first AR device, presenting to a user thereof first information that visually augments at least one object computationally recognized in a segmented visual field thereof, the first information retrieved over a wide-area network from a network service platform based on correspondence of the computationally recognized object with an object descriptor maintained within the network service platform; and

at the first AR device, presenting to the user thereof second information that visually augments at least one other object computationally recognized in the segmented visual field, wherein the second information is not directly retrieved from the network service platform, but rather is received from a second AR device locationally proximate to the first AR device based on prior computational recognition at the second AR device of a corresponding object within a visual field of the second AR device.

2. The method of claim 1, wherein the prior computational recognition at the second AR device of the at least one other object was performed by the network service platform and provided to the second AR device by the network service platform.

3. The method of claim 1, further comprising:

providing the object descriptor of the at least one object computationally recognized and the first information to the second AR device.

4. The method of claim 3, further comprising:

providing third information that the first AR device associated with the object computationally recognized in the segmented visual field to the second AR device with the object descriptor.

5. The method of claim 1, wherein the first information retrieved over a wide-area network from the network service platform based on correspondence of the computationally recognized object with an object descriptor maintained within the network service platform is performed subsequently to determining that the second AR device locationally proximate to the first AR device does not include prior computational recognition at the second AR device of the corresponding object within the visual field of the second AR device.

6. The method of claim 1, further comprising:

modifying, at the first AR device, the second information to third information based on received user input instruction; and

providing, by the first AR device, the third information to any AR device consuming the second information.

7. The method of claim 1, further comprising:

providing, by the first AR device, location data of the first AR device to the network service platform over time, wherein the first information is retrieved over the wide-area network from the network service platform based on a known location of the at least one object, a current location of the first AR device, and a direction that the first AR device is travelling within a physical environment.

8. A collaborative augmented reality system, comprising:

a communication system;

a processor coupled to the communication system; and

a memory system coupled to the processor and storing instructions which, when executed by the processor, cause the processor to perform operations comprising;

presenting to a user thereof first information that visually augments at least one object computationally recognized in a segmented visual field thereof, the first information retrieved over a wide-area network from a network service platform based on correspondence of the computationally recognized object with an object descriptor maintained within the network service platform; and

presenting to the user thereof second information that visually augments at least one other object computationally recognized in the segmented visual field, wherein the second information is not directly retrieved from the network service platform, but rather is received from an augmented reality (AR) device locationally proximate to the communication system based on prior computational recognition at the AR device of a corresponding object within a visual field of the AR device.

9. The system of claim 8, wherein the prior computational recognition at the AR device of the at least one other object was performed by the network service platform and provided to the AR device by the network service platform.

10. The system of claim 8, wherein the operations further comprise:

providing the object descriptor of the at least one object computationally recognized and the first information to the AR device.

11. The system of claim 10, wherein the operations further comprise:

providing, via the communication system, third information that the processor associated with the object computationally recognized in the segmented visual field to the AR device with the object descriptor.

12. The system of claim 8, wherein the first information retrieved over the wide-area network from the network service platform based on correspondence of the computationally recognized object with the object descriptor maintained within the network service platform is performed subsequently to determining that the AR device locationally proximate to the communication system does not include prior computational recognition at the AR device of the corresponding object within the visual field of the AR device.

13. The system of claim 8, wherein the operations further comprise:

modifying the second information to third information based on received user input instruction; and

providing the third information to any AR device consuming the second information.

14. A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising:

at a first Augmented Reality (AR) device, presenting to a user thereof first information that visually augments at least one object computationally recognized in a segmented visual field thereof, the first information retrieved over a wide-area network from a network service platform based on correspondence of the computationally recognized object with an object descriptor maintained within the network service platform; and

at the first AR device, presenting to the user thereof second information that visually augments at least one other object computationally recognized in the segmented visual field, wherein the second information is not directly retrieved from the network service platform, but rather is received from a second AR device locationally proximate to the first AR device based on prior computational recognition at the second AR device of a corresponding object within a visual field of the second AR device.

15. The non-transitory machine-readable medium of claim 14, wherein the prior computational recognition at the second AR device of the at least one other object was performed by the network service platform and provided to the second AR device by the network service platform.

16. The non-transitory machine-readable medium of claim 14, wherein the operations further comprise:

providing the object descriptor of the at least one object computationally recognized and the first information to the second AR device.

17. The non-transitory machine-readable medium of claim 16, wherein the operations further comprise:

providing third information that the first AR device associated with the object computationally recognized in the segmented visual field to the second AR device with the object descriptor.

18. The non-transitory machine-readable medium of claim 14, wherein the first information retrieved over the wide-area network from the network service platform based on correspondence of the computationally recognized object with the object descriptor maintained within the network service platform is performed subsequently to determining that the second AR device locationally proximate to the first AR device does not include prior computational recognition at the second AR device of the corresponding object within the visual field of the second AR device.

19. The non-transitory machine-readable medium of claim 14, wherein the operations further comprise:

modifying the second information to third information based on received user input instruction; and

providing the third information to any AR device consuming the second information.

20. The non-transitory machine-readable medium of claim 14, wherein the operations further comprise:

providing location data of the first AR device to the network service platform over time, wherein the first information is retrieved over the wide-area network from the network service platform based on a known location of the at least one object, a current location of the first AR device, and a direction that the first AR device is travelling within a physical environment.