Device-provided tracking data for augmented reality

• US 9,886,162 B2
• Filed: 10/06/2014
• Issued: 02/06/2018
• Est. Priority Date: 06/02/2014
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Berelejis, Gabriel, Chandhok, Ravinder Paul
• Primary Examiner(s): Wang, Yuehan
• Application Number:
  US14/507,591
• Publication Number:
  US 20150347850A1
• Time in Patent Office: 1,219 Days
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04Q 2209/43 using wireless personal area networks [WPAN], e.g. 802.15, 802.15.1, 802.15.4, Bluetooth or ZigBee
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

A networked device may provide tracking data associated with the networked device to an augmented reality (AR) device via a local network. The tracking data may describe one or more trackable features of the networked device. The AR device may utilize the tracking data to detect the networked device in a camera view. The AR device may generate an augmented reality view in association with the networked device in response to detecting at least one trackable feature of the networked device in the camera view of the AR device. At least a portion of the augmented reality view can be augmented with AR properties associated with the networked device when the networked device is positioned in the camera view.

Claims

1. A method performed by an augmented reality (AR) device, the method comprising:

receiving tracking data in a device interoperability framework protocol message from a networked device via a local network, wherein the tracking data includes descriptions of one or more visual properties of the networked device; and

generating an augmented reality view in association with the networked device in response to detecting at least one of the one or more visual properties of the networked device in an image acquired from a camera view of the AR device.

2. The method of claim 1, further comprising:

determining, based at least in part on the tracking data, a position of the networked device in the camera view of the AR device.

3. The method of claim 1, wherein generating the augmented reality view includes augmenting at least a portion of the augmented reality view with AR properties associated with the networked device.

4. The method of claim 1, wherein the one or more visual properties includes at least one member selected from the group consisting of size, shape, color, edges, feature points, corners, and flat surfaces.

5. The method of claim 1, wherein receiving the tracking data comprises:

determining that the networked device is capable of providing the tracking data in accordance with a device interoperability framework protocol of the local network;

sending a query to the networked device to request the tracking data from the networked device; and

receiving the tracking data in response to the query.

6. The method of claim 5, further comprising:

determining that the networked device is in proximity of the AR device prior to sending the query to the networked device.

7. The method of claim 1, wherein receiving the tracking data comprises receiving a broadcast message from the networked device, the broadcast message including the tracking data.

8. The method of claim 1, wherein the local network comprises a wireless local area network (WLAN).

9. The method of claim 1, wherein the tracking data comprises a multi-dimensional model for describing the networked device, and wherein the multi-dimensional model enables markerless detection of the networked device.

10. The method of claim 1, wherein the tracking data is received via a short-range radio frequency communication medium.

11. The method of claim 1, wherein the tracking data is included in an augmented reality property data set associated with the networked device, the augmented reality property data set embedded in the networked device.

12. The method of claim 1, wherein generating the augmented reality view comprises:

obtaining the image using an integrated camera at the AR device;

detecting at least one of the one or more visual properties of the networked device in the image; and

displaying at least a portion of the image augmented with AR properties associated with the networked device.

13. The method of claim 1, further comprising:

displaying the augmented reality view using an integrated display apparatus at the AR device.

14. An augmented reality (AR) device, comprising:

a processor; and

memory storing instructions, which when executed by the processor, cause the AR device to;

receive tracking data in a device interoperability framework protocol message from a networked device via a local network, wherein the tracking data includes descriptions of one or more visual properties of the networked device; and

generate an augmented reality view in association with the networked device in response to detecting at least one of the one or more visual properties of the networked device in an image acquired from a camera view of the AR device.

15. The AR device of claim 14, wherein the instructions further cause the processor to:

determine, based at least in part on the tracking data, a position of the networked device in the camera view of the AR device.

16. The AR device of claim 14, wherein the instructions further cause the processor to:

execute an AR application that generates the augmented reality view based on the camera view, wherein the augmented reality view is augmented with AR properties associated with the networked device.

17. The AR device of claim 14, further comprising:

a display apparatus that displays the augmented reality view to a user of the AR device.

18. The AR device of claim 14, further comprising:

a network interface for communicating via the local network,

wherein the instructions, when executed by the processor, further cause the AR device to;

determine that the networked device is capable of providing the tracking data in accordance with a device interoperability framework protocol of the local network;

send a query to the networked device to request the tracking data from the networked device; and

receive the tracking data in response to the query.

19. The AR device of claim 18, wherein the network interface sends the query in response to the AR device determining that the networked device is in proximity of the AR device.

20. The AR device of claim 14, further comprising:

a network interface that receives a broadcast message from the networked device, the broadcast message including the tracking data.

21. A method performed by a networked device, the method comprising:

storing tracking data that includes descriptions of one or more visual properties of the networked device, the tracking data for enabling visual detection of the networked device by an augmented reality (AR) device; and

transmitting the tracking data in a device interoperability framework protocol message to the AR device via a local network.

22. The method of claim 21, wherein the one or more visual properties includes at least one member selected from the group consisting of size, shape, color, edges, feature points, corners, and flat surfaces.

23. The method of claim 21, further comprising:

receiving a query from the AR device requesting the tracking data,

wherein providing the tracking data is in response to the query.

24. The method of claim 21, further comprising:

providing an indication to the AR device that the networked device is capable of providing the tracking data in accordance with a device interoperability framework protocol of the local network.

25. The method of claim 21, further comprising:

broadcasting the tracking data via the local network in accordance with a device discovery protocol.

26. The method of claim 21, wherein the tracking data is included in an augmented reality data set stored at the networked device and provided to the AR device in response to a request from the AR device.

27. The method of claim 21, wherein the tracking data received from the networked device comprises a multi-dimensional model describing the networked device, wherein the multi-dimensional model enables markerless detection of the networked device.

28. A networked device, comprising:

a network interface to communicate via a local network;

a processor; and

a memory storing tracking data that includes descriptions of one or more visual properties of the networked device, the tracking data for enabling visual detection of the networked device by an augmented reality (AR) device,

the memory storing instructions, which when executed by the processor, cause the networked device to;

transmit the tracking data in a device interoperability framework protocol message to the AR device via the local network.

29. The networked device of claim 28, wherein the instructions cause the processor to:

receive a query from the AR device requesting the tracking data, and

provide the tracking data is in response to the query.

30. The networked device of claim 28, wherein the instructions cause the processor to:

provide an indication to the AR device that the networked device is capable of providing the tracking data in accordance with a device interoperability framework protocol of the local network.