Systems and methods for obtaining and using augmentation data and for sharing usage data

• US 9,111,384 B2
• Filed: 12/11/2012
• Issued: 08/18/2015
• Est. Priority Date: 10/05/2012
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr., Tegreene, Clarence T.
• Primary Examiner(s): Zarka, David
• Assistant Examiner(s): PRINGLE-PARKER, JASON A
• Application Number:
  US13/711,095
• Publication Number:
  US 20140098132A1
• Time in Patent Office: 980 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Computationally implemented methods and systems include receiving augmentation data associated with one or more first augmentations, the one or more first augmentations having been included in a first augmented view of a first actual scene that was remotely displayed at a remote augmented reality (AR) device, displaying one or more second augmentations in a second augmented view of a second actual scene, the displaying of the one or more second augmentations being in response, at least in part, to the augmentation data, and transmitting to the remote AR device usage data that indicates usage information related at least to usage or non-usage of the received augmentation data. In addition to the foregoing, other aspects are described in the claims, drawings, and text.

Claims

1. A system, comprising:

an augmentation data acquiring module configured to acquire from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device;

an augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene; and

a usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations.

2. The system of claim 1, wherein said augmentation data acquiring module configured to acquire from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device comprises:

an augmentation content information acquiring module configured to acquire data that provides content information related to the one or more first augmentations.

3. The system of claim 1, wherein said augmentation data acquiring module configured to acquire from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device comprises:

a visual item information acquiring module configured to acquire data that provides information related to one or more visual items that are affiliated with the one or more first augmentations and that were present in the first actual scene.

4. The system of claim 1, wherein said augmentation data acquiring module configured to acquire from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device comprises:

an augmentation format information acquiring module configured to acquire data that provides format information related to one or more formats of the one or more first augmentations.

5. The system of claim 4, wherein said augmentation format information acquiring module configured to acquire data that provides format information related to one or more formats of the one or more first augmentations comprises:

a visibility rule acquiring module configured to acquire data that provides one or more visibility rules that include one or more directives for displaying the one or more first augmentations.

6. The system of claim 5, wherein said visibility rule acquiring module configured to acquire data that provides one or more visibility rules that include one or more directives for displaying the one or more first augmentations comprises:

a visibility rule acquiring module configured to acquire data that provides one or more visibility rules that directs the one or more first augmentations to be affiliated with one or more specified visual items and that directs inclusion of the one or more first augmentations in an augmented view of an actual scene.

7. The system of claim 5, wherein said visibility rule acquiring module configured to acquire data that provides one or more visibility rules that include one or more directives for displaying the one or more first augmentations comprises:

a visibility rule acquiring module configured to acquire data that provides one or more visibility rules that directs the one or more first augmentations to be at least semi-transparent in an augmented view of an actual scene in order to prevent one or more priority visual items that are present in the actual scene from being at least completely obfuscated by the one or more first augmentations in the augmented view.

8. The system of claim 7, wherein said visibility rule acquiring module configured to acquire data that provides one or more visibility rules that directs the one or more first augmentations to be at least semi-transparent in an augmented view of an actual scene in order to prevent one or more priority visual items that are present in the actual scene from being at least completely obfuscated by the one or more first augmentations in the augmented view comprises:

a visibility rule acquiring module configured to acquire data that provides one or more visibility rules that directs the one or more first augmentations to be conditionally transparent in the augmented view of the actual scene so long as the one or more priority visual items are detected as being present in the actual scene.

9. The system of claim 5, wherein said visibility rule acquiring module configured to acquire data that provides one or more visibility rules that include one or more directives for displaying the one or more first augmentations comprises:

a visibility rule acquiring module configured to acquire one or more visibility rules that direct display of one or more functional augmentations.

10. The system of claim 9, wherein said visibility rule acquiring module configured to acquire one or more visibility rules that direct display of one or more functional augmentations comprises:

a visibility rule acquiring module configured to acquire data that provides one or more visibility rules that direct display of one or more functional augmentations that are in a form of one or more functional tags that when selected or activated results in display of information related to one or more visual items present in an augmented view and/or in an actual scene.

11. The system of claim 5, wherein said visibility rule acquiring module configured to acquire data that provides one or more visibility rules that include one or more directives for displaying the one or more first augmentations comprises:

a visibility rule acquiring module configured to acquire data that provides one or more visibility rules that directs one or more identified priority visual items to have visibility priority over the one or more first augmentations in an augmented view of an actual scene.

12. The system of claim 1, wherein said augmentation data acquiring module configured to acquire from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device comprises:

a request transmitting module configured to transmit a request to the remote AR device for the augmentation data.

13. The system of claim 1, wherein said augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene comprises:

an augmentation presenting module configured to present locally by presenting the one or more second augmentations in the second augmented view via one or more display monitors.

14. The system of claim 1, wherein said augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene comprises:

an augmentation presenting module configured to present one or more second augmentations in the second augmented view that are same or substantially same as the one or more first augmentations that were remotely displayed at the remote AR device.

15. The system of claim 1, wherein said augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene comprises:

a modified augmentation presenting module configured to present one or more second augmentations in the second augmented view that is or are modified version or versions of the one or more first augmentations that were remotely displayed at the remote AR device.

16. The system of claim 1, wherein said augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed.

17. The system of claim 16, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on content information and one or more visibility rules provided by the received augmentation data and based on the environmental context in which the second augmented view is to be displayed.

18. The system of claim 16, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on content information and one or more visibility rules provided by the received augmentation data, based on one or more visibility rules provided by one or more sources other than the remote AR device, and based on the environmental context in which the second augmented view is to be displayed.

19. The system of claim 18, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on content information and one or more visibility rules provided by the received augmentation data, based on one or more visibility rules provided by one or more sources other than the remote AR device, and based on the environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on content information and one or more visibility rules provided by the received augmentation data, based on one or more visibility rules that were locally provided, and based on the environmental context in which the second augmented view is to be displayed.

20. The system of claim 16, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on content information provided by the received augmentation data, based on one or more locally provided visibility rules, and based on the environmental context in which the second augmented view is to be displayed.

21. The system of claim 16, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on social networking data that indicates the environmental context in which the second augmented view is to be displayed and that were obtained through one or more social networking sources.

22. The system of claim 16, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on audio data indicative of the environmental context in which the second augmented view is to be displayed.

23. The system of claim 16, wherein said augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on environmental context in which the second augmented view is to be displayed comprises:

an augmentation data and context based augmentation providing module configured to provide the one or more second augmentations based on the received augmentation data and based on movement data indicative of the environmental context in which the second augmented view is to be displayed.

24. The system of claim 1, wherein said augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene comprises:

a difference indicator presenting module configured to present one or more indicators that indicate differences between the one or more first augmentations and the one or more second augmentations.

25. The system of claim 24, wherein said difference indicator presenting module configured to present one or more indicators that indicate differences between the one or more first augmentations and the one or more second augmentations comprises:

a modification indicator presenting module configured to present one or more modifications that were applied to the one or more first augmentations in order to generate the one or more second augmentations.

26. The system of claim 17, wherein said difference indicator presenting module configured to present one or more indicators that indicate differences between the one or more first augmentations and the one or more second augmentations comprises:

a degree of correspondence presenting module configured to present one or more degrees of correspondence between the one or more first augmentations and the one or more second augmentations.

27. The system of claim 17, wherein said difference indicator presenting module configured to present one or more indicators that indicate differences between the one or more first augmentations and the one or more second augmentations comprises:

a degree of difference presenting module configured to present one or more degrees of difference between the one or more first augmentations and the one or more second augmentations.

28. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

a usage data relaying module configured to relay back to the remote AR device usage data that indicates that the augmentation data was successfully received.

29. The system of claim 28, wherein said usage data relaying module configured to relay back to the remote AR device usage data that indicates that the augmentation data was successfully received comprises:

a usage data relaying module configured to relay the usage data back to the remote AR device in response to receiving the augmentation data successfully.

30. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

a usage data relaying module configured to relay back to the remote AR device usage data that indicates that the one or more second augmentations have been displayed.

31. The system of claim 30, wherein said usage data relaying module configured to relay back to the remote AR device usage data that indicates that the one or more second augmentations have been displayed comprises:

a usage data relaying module configured to relay back to the remote AR device the usage data in response to the one or more second augmentations being locally displayed.

32. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

an augmentation relaying module configured to relay back to the remote AR device usage data that includes the one or more second augmentations.

33. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

a difference indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more differences between the one or more first augmentations and the one or more second augmentations.

34. The system of claim 33, wherein said difference indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more differences between the one or more first augmentations and the one or more second augmentations comprises:

a modification indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more modifications that were made to the one or more first augmentations in order to generate the one or more second augmentations.

35. The system of claim 33, wherein said difference indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more differences between the one or more first augmentations and the one or more second augmentations comprises:

a degree of correspondence indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more degrees of correspondence between the one or more first augmentations that was presented by the remote AR device and the one or more second augmentations that was presented locally.

36. The system of claim 33, wherein said difference indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more differences between the one or more first augmentations and the one or more second augmentations comprises:

a degree of difference indicating data relaying module configured to relay back to the remote AR device usage data that indicates one or more degrees of difference between the one or more first augmentations that was presented by the remote AR device and the one or more second augmentations that was presented locally.

37. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates changes to the display of the one or more second augmentations.

38. The system of claim 37, wherein said augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates changes to the display of the one or more second augmentations comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations have replaced in the second augmented view the one or more second augmentations.

39. The system of claim 38, wherein said augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations have replaced in the second augmented view the one or more second augmentations comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations that include less content information than the one or more second augmentations have replaced the one or more second augmentations in the second augmented view.

40. The system of claim 38, wherein said augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations have replaced in the second augmented view the one or more second augmentations comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations that are one or more functional tags for retrieving and displaying the one or more second augmentations have replaced the one or more second augmentations in the second augmented view.

41. The system of claim 40, wherein said augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations that are one or more functional tags for retrieving and displaying the one or more second augmentations have replaced the one or more second augmentations in the second augmented view comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that one or more third augmentations that are one or more functional tags for retrieving and displaying the one or more second augmentations have replaced the one or more second augmentations in the second augmented view, the one or more third augmentations that are to replace the one or more second augmentations having substantially no content information included in the one or more second augmentations.

42. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that the one or more second augmentations are no longer being locally presented.

43. The system of claim 42, wherein said augmentation change indicating data relaying module configured to relay back to the remote AR device usage data that indicates that the one or more second augmentations are no longer being locally presented comprises:

an augmentation change indicating data relaying module configured to relay back to the remote AR device the usage data that indicates that the one or more second augmentations are no longer being locally presented, the relaying back to the remote AR device of the usage data being in response to one or more visual items affiliated with the one or more second augmentations no longer being visually detected.

44. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

a usage data relaying module configured to relay back to the remote AR device the usage data, the usage data to facilitate the remote AR device to display one or more third augmentations that directly or indirectly indicate that the transmitted augmentation data was successfully received.

45. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

a usage data relaying module configured to relay back to the remote AR device the usage data, the usage data to facilitate the remote AR device to display one or more third augmentations that directly or indirectly indicate that the one or more second augmentations was displayed.

46. The system of claim 1, wherein said usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the system, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations comprises:

a usage data relaying module configured to relay back to the remote AR device the usage data, the usage data to facilitate the remote AR device to display one or more third augmentations that directly or indirectly indicate one or more differences between the one or more first augmentations and the one or more second augmentations.

47. An article of manufacture, comprising:

a non-transitory storage medium bearing;

one or more instructions for acquiring from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at a the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device;

one or more instructions for presenting locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene; and

one or more instructions for relaying via the one or more wireless and/or networks usage data to the remote AR device, the usage data to be relayed back to the AR device indicating usage information related at least to usage or non-usage of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations.

48. A wearable augmented reality (AR) device, comprising:

a coupling device for coupling the wearable AR device to a user'"'"'s head;

one or more display monitors;

an augmentation data acquiring module configured to acquire from a remote augmented reality (AR) device via one or more wireless and/or networks augmentation data associated with one or more first augmentations that were remotely displayed at a the remote AR device, the one or more first augmentations having been included in a first augmented view of a first actual scene that was displayed by the remote AR device;

an augmentation presenting module configured to present locally one or more second augmentations in response, at least in part, to the acquired augmentation data, the one or more second augmentations to be presented through a second augmented view of a second actual scene; and

a usage data relaying module configured to relay back to the remote AR device, via the one or more wireless and/or networks, usage data, the usage data to be relayed back to the remote AR device indicating usage information related at least to usage or non-usage, by the wearable AR device, of the received augmentation data, the usage data to facilitate the remote AR device to display one or more third augmentations.