Systems and methods for rendering a virtual content object in an augmented reality environment

• US 10,661,170 B2
• Filed: 10/17/2018
• Issued: 05/26/2020
• Est. Priority Date: 10/27/2017
• Status: Active Grant

• Current Assignee: Nicholas T. Hariton
• Original Assignee: Nicholas T. Hariton
• Inventors: Hariton, Nicholas T.
• Primary Examiner(s): Laneau, Ronald
• Application Number:
  US16/162,947
• Publication Number:
  US 20190126149A1
• Time in Patent Office: 587 Days
• Field of Search: None
• CPC Class Codes: A63F 13/216 using geographical information, e.g. location of the game device or player using GPS
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/822 Strategy games; Role-playing games A63F13/825, A63F13/828 take precedence
• CPC Class Codes: A63F 2003/00996 Board games available as video games
• CPC Class Codes: A63F 2300/8076 Shooting
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G07C 15/006 electronically
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

Systems and methods for rendering of a virtual content object in an augmented reality environment based on a physical marker are discussed herein. Virtual content objects may be rendered by a display device in an augmented reality environment based on the field of view seen through the display device and a position of a marker (and one or more linkage points associated with the marker) in the real world. When rendered in the augmented reality environment, the virtual content objects may be visualized from any angle, from the exterior or interior of the object, and manipulated in response to user input. Virtual content objects and/or user visualizations of virtual content objects may be shared with other users (local and/or remote), enabling multiple users to potentially build, modify, and/or interact with a virtual content object simultaneously and/or cooperatively.

Claims

1. A system configured to render a virtual content object in an augmented reality environment based on a physical marker, the system comprising:

a presentation device configured to convey information to a user such that images of virtual content are perceived by the user in the augmented reality environment as though the virtual content were present in the real world;

electronic storage configured to store virtual content information defining one or more virtual content objects and a correlation between multiple linkage points and a reference frame of each virtual content object, wherein a position of each of the multiple linkage points in the real world defines the reference frame with respect to the real world;

one or more physical computer processors configured by computer readable instructions to;

obtain first virtual content information from the electronic storage, wherein the first virtual content information defines the virtual content object based on multiple parameters;

identify one or more linkage points visible within a field of view of a user via the presentation device and defined with respect to an object in the real world;

generate an image of the virtual content object to be presented in the augmented reality environment, wherein the image of the virtual content object is generated based on the first virtual content information, the field of view of the user, and the correlation between the multiple linkage points and the reference frame of the virtual content object; and

cause the image to be presented in the augmented reality environment via the presentation device.

2. The system of claim 1, wherein the one or more processors are further configured to:

obtain user input from the user indicating a request to modify a first parameter of the multiple parameters of the virtual content object;

generate second virtual content information defining the virtual content object based on the request, wherein the second virtual content information is different than the first virtual content information;

cause the second virtual content information to be stored in electronic storage;

generate a modified image of the virtual content object based on the second virtual content information; and

cause the modified image of the virtual content object to be presented in the augmented reality environment via the presentation device based on the request.

3. The system of claim 1, wherein the one or more processors are further configured to:

receive an indication of multiple virtual content objects stored in the electronic storage, wherein the multiple virtual content objects include the virtual content object;

cause a selectable list of the multiple virtual content objects to be presented to the user; and

receive a selection indicating the virtual content object, wherein the virtual content information is obtained based on the selection.

4. The system of claim 1, wherein the field of a view of the user is defined based on location information and orientation information, the location information indicating at least a current location associated with the presentation device, and the orientation information indicating at least a pitch angle, a roll angle, and a yaw angle associated with the presentation device.

5. The system of claim 1, wherein the image of the virtual content object is generated based on a position of the presentation device, wherein the one or more processors are further configured to:

obtain an indication of an updated position of the presentation device in the real world at a second time, wherein the updated position of the presentation device is different than the position of the presentation device; and

cause an updated image of the virtual content object to be presented in the augmented reality environment via the presentation device in real-time, wherein the updated image is generated based on the updated position of the presentation device in the real world.

6. The system of claim 2, wherein the one or more processors are further configured to:

cause the first virtual content information defining the virtual content object to be accessible by a second user device of a second user, wherein the virtual content object is viewable via the presentation device and a second presentation device associated with the second user device simultaneously.

7. The system of claim 6, wherein the one or more processors are further configured to:

receive an indication of a second modified image of the virtual content object, wherein the second modified image is different than the modified image and is based on second user input from the second user; and

cause the second modified image of the virtual content object to be presented in the augmented reality environment via the presentation device.

8. The system of claim 6, wherein the virtual content object is simultaneously viewable via the presentation device based on the one or more linkage points and via the second presentation device based on one or more second linkage points, wherein the one or more second linkage points are different than the one or more linkage points.

9. The system of claim 6, wherein the first virtual content information defines access controls for the virtual content object, the access controls identifying one or more rules for modifying the virtual content object and an administrative user of the user and the second user, wherein the one or more processors are further programmed to:

receive an indication of a request to modify the first parameter of the multiple parameters of the virtual content object from the second user, wherein the request from the user and the request from the second user are conflicting;

cause a prompt to be provided to the administrative user requesting an indication of whether the request to modify from the user or the request to modify from the second user is approved; and

cause the second virtual content information to be generated based on the request from the user responsive to user input received from the administrative user indicating that the request to modify from the user is approved.

10. The system of claim 6, wherein the first virtual content information defines access controls for the virtual content object, the access controls identifying one or more rules for modifying the virtual content object and an administrative user of the user and the second user that must approve all requests to modify a virtual content object before they are implemented.

11. The system of claim 2, wherein the one or more processors are further configured to:

cause the modified image of the virtual content object to be transmitted to a user device of a second user remote from the presentation device, wherein the transmitted modified image of the virtual content object is viewable via a second presentation device of the second user.

12. The system of claim 11, wherein the transmitted image of the modified image of the virtual content object is viewable via the second presentation device in real-time.

13. The system of claim 6, wherein a second image of the virtual content object presented to a second user via a second presentation device is based on a second field of view of the second user, wherein the image of the virtual content object is different than the second image of the virtual content object such that perspective of the user of the virtual content object and perspective of the second user of the virtual content object are different.

14. The system of claim 1, wherein the image of the virtual content object presented in the augmented reality environment is perceived as being the same size, a larger size, or a smaller size as an object depicted by the virtual content object would appear in the real world.

15. The system of claim 1, wherein the virtual content object is constructed based on a template associated with one or more predefined parameters, the one or more predefined parameters comprising at least an identification of the correlation between the multiple linkage points and the reference frame of the virtual content object.

16. The system of claim 1, wherein the virtual content object comprises an interactive space associated with a game, and wherein the one or more processors are further configured to:

obtain second virtual content information defining a second virtual content object comprising die, wherein the second virtual content object is associated with the game and a first random number generator;

obtain third virtual content information defining a third virtual content object comprising die, wherein the third virtual content object is associated with the game and a second random number generator;

generate an image of the second virtual content object and an image of the third virtual content object to be presented in the augmented reality environment;

receive user input indicating a request to roll the dice depicted by the second virtual content object and the third virtual content object;

responsive to the request to roll the dice, obtain an output of the first random number generator and an output of the second random number generator;

cause the image of the second virtual content object to be animated, wherein animating the image of the second virtual content object causes the second virtual content object to appear to roll and an outcome of the roll is based on the output of the first random number generator, and wherein the animation of the image of the second virtual content object is based on a correlation between the reference frame of the virtual content object and a reference frame of the second virtual content object; and

cause the image of the third virtual content object to be animated, wherein animating the image of the third virtual content object causes the third virtual content object to appear to roll and an outcome of the roll is based on the output of the second random number generator, and wherein the animation of the image of the third virtual content object is based on a correlation between the reference frame of the virtual content object and a reference frame of the third virtual content object.

17. The system of claim 1, wherein multiple parameters include sounds associated with the virtual content object, animations associated with the virtual content object, and/or haptic features of the virtual content object.

18. A method of rendering a virtual content object in an augmented reality environment based on a physical marker, the method comprising:

obtaining first virtual content information from an electronic storage, wherein the first virtual content information defines the virtual content object and a correlation between multiple linkage points and a reference frame of the virtual content object, wherein the virtual content object is defined based on multiple parameters, and wherein a position of each of the multiple linkage points in the real world defines the reference frame with respect to the real world;

identifying one or more linkage points visible within a field of view of a user via a presentation device and defined with respect to an object in the real world, wherein the presentation device is configured to generate light and provide the light to an eye of a user such that the light forms images of virtual content configured to be perceived in the augmented reality environment as if it were present in the real world;

generating an image of the virtual content object to be presented in the augmented reality environment, wherein the image of the virtual content object is generated based on the first virtual content information, the field of view of the user, and the correlation between the multiple linkage points and the reference frame of the virtual content object; and

causing the image to be presented in the augmented reality environment via the presentation device.

19. The method of claim 18, the method further comprising:

obtaining user input from the user indicating a request to modify a first parameter of the multiple parameters of the virtual content object;

generating second virtual content information defining the virtual content object based on the request, wherein the second virtual content information is different than the first virtual content information;

causing the second virtual content information to be stored in electronic storage;

generating a modified image of the virtual content object based on the second virtual content information; and

causing the modified image of the virtual content object to be presented in the augmented reality environment via the presentation device based on the request.

20. The method of claim 18, the method further comprising:

receiving an indication of multiple virtual content objects stored in the electronic storage, wherein the multiple virtual content objects include the virtual content object;

causing a selectable list of the multiple virtual content objects to be presented to the user; and

receiving a selection indicating the virtual content object, wherein the virtual content information is obtained based on the selection.

21. The method of claim 18, wherein the field of a view of the user is defined based on location information and orientation information, the location information indicating at least a current location associated with the presentation device, and the orientation information indicating at least a pitch angle, a roll angle, and a yaw angle associated with the presentation device.