Methods and Systems for Dynamically Customizing a Scene for Presentation to a User

• US 20180122142A1
• Filed: 10/31/2016
• Published: 05/03/2018
• Est. Priority Date: 10/31/2016
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Egeler, Christian, Hansen, Xavier, Khalid, Mohammad Raheel
• Granted Patent: US 10,388,072 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/61 Scene description

Abstract

An exemplary scene customization system detects a selection, by a user, of a customization scheme to be applied to an original immersive scene throughout a scene customization session to be experienced by the user by way of an artificial reality device. The system dynamically generates a custom immersive scene based on an application of the selected customization scheme to the original immersive scene by receiving data representative of the original immersive scene, developing a dynamic volumetric model of the original immersive scene based on the received data representative of the original immersive scene, identifying one or more objects included in the original immersive scene, and replacing the one or more objects with one or more custom objects in accordance with the applied customization scheme. The system also presents the custom immersive scene to the user by way of the artificial reality device throughout the scene customization session.

Claims

1. A method comprising:

detecting, by a scene customization system, a selection, by a user, of a customization scheme to be applied to an original immersive scene throughout a scene customization session to be experienced by the user by way of an artificial reality device;

dynamically generating, by the scene customization system and based on an application of the selected customization scheme to the original immersive scene, a custom immersive scene by

receiving data representative of the original immersive scene,

developing, based on the received data representative of the original immersive scene, a dynamic volumetric model of the original immersive scene,

identifying, within the dynamic volumetric model of the original immersive scene, one or more objects included in the original immersive scene, and

replacing, within the dynamic volumetric model and in accordance with the applied customization scheme, the one or more identified objects with one or more custom objects; and

presenting, by the scene customization system throughout the scene customization session, the custom immersive scene to the user by way of the artificial reality device.

2. The method of claim 1, wherein:

the identifying of the one or more identified objects included in the original immersive scene includes;

identifying, based on the received data representative of the original immersive scene, one or more features of a particular object included in the one or more identified objects,

matching the one or more features of the particular object included in the one or more identified objects with one or more corresponding features of a known object documented in an object library associated with the selected customization scheme, and

determining, based on the matching of the one or more features, that the particular object is an instance of the known object documented in the object library; and

the replacing of the one or more identified objects includes replacing the particular object determined to be the instance of the known object with a particular custom object that is predetermined, within the selected customization scheme, to correspond to the known object.

3. The method of claim 1, wherein:

the identifying of the one or more identified objects included in the original immersive scene includes;

identifying, based on the received data representative of the original immersive scene, one or more features of a particular object included in the one or more identified objects,

determining that the one or more features of the particular object included in the one or more identified objects are characteristic of a known object type documented in an object library associated with the selected customization scheme, and

determining, based on the determining that the one or more features are characteristic of the known object type, that the particular object is of the known object type documented in the object library; and

the replacing of the one or more identified objects includes replacing the particular object determined to be of the known object type with a particular custom object that is predetermined, within the selected customization scheme, to correspond to the known object type.

4. The method of claim 1, wherein:

the original immersive scene includes a first real-world scene that is distinct from a second real-world scene surrounding the user throughout the scene customization session;

the receiving of the data representative of the original immersive scene includes receiving data transmitted by a scene modeling system, the scene modeling system having elements that are disposed in a vicinity of the first real-world scene to capture data representative of the first real-world scene; and

the developing of the dynamic volumetric model of the original immersive scene includes

receiving, as part of the received data transmitted by the scene modeling system, data representative of a dynamic volumetric model of the first real-world scene, the data representative of the dynamic volumetric model of the first real-world scene created by the scene modeling system based on the captured data representative of the first real-world scene, and

continually updating the received dynamic volumetric model of the first real-world scene based on the received data transmitted by the scene modeling system as the data transmitted by the scene modeling system continues to be transmitted and received.

5. The method of claim 1, wherein:

the original immersive scene includes a real-world scene surrounding the user throughout the scene customization session;

the receiving of the data representative of the original immersive scene includes directly capturing, throughout the scene customization session, data representative of the real-world scene surrounding the user; and

the developing of the dynamic volumetric model of the original immersive scene includes

creating a dynamic volumetric model of the real-world scene surrounding the user based on the directly captured data representative of the real-world scene surrounding the user, and

continually updating the created dynamic volumetric model of the real-world scene surrounding the user based on the directly captured data representative of the real-world scene surrounding the user as the data representative of the real-world scene surrounding the user continues to be directly captured throughout the scene customization session.

6. The method of claim 5, wherein the data representative of the real-world scene surrounding the user, and upon which the creating and the continually updating of the dynamic volumetric model is based, is captured by way of one or more sensors of the artificial reality device used by the user to experience the scene customization session.

7. The method of claim 1, wherein the dynamically generating of the custom immersive scene is further performed by:

identifying, within the dynamic volumetric model of the original immersive scene, one or more non-augmentable objects included in the original immersive scene, the one or more non-augmentable objects corresponding with predefined entries on a non-augmentable object list associated with the selected customization scheme; and

abstaining from replacing, within the dynamic volumetric model and in accordance with the non-augmentable object list associated with the selected customization scheme, the one or more non-augmentable objects with the one or more custom objects.

8. The method of claim 1, further comprising determining, by the scene customization system, that the original immersive scene includes a known real-world scene documented in a scene library associated with the selected customization scheme;

wherein the dynamically generating of the custom immersive scene based on the application of the selected customization scheme to the original immersive scene is customized according to at least one preconfigured customization documented within the scene library in association with the known real-world scene.

9. The method of claim 1, further comprising:

determining, by the scene customization system while the presenting of the custom immersive scene to the user is ongoing, that another user using another artificial reality device is experiencing the original immersive scene; and

providing, by the scene customization system in response to the determining that the other user is experiencing the original immersive scene, data representative of the custom immersive scene being presented to the user by way of the artificial reality device, the data representative of the custom immersive scene configured to facilitate another dynamic generation of the custom immersive scene and another presentation of the custom immersive scene to the other user by way of the other artificial reality device.

10. The method of claim 1, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

11. A method comprising:

detecting, by a scene customization system, a selection, by a user, of a customization scheme from a library of customization schemes provided by an augmented reality provider associated with the scene customization system, the customization scheme selected to be applied to a real-world scene surrounding the user throughout a scene customization session to be experienced by the user by way of an immersive headset device;

dynamically generating, by the scene customization system and based on an application of the selected customization scheme to the real-world scene surrounding the user, a custom immersive scene by

directly capturing, throughout the scene customization session, data representative of the real-world scene surrounding the user,

creating, based on the directly captured data representative of the real-world scene surrounding the user, a dynamic volumetric model of the real-world scene surrounding the user,

continually updating, based on the directly captured data representative of the real-world scene surrounding the user as the data representative of the real-world scene surrounding the user continues to be directly captured throughout the scene customization session, the created dynamic volumetric model of the real-world scene surrounding the user,

identifying, within the dynamic volumetric model of the real-world scene surrounding the user, one or more objects in the real-world scene surrounding the user, and

replacing, within the dynamic volumetric model and in accordance with the applied customization scheme, the one or more identified objects with one or more custom objects; and

presenting, by the scene customization system throughout the scene customization session, the custom immersive scene to the user by way of the immersive headset device.

12. The method of claim 11, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

13. A system comprising:

at least one physical computing device that

detects a selection, by a user, of a customization scheme to be applied to an original immersive scene throughout a scene customization session to be experienced by the user by way of an artificial reality device;

dynamically generates, based on an application of the selected customization scheme to the original immersive scene, a custom immersive scene by

receiving data representative of the original immersive scene,

developing, based on the received data representative of the original immersive scene, a dynamic volumetric model of the original immersive scene,

identifying, within the dynamic volumetric model of the original immersive scene, one or more objects included in the original immersive scene, and

replacing, within the dynamic volumetric model and in accordance with the applied customization scheme, the one or more identified objects with one or more custom objects; and

presents, throughout the scene customization session, the custom immersive scene to the user by way of the artificial reality device

14. The system of claim 13, wherein:

the at least one physical computing device performs the identifying by;

identifying, based on the received data representative of the original immersive scene, one or more features of a particular object included in the one or more identified objects,

matching the one or more features of the particular object included in the one or more identified objects with one or more corresponding features of a known object documented in an object library associated with the selected customization scheme, and

determining, based on the matching of the one or more features, that the particular object is an instance of the known object documented in the object library; and

the at least one physical computing device performs the replacing of the one or more identified objects by replacing the particular object determined to be the instance of the known object with a particular custom object that is predetermined, within the selected customization scheme, to correspond to the known object.

15. The system of claim 13, wherein:

the at least one physical computing device performs the identifying by;

identifying, based on the received data representative of the original immersive scene, one or more features of a particular object included in the one or more identified objects,

determining that the one or more features of the particular object included in the one or more identified objects are characteristic of a known object type documented in an object library associated with the selected customization scheme, and

determining, based on the determining that the one or more features are characteristic of the known object type, that the particular object is of the known object type documented in the object library; and

the at least one physical computing device performs the replacing of the one or more identified objects by replacing the particular object determined to be of the known object type with a particular custom object that is predetermined, within the selected customization scheme, to correspond to the known object type.

16. The system of claim 13, wherein:

the original immersive scene includes a first real-world scene that is distinct from a second real-world scene surrounding the user throughout the scene customization session;

the at least one physical computing device performs the receiving of the data representative of the original immersive scene by receiving data transmitted by a scene modeling system, the scene modeling system having elements that are disposed in a vicinity of the first real-world scene to capture data representative of the first real-world scene; and

the at least one physical computing device performs the developing of the dynamic volumetric model of the original immersive scene by;

receiving, as part of the received data transmitted by the scene modeling system, data representative of a dynamic volumetric model of the first real-world scene, the data representative of the dynamic volumetric model of the first real-world scene created by the scene modeling system based on the captured data representative of the first real-world scene, and

continually updating the received dynamic volumetric model of the first real-world scene based on the received data transmitted by the scene modeling system as the data transmitted by the scene modeling system continues to be transmitted and received.

17. The system of claim 13, wherein:

the original immersive scene includes a real-world scene surrounding the user throughout the scene customization session;

the at least one physical computing device performs the receiving of the data representative of the original immersive scene by directly capturing, throughout the scene customization session and by way of one or more sensors of the artificial reality device, data representative of the real-world scene surrounding the user; and

the at least one physical computing device performs the developing of the dynamic volumetric model of the original immersive scene by

creating a dynamic volumetric model of the real-world scene surrounding the user based on the directly captured data representative of the real-world scene surrounding the user, and

continually updating the created dynamic volumetric model of the real-world scene surrounding the user based on the directly captured data representative of the real-world scene surrounding the user as the data representative of the real-world scene surrounding the user continues to be directly captured throughout the scene customization session.

18. The system of claim 13, wherein the at least one physical computing device further dynamically generates the custom immersive scene by:

identifying, within the dynamic volumetric model of the original immersive scene, one or more non-augmentable objects included in the original immersive scene, the one or more non-augmentable objects corresponding with predefined entries on a non-augmentable object list associated with the selected customization scheme; and

abstaining from replacing, within the dynamic volumetric model and in accordance with the non-augmentable object list associated with the selected customization scheme, the one or more non-augmentable objects with the one or more custom objects.

19. The system of claim 13, wherein:

the at least one physical computing device determines that the original immersive scene includes a known real-world scene documented in a scene library associated with the selected customization scheme; and

the dynamic generation of the custom immersive scene based on the application of the selected customization scheme to the original immersive scene is customized according to at least one preconfigured customization documented within the scene library in association with the known real-world scene.

20. The system of claim 13, wherein the at least one physical computing device further:

determines, while the presenting of the custom immersive scene to the user is ongoing, that another user using another artificial reality device is experiencing the original immersive scene; and

provides, in response to the determination that the other user is experiencing the original immersive scene, data representative of the custom immersive scene being presented to the user by way of the artificial reality device, the data representative of the custom immersive scene configured to facilitate another dynamic generation of the custom immersive scene and another presentation of the custom immersive scene to the other user by way of the other artificial reality device.