Methods and systems for creating and manipulating an individually-manipulable volumetric model of an object

• US 10,204,444 B2
• Filed: 04/28/2016
• Issued: 02/12/2019
• Est. Priority Date: 04/28/2016
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Inventors: Khalid, Mohammad Raheel, Jaafar, Ali, Breitenfeld, Denny, Hansen, Xavier, Egeler, Christian, Kamal, Syed, Chandrasiri, Lama Hewage Ravi Prathapa, Smith, Steven L.
• Primary Examiner(s): Nguyen, Phong X
• Application Number:
  US15/141,717
• Publication Number:
  US 20170316606A1
• Time in Patent Office: 1,020 Days
• Field of Search: 345420, 345630-633, 345653
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

An exemplary virtual reality media provider system (“system”) includes a configuration of synchronous video and depth capture devices disposed at fixed positions in a vicinity of a first object located in a natural setting along with one or more additional objects. The video and depth capture devices capture two-dimensional video data and depth data for a surface the first object. The system distinguishes the first object from a second object included in the one or more additional objects located in the natural setting and generates an individually-manipulable volumetric model of the first object. The individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device.

Claims

1. A method comprising:

capturing, by a virtual reality media provider system including a configuration of synchronous video and depth capture devices disposed at fixed positions in a vicinity of a first object, two-dimensional video data and depth data for a surface of the first object, the first object located in a natural setting along with one or more additional objects;

distinguishing, by the virtual reality media provider system, the first object from a second object included in the one or more additional objects located in the natural setting along with the first object, the distinguishing including performing a plurality of techniques in combination with one another, the plurality of techniques including;

determining, based on the captured two-dimensional video data, that the first object is moving laterally in relation to the second object, and

determining, based on the captured depth data, that the first object is located at least a predetermined distance away from the second object; and

generating, by the virtual reality media provider system based on the captured depth data and the captured two-dimensional video data, an individually-manipulable volumetric model of the first object, the individually-manipulable volumetric model of the first object configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device, the immersive virtual reality world based on virtual reality media content provided to the media player device and representative of the immersive virtual reality world.

2. The method of claim 1, wherein the capturing of the two-dimensional video data and depth data for the surface of the first object, the distinguishing of the first object from the second object, and the generating of the individually-manipulable volumetric model are each performed by the virtual reality media provider system in real time while the first object is located in the natural setting.

3. The method of claim 1, wherein:

the natural setting is a setting of a real-world event;

the fixed positions in the vicinity of the first object where the synchronous video and depth capture devices are disposed include a plurality of fixed positions surrounding the first object within the setting of the real-world event.

4. The method of claim 1, wherein the synchronous video and depth capture devices are unitary devices configured to capture both the two-dimensional video data and the depth data.

5. The method of claim 1, wherein the capturing of the depth data for the surface of the first object located in the natural setting is performed using at least one of a stereoscopic triangulation depth capture technique, a time-of-flight depth capture technique, and an infrared pattern analysis depth capture technique.

6. The method of claim 1, wherein the plurality of techniques performed in combination with one another in the distinguishing of the first object from the second object further includes at least one of:

identifying, based on the captured two-dimensional video data, the first object and the second object and determining that the first identified object is different from the second identified object; and

determining, based on the captured two-dimensional video data from at least one video and depth capture device, that a representation of the first object in the captured two-dimensional video data does not overlap in the captured two-dimensional video data with a representation of the second object from a vantage point of the at least one video and depth capture device.

7. The method of claim 1, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by inserting the individually-manipulable volumetric model of the first object into the immersive virtual reality world at a particular location within the immersive virtual reality world.

8. The method of claim 1, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by replacing the individually-manipulable volumetric model of the first object included within the immersive virtual reality world with an individually-manipulable volumetric model of a third object within the immersive virtual reality world.

9. The method of claim 8, wherein:

the first object is a first person and the third object is a second person; and

the individually-manipulable volumetric model of the second person performs a same action in the immersive virtual reality world that the individually-manipulable volumetric model of the first person would perform if not replaced.

10. The method of claim 1, wherein the individually-manipulable volumetric model of the first object is configured to be individually manipulated with respect to the immersive virtual reality world by removing the individually-manipulable volumetric model of the first object from the immersive virtual reality world.

11. The method of claim 1, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

12. A method comprising:

capturing, in real time by a virtual reality media provider system including a configuration of synchronous video and depth capture devices disposed at fixed positions in a vicinity of a first person, two-dimensional video data and depth data for a surface of the first person, the first person located in a natural setting along with one or more objects;

distinguishing, in real time by the virtual reality media provider system, the first person from the one or more objects located in the natural setting along with the first person, the distinguishing including performing a plurality of techniques in combination with one another, the plurality of techniques including;

determining, based on the captured two-dimensional video data, that the first person is moving laterally in relation to the one or more objects, and

determining, based on the captured depth data, that the first person is located at least a predetermined distance away from the one or more objects;

generating, in real time by the virtual reality media provider system based on the captured depth data and the captured two-dimensional video data, an individually-manipulable volumetric model of the first person;

providing, by the virtual reality media provider system to a media player device associated with a user, virtual reality media content representative of an immersive virtual reality world and including the individually-manipulable volumetric model of the first person performing an action within the immersive virtual reality world; and

replacing, in real time by the virtual reality media provider system within the virtual reality media content, the individually-manipulable volumetric model of the first person with an individually-manipulable volumetric model of a second person such that the individually-manipulable volumetric model of the second person performs the action within the immersive virtual reality world while the user is experiencing the immersive virtual reality world using the media player device.

13. The method of claim 12, embodied as computer-executable instructions on at least one non-transitory computer-readable medium.

14. A system comprising:

a configuration of synchronous video and depth capture devices disposed at fixed positions in a vicinity of a first object and that capture two-dimensional video data and depth data for a surface of the first object, the first object located in a natural setting along with one or more additional objects;

one or more processors communicatively coupled to the configuration of synchronous video and depth capture devices; and

one or more non-transitory memories storing instructions that, when executed by the one or more processors, cause the one or more processors to;

distinguish the first object from a second object included in the one or more additional objects located in the natural setting along with the first object by performing a plurality of techniques in combination with one another, the plurality of techniques including

determining, based on the captured two-dimensional video data, that the first object is moving laterally in relation to the second object, and

determining, based on the captured depth data, that the first object is located at least a predetermined distance away from the second object, and

generate, based on the captured depth data and the captured two-dimensional video data, an individually-manipulable volumetric model of the first object, the individually-manipulable volumetric model of the first object configured to be individually manipulated with respect to an immersive virtual reality world while a user of a media player device is experiencing the immersive virtual reality world using the media player device, the immersive virtual reality world based on virtual reality media content provided to the media player device and representative of the immersive virtual reality world.

15. The system of claim 14, wherein:

the configuration of synchronous video and depth capture devices capture the two-dimensional video data and depth data for the surface of the first object in real time while the first object is located in the natural setting; and

the instructions cause the one or more processors to distinguish the first object from the second object and generate the individually-manipulable volumetric model in real time while the first object is located in the natural setting.

16. The system of claim 14, wherein:

the natural setting is a setting of a real-world event;

the fixed positions in the vicinity of the first object where the synchronous video and depth capture devices are disposed include a plurality of fixed positions surrounding the first object within the setting of the real-world event.

17. The system of claim 14, wherein the plurality of techniques performed in combination with one another in the distinguishing of the first object from the second object further includes at least one of:

identifying, based on the captured two-dimensional video data, the first object and the second object and determining that the first identified object is different from the second identified object; and

determining, based on the captured two-dimensional video data from at least one video and depth capture device, that a representation of the first object in the captured two-dimensional video data does not overlap in the captured two-dimensional video data with a representation of the second object from a vantage point of the at least one video and depth capture device.

18. The system of claim 14, wherein the instructions cause the one or more processors to individually manipulate the individually-manipulable volumetric model of the first object with respect to the immersive virtual reality world by inserting the individually-manipulable volumetric model of the first object into the immersive virtual reality world at a particular location within the immersive virtual reality world.

19. The system of claim 14, wherein the instructions cause the one or more processors to individually manipulate the individually-manipulable volumetric model of the first object with respect to the immersive virtual reality world by replacing the individually-manipulable volumetric model of the first object included within the immersive virtual reality world with an individually-manipulable volumetric model of a third object within the immersive virtual reality world.

20. The system of claim 19, wherein:

the first object is a first person and the third object is a second person; and

the individually-manipulable volumetric model of the second person performs a same action in the immersive virtual reality world that the individually-manipulable volumetric model of the first person would perform if not replaced.