3D virtual reality model sharing and monetization electronic platform system

• US 10,116,913 B2
• Filed: 07/09/2016
• Issued: 10/30/2018
• Est. Priority Date: 07/09/2015
• Status: Active Grant

• Current Assignee: DoubleMe, Inc.
• Original Assignee: DoubleMe, Inc.
• Inventors: Kim, Heekwan
• Primary Examiner(s): Atala, Jamie J
• Assistant Examiner(s): Boylan, James T
• Application Number:
  US15/206,257
• Publication Number:
  US 20170013247A1
• Time in Patent Office: 843 Days
• Field of Search: 348 47
• CPC Class Codes: G03H 1/0443 Digital holography, i.e. recording holograms with digital recording means holobject computation G03H1/0866
• CPC Class Codes: G03H 1/0476 Holographic printer G03H1/268 takes precedence
• CPC Class Codes: G03H 2001/0452 arranged to record an image of the object holographic microscope G03H2001/005
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G16B 25/00 ICT specially adapted for hybridisation; ICT specially adapted for gene or protein expression
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762

Abstract

A novel electronic system provides fast three-dimensional model generation, social content sharing of dynamic three-dimensional models, and monetization of the dynamic three-dimensional models created by casual consumers. In one embodiment, a casual consumer utilizes a dedicated real-time 3D model reconstruction studio with multiple camera angles, and then rapidly create dynamic 3D models with novel computational methods performed in scalable graphics processing units. In another embodiment, uncalibrated multiple sources of video recording of a targeted object are provided by a plurality of commonly-available consumer video recording devices (e.g. a smart phone, a camcorder, a digital camera, etc.) located at different angles, after which the uncalibrated multiple sources of video recording are transmitted to a novel cloud computing system for real-time temporal, spatial, and photometrical calibration and 3D model reconstruction. The dynamic 3D models can be uploaded, listed, and shared among content creators and viewers in an electronic sharing platform.

Claims

1. A three-dimensional body double-generating, social sharing, and monetization electronic system comprising:

a HoloPortal electronic system that incorporates a dedicated physical studio space including a center stage, a plurality of stationary cameras surrounding the center stage, and a 3D reconstruction electronic system, which is configured to capture, calculate, reconstruct, and generate graphical transformation of a target object to create a 3D body double model from pre-calibrated image sources from the plurality of stationary cameras;

a HoloCloud electronic system comprising uncalibrated portable video recording devices positioned at multiple-angle views around the target object to generate uncalibrated raw multiple-angle video data streams, a cloud computing resource containing a scalable number of graphics processing units (GPU'"'"'s) that receive the uncalibrated raw multiple-angle video data streams from the uncalibrated portable video recording devices, a pre-processing module in the cloud computing resource that calibrates temporal, spatial, and photometrical variables deduced from the uncalibrated raw multiple angle video data streams as a post-capture process, which in turn generates background 3D geometry and 360-degree virtual reality videos, and a 3D reconstruction module in the cloud computing resource for providing depth map computations, voxel grid reconstructions, and deformed mesh generations for creation of another 3D body double model that resembles the target object;

a 3D model and content database configured to store the 3D body double model created from the HoloPortal electronic system or the HoloCloud electronic system;

an electronic 3D content sharing software executed on a computer server connected to the 3D model and content database, wherein the electronic 3D content sharing software configures the computer server to upload, list, transmit, and share 3D model animations and 3D contents that are created from the HoloPortal electronic system and the HoloCloud electronic system; and

a client-side 3D content viewer and management user interface executed on a notebook computer, a desktop computer, a mobile communication device, or a web server, wherein the client-side 3D content viewer and management user interface is configured to purchase, sell, transmit, receive, or playback a 3D content incorporating the 3D body double model via the electronic 3D content sharing software and the 3D model and content database.

2. The three-dimensional body double-generating, social sharing, and monetization electronic system of claim 1, further comprising a wireless transceiver in the HoloCloud electronic system that transmits the uncalibrated raw multiple-angle video data streams from the general-purpose and uncalibrated portable video recording devices to the scalable number of GPU'"'"'s in the cloud computing resource.

3. The three-dimensional body double-generating, social sharing, and monetization electronic system of claim 1, wherein the client-side 3D content viewer and management user interface is a mobile application for the mobile communication device.

4. The three-dimensional body double-generating, social sharing, and monetization electronic system of claim 1, wherein the client-side 3D content viewer and management user interface is a computer application for the notebook computer or the desktop computer.

5. The three-dimensional body double-generating, social sharing, and monetization electronic system of claim 1, wherein the client-side 3D content viewer and management user interface is a web interface accessible via a web browser on a consumer'"'"'s electronic device, which is connected to the web server.

6. The three-dimensional body double-generating, social sharing, and monetization electronic system of claim 1, wherein the 3D model animations and 3D contents that are created from the HoloPortal electronic system and the HoloCloud electronic system are shared, bought, and sold from one entity to another entity over the electronic 3D content sharing software executed on the computer server connected to the 3D model and content database.