HoloPortal and HoloCloud system and method of operation

• US 10,516,868 B2
• Filed: 07/09/2016
• Issued: 12/24/2019
• Est. Priority Date: 07/09/2015
• Status: Active Grant

• Current Assignee: DoubleMe, Inc.
• Original Assignee: DoubleMe, Inc.
• Inventors: Kim, Heekwan
• Primary Examiner(s): Ustaris, Joseph G
• Assistant Examiner(s): Tchoussou, Boubacar Abdou
• Application Number:
  US15/206,255
• Publication Number:
  US 20170013246A1
• Time in Patent Office: 1,263 Days
• CPC Class Codes: G03H 1/0443 Digital holography, i.e. recording holograms with digital recording means holobject computation G03H1/0866
• CPC Class Codes: G03H 1/0476 Holographic printer G03H1/268 takes precedence
• CPC Class Codes: G03H 2001/0452 arranged to record an image of the object holographic microscope G03H2001/005
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G16B 25/00 ICT specially adapted for hybridisation; ICT specially adapted for gene or protein expression
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762

Abstract

A novel electronic system provides fast three-dimensional model generation, social content sharing of dynamic three-dimensional models, and monetization of the dynamic three-dimensional models created by casual consumers. In one embodiment, a casual consumer utilizes a dedicated real-time 3D model reconstruction studio with multiple camera angles, and then rapidly create dynamic 3D models with novel computational methods performed in scalable graphics processing units. In another embodiment, uncalibrated multiple sources of video recording of a targeted object are provided by a plurality of commonly-available consumer video recording devices (e.g. a smart phone, a camcorder, a digital camera, etc.) located at different angles, after which the uncalibrated multiple sources of video recording are transmitted to a novel cloud computing system for real-time temporal, spatial, and photometrical calibration and 3D model reconstruction. The dynamic 3D models can be uploaded, listed, and shared among content creators and viewers in an electronic sharing platform.

Claims

1. A three-dimensional (3D) body double-generating electronic system that provides two separate 3D model generation pathways as a combined turnkey solution to a casual consumer who is not a graphics expert, the system comprising:

a three-dimensional (3D) model creation studio that incorporates cameras positioned on a multiple number of angles around a stage, where a first target object brought into the 3D model creation studio by the casual consumer is placed for video footage recording at the multiple number of angles around the stage, and wherein the 3D model creation studio further includes a near real-time 3D reconstruction electronic system to create a first 3D body double model that resembles the first target object brought in by the casual consumer;

a cloud-based 3D model creation platform that utilizes portable video recording devices positioned at multiple-angle views around a second target object and a scalable number of graphics processing units (GPU'"'"'s) in a cloud computing resource for creation of a second 3D body double model that resembles the second target object;

a 3D model and content database that stores the first 3D body double model from the 3D model creation studio and the second 3D body double model from the cloud-based 3D model creation platform, wherein the two separate 3D model generation pathways provided by the 3D model creation studio and the cloud-based 3D model creation platform are the combined turnkey solution to the casual consumer who does not have operational knowledge or ownership of specialized 3D equipment for creating 3D body double models;

an electronic 3D content sharing software executed on a computer server connected to the 3D model and content database, wherein the electronic 3D content sharing software controls the computer server to upload, list, transmit, and share 3D model animations and 3D contents that are created from the 3D model creation studio or the cloud-based 3D model creation platform;

a client-side 3D content viewer and management user interface executed on a notebook computer, a desktop computer, a mobile communication device, or a web server, wherein the client-side 3D content viewer and management user interface enables a user to purchase, sell, transmit, receive, or playback a 3D content incorporating the first 3D body double model or the second 3D body double model via the electronic 3D content sharing software and the 3D model and content database;

a plurality of wireless transceivers integrated into the portable video recording devices, wherein the wireless transceivers transmit multiple-angle video data streams from the portable video recording devices to the scalable number of GPU'"'"'s in the cloud computing resource; and

the cloud computing resource comprising one or more cloud computing servers that include one or more GPU'"'"'s, wherein the one or more cloud computing servers are connected to each other to formulate a scalable cloud computing network, and wherein the scalable cloud computing network allocates and manages the scalable number of GPU'"'"'s to achieve a creation of the second 3D body double model based on a particular calculation load requirement for the graphical transformations associated with the second target object.

2. The three-dimensional body double-generating electronic system of claim 1, wherein the scalable cloud computing network allocates a larger number of GPU'"'"'s to process the graphical transformations associated with the second target object when the multiple-angle video data streams have a higher calculation load requirement due to at least one of a higher video feed count, a longer duration, or a higher resolution for the multiple-angle video data streams.

3. The three-dimensional body double-generating electronic system of claim 1, wherein the scalable cloud computing network allocates a smaller number of GPU'"'"'s to process the graphical transformations associated with the second target object when the multiple-angle video data streams has a lower calculation load requirement due to at least one of a lower video feed count, a shorter duration, or a lower resolution for the multiple-angle video data streams.