Real-time 3D virtual or physical model generating apparatus for HoloPortal and HoloCloud system

• US 10,516,869 B2
• Filed: 07/09/2016
• Issued: 12/24/2019
• Est. Priority Date: 07/09/2015
• Status: Active Grant

• Current Assignee: DoubleMe, Inc.
• Original Assignee: DoubleMe, Inc.
• Inventors: Kim, Heekwan
• Primary Examiner(s): Ustaris, Joseph G
• Assistant Examiner(s): Tchoussou, Boubacar Abdou
• Application Number:
  US15/206,259
• Publication Number:
  US 20170010584A1
• Time in Patent Office: 1,263 Days
• CPC Class Codes: G03H 1/0443 Digital holography, i.e. recording holograms with digital recording means holobject computation G03H1/0866
• CPC Class Codes: G03H 1/0476 Holographic printer G03H1/268 takes precedence
• CPC Class Codes: G03H 2001/0452 arranged to record an image of the object holographic microscope G03H2001/005
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G16B 25/00 ICT specially adapted for hybridisation; ICT specially adapted for gene or protein expression
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762

Abstract

A novel electronic system provides fast three-dimensional model generation, social content sharing of dynamic three-dimensional models, and monetization of the dynamic three-dimensional models created by casual consumers. In one embodiment, a casual consumer utilizes a dedicated real-time 3D model reconstruction studio with multiple camera angles, and then rapidly create dynamic 3D models with novel computational methods performed in scalable graphics processing units. In another embodiment, uncalibrated multiple sources of video recording of a targeted object are provided by a plurality of commonly-available consumer video recording devices (e.g. a smart phone, a camcorder, a digital camera, etc.) located at different angles, after which the uncalibrated multiple sources of video recording are transmitted to a novel cloud computing system for real-time temporal, spatial, and photometrical calibration and 3D model reconstruction. The dynamic 3D models can be uploaded, listed, and shared among content creators and viewers in an electronic sharing platform.

Claims

1. A three-dimensional (3D) body double-generating electronic system that provides two separate 3D model generation pathways as a combined turnkey solution to a casual consumer who is not a graphics expert, the system comprising:

a three-dimensional (3D) model creation studio that incorporates cameras positioned on a multiple number of angles around a stage, where a first target object brought into the 3D model creation studio by the casual consumer is placed for video footage recording at the multiple number of angles around the stage, and wherein the 3D model creation studio further includes a near real-time 3D reconstruction electronic system to create a first 3D body double model that resembles the first target object brought in by the casual consumer;

a cloud-based 3D model creation platform that utilizes portable video recording devices positioned at multiple-angle views around a second target object, and a scalable number of graphics processing units (GPU'"'"'s) in a cloud computing resource for creation of a second 3D body double model that resembles the second target object;

a 3D model and content database that stores the first 3D body double model from the 3D model creation studio and the second 3D body double model from the cloud-based 3D model creation platform, wherein the two separate 3D model generation pathways provided by the 3D model creation studio and the cloud-based 3D model creation platform are the combined turnkey solution to the casual consumer who does not have operational knowledge or ownership of specialized 3D equipment for creating 3D body double models;

a 3D printer device connected to the 3D model creation studio and the cloud-based 3D model creation platform, wherein the 3D printer device produces a physical replica of the first 3D body double model from the 3D model creation studio or the second 3D body double model from the cloud-based 3D model creation platform, when a user requests production of the physical replica;

an electronic 3D content sharing software executed on a computer server connected to the 3D model and content database, wherein the electronic 3D content sharing software configures the computer server to upload, list, transmit, and share 3D model animations and 3D contents derived from the two separate 3D model generation pathways, which are provided by the 3D model creation studio and the cloud-based 3D model creation platform as the combined turnkey solution to the casual consumer;

a client-side 3D content viewer and management user interface executed on an electronic device, wherein the client-side 3D content viewer and management user interface enables the user to purchase, sell, transmit, receive, or playback a 3D content incorporating the first 3D body double model or the second 3D body double model via the electronic 3D content sharing software and the 3D model and content database.

2. The three-dimensional (3D) body double-generating electronic system of claim 1, wherein the client-side 3D content viewer and management user interface is a mobile application for the electronic device.

3. The three-dimensional (3D) body double-generating electronic system of claim 1, wherein the client-side 3D content viewer and management user interface is a computer application for the electronic device.

4. The three-dimensional (3D) body double-generating electronic system of claim 1, wherein the client-side 3D content viewer and management user interface is a web interface accessible via a web browser on a consumer'"'"'s electronic device.

5. The three-dimensional (3D) body double-generating electronic system of claim 1, wherein the 3D model animations and 3D contents derived from the first 3D body double model or the second 3D body double model are shared, bought, and sold from one entity to another entity over the electronic 3D content sharing software executed on the computer server connected to the 3D model and content database.