Method using 3D geometry data for virtual reality image presentation and control in 3D space

• US 9,286,718 B2
• Filed: 09/27/2013
• Issued: 03/15/2016
• Est. Priority Date: 09/27/2013
• Status: Expired due to Fees

• Current Assignee: Ortery Technologies, Inc.
• Original Assignee: Ortery Technologies, Inc.
• Inventors: Lai, Peng-Cheng, Wang, Douglas Wei-Ming
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US14/039,814
• Publication Number:
  US 20150091905A1
• Time in Patent Office: 900 Days
• Field of Search: 345/419, 345/581, 345/582, 345/473, 345/474, 345/475, 345/154, 382/154
• US Class Current: 1/1
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

A method uses a collection of photographic images of an object taken from different viewing angles, along with a set of key geographic parameters for each frame of the photo images, to associate with the object'"'"'s 3D modeling data, which can use for presenting high quality, photo-realistic 3D image in real time at a computing device. It also provides 3D geometry data for physical applications, which uses an automatic or a manual photo-taking system, and the imported 3D modeling data of the same object with the same hardware system or from an independent 3D geometry scanning system, composes these information into a complete package of files by a manual, semi-automatic or automatic software tool, and then to be presented with a viewing program with 3D environment. It can also be extended to the stereoscopic system and offers real time physical manipulation capabilities, the high quality, realistic visual effects.

Claims

1. A method of matching a 2D photo image file with a 3D mesh by using matrix transformation with six degrees of freedom for a solid object on the 2D view window, wherein 3D geometry parameters of the 3D mesh are complied with 2D photo image parameters of the 2D image file for high image quality, photo-realistic virtual reality presentation and physical augmented reality applications, and the matrix transformation with six degrees of freedom for the solid object is done manually by at least one of the following steps:

manually moving the tip of the axis for controlling the values of θ

;

manually moving the tip of the axis for controlling the values of φ

;

manually rotating a body axis of the solid object for controlling the value of ω

; and

scaling a size of the 3D mesh for matching with the 2D photo image parameters being selected from the 2D photo image file until the whole set of the 2D photo image file has been used up, wherein no less than three 2D photo images in the 2D photo image file are selected manually from the body axis in horizontal and vertical directions with calculation assisted.

2. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 1, wherein 2D photo images in the 2D photo image file is optionally processed at least one of the following:

removing image backgrounds thereof;

compressed to JPEG format with hierarchical pixel resolution, transparency information; and

saved in the 2D photo image file.

3. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 1, wherein the 3D geometry parameters can be generated from a certain wavelength of visible optics camera, laser beam or invisible infrared and reflection capturing system, by getting the silhouette of 2D photo images in the 2D photo image file, or the depth data of each of object geometry of the solid object.

4. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 1, wherein the high image quality, photo-realistic virtual reality presentation is a file system at the internet server constructed to provide an end user a viewing mechanism to see high resolution photo images and the 3D mesh, and the physical augmented reality applications are for 3D measurement or 3D control functions.

5. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 4, wherein the high resolution photo images and the 3D mesh are viewed together in a different transparency way.

6. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 4, wherein the high image quality, photo-realistic virtual reality presentation further extends to a stereoscopic system with viewing window of left stereogram and right stereogram for left eye and right eye, respectively.

7. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 4, wherein the high image quality, photo-realistic virtual reality presentation further extends to show a smoothness of 2D photo images in the 2D photo image file in a 3D space through angular morphing of the 2D photo images in at least one of a θ

direction and a φ

direction.

8. A method of matching a 2D photo image file with a 3D mesh by using matrix transformation with six degrees of freedom for a solid object on the 2D view window, wherein 3D geometry parameters of the 3D mesh are complied with 2D photo image parameters of the 2D image file for high image quality, photo-realistic virtual reality presentation and physical augmented reality applications, and the matrix transformation with six degrees of freedom for the solid object is done automatically by programmatically automatic matching of the 2D photo image parameters with the 3D geometry parameters while a 3D geometry scan mechanism provides parameter relations between 2D photo images in the 2D photo image file and the 3D mesh.

9. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 8, wherein 2D photo images in the 2D photo image file is optionally processed at least one of the following:

removing image backgrounds thereof;

compressed to JPEG format with hierarchical pixel resolution, transparency information; and

saved in the 2D photo image file.

10. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 8, wherein the 3D geometry parameters can be generated from a certain wavelength of visible optics camera, laser beam or invisible infrared and reflection capturing system, by getting the silhouette of 2D photo images in the 2D photo image file, or the depth data of each of object geometry of the solid object.

11. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 8, wherein the high image quality, photo-realistic virtual reality presentation is a file system at the internet server constructed to provide an end user a viewing mechanism to see high resolution photo images and the 3D mesh, and the physical augmented reality applications are for 3D measurement or 3D control functions.

12. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 11, wherein the high resolution photo images and the 3D mesh are viewed together in a different transparency way.

13. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 11, wherein the high image quality, photo-realistic virtual reality presentation further extends to a stereoscopic system with viewing window of left stereogram and right stereogram for left eye and right eye, respectively.

14. The method of matching a 2D photo image file with a 3D mesh as claimed in claim 11, wherein the high image quality, photo-realistic virtual reality presentation further extends to show a smoothness of 2D photo images in the 2D photo image file in a 3D space through angular morphing of the 2D photo images in at least one of a θ

direction and a φ

direction.