3D VISUALIZATION SYSTEM

• US 20110134220A1
• Filed: 12/07/2010
• Published: 06/09/2011
• Est. Priority Date: 12/07/2009
• Status: Active Grant

• Current Assignee: Photon-X, Inc.
• Original Assignee: Photon-X, Inc.
• Inventors: Barbour, Blair Allen, Ackerson, David Scott
• Granted Patent: US 8,736,670 B2
• Time in Patent Office: Days
• US Class Current: 348/44
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: H04N 13/207 using a single 2D image sensor

Abstract

In accordance with one aspect, the present invention provides a real-time 3D visualization system. The system includes means for conveying electromagnetic energy emanating from one or more 3D surfaces including a scene and means for sensing spatial phase characteristics of the electromagnetic energy as the configuration of the 3D surfaces relative to the system changes. The system includes means for creating a 3D scene model utilizing the spatial phase characteristics sensed in a plurality of configurations and means for displaying the 3D scene model in real-time. The means for displaying includes means for synthesizing depth cues.

Claims

1. A real-time 3D visualization system, including:

a. means for conveying electromagnetic energy emanating from one or more 3D surfaces included by a scene,

b. means for sensing spatial phase characteristics of the electromagnetic energy as the configuration of the 3D surfaces relative to the system changes,

c. means for creating a 3D scene model utilizing the spatial phase characteristics sensed in a plurality of configurations, and

d. means for displaying the 3D scene model in real-time, wherein said means for displaying includes means for synthesizing depth cues.

2. The system of claim 1 wherein the means for creating the 3D scene model further includes means to represent 3D surfaces with one or more of the following morphologies:

rigid, deformable and particle.

3. The system of claim 1 wherein the means for creating a 3D scene model further includes means for representing the reflectance field of the 3D surfaces, with one or more of the following lambertian, specular, and known bidirectional reflectance distribution function

4. The system of claim 1 wherein the 3D scene model is solid, spatially sorted and hierarchical.

5. The system of claim 1 wherein the means for conveying electromagnetic energy includes one or more of a refractive element, a reflective element, a diffractive element, a lens, a mirror, a fiber optic element, a microlens array, a baffle, a micromirror array, a catoptric element, a holographic optical element, a diffractive optical element, a beam steering mechanism, an element including metamaterials or an element including birefringents.

6. The system of claim 1 wherein the means for sensing spatial phase characteristics includes one or more of a focal plane array, a direct image sensor, a multi-sampling sensor, a photon counter, a frequency selective surface, a plasmonic crystal, a micropolarizer array, quantum dots, a form birefringent, an antenna-coupled metal-oxide-metal diode, a liquid crystal, a nematic liquid crystal, a ferroelectric liquid crystal, a linear polarizer, a wave plate, a beam splitter or a light emitting diode.

7. The system of claim 1 wherein the means for sensing spatial phase characteristics includes one or more on-chip means to increase dynamic range, including one or more of an active pixel sensor, a delta-sigma converter and a subtraction technique under which sets of adjacent pixels are subtracted to form auxiliary on-chip differences that can be used to compute the orientations of surface elements.

8. The system of claim 1 which further includes means for sensing one or more spectral characteristics of the electromagnetic energy in one or more spectral bands, the means for creating a 3D scene model utilizing the spectral characteristics.

9. The system of claim 1 which further includes means for sensing one or more range characteristics to the surfaces, the means for creating a 3D scene model utilizing the range characteristics.

10. The system of claim 1 which further includes means for sensing one or more location characteristics of the system or the surfaces, the means for creating a 3D scene model utilizing the location characteristics.

11. The system of claim 1 wherein the means for creating a 3D scene model further includes means for utilizing the spatial phase characteristics as scattering modality changes between at least two scattering modalities, including a plurality of specular reflection, diffuse reflection, micro facet reflection, retro-reflection, transmission and emission.

12. The system of claim 1 which further includes means for determining from the spatial phase characteristics one or more of a tag location or information encoded into the tag including a 3D image, the means for creating a 3D scene model utilizing the location or the information encoded into the tag.

13. The system of claim 1 wherein means for processing the 3D scene model and means for displaying the 3D scene model includes means for image compression and decompression.

14. The system of claim 1 wherein said means for displaying the 3D scene model includes a means for tracking a display observer'"'"'s head in order to synthesize a motion parallax depth cue.

15. The system of claim 1 which further includes means for providing functionality including one of automatic audio, manual audio, autofocus, manual focus, automatic exposure, manual exposure, automatic white balance, manual white balance, headphone jack, external microphone, filter rings, lens adapters, digital zoom, optical zoom, playback and record controls, rechargeable batteries, synchronization with other the apparatus and image stabilization.

16. The system of claim 1 which further includes means for extracting information about of the 3D surfaces using the 3D scene model, thereby creating real-time information about the 3D surfaces.

17. The system of claim 1 which further includes means for refining the 3D scene model by communicating with an operator or an automated control system to locate the system at one or more desired locations.

18. The system of claim 1 wherein the information extracted by the means for extracting real-time information is one or more of a 3D video, a compressed 3D video, a noise suppressed 3D video, a route, a descriptions, an anomaly, a change, a feature, a shape, sizes, poses, dimensions, motions, speeds, velocities, accelerations, expressions, gestures, emotions, deception, postures, activities, behaviors, faces, lips, ears, eyes, irises, veins, moles, wounds, birthmarks, freckles, scars, wrinkles, fingerprints, thumbprints, palm prints, warts, categories, identities, instances, surfaces of internal organs, breasts, skin tumors, skin cancers, dysmorphologies, abnormalities, teeth, gums, facial expressions, facial macro expressions, facial micro expressions, facial subtle expressions, head gestures, hand gestures, arm gestures, gaits, body gestures, wagging tails, athletic motions, fighting positions, lip reading, crawling, talking, screaming, barking, breathing, running, galloping, eating, gun raising, axe swinging, phone talking, guitar playing, crowd behavior, health, mental state, range of motion, performance, weight, volume and concealed objects.