3D visualization system

• US 8,736,670 B2
• Filed: 12/07/2010
• Issued: 05/27/2014
• Est. Priority Date: 12/07/2009
• Status: Active Grant

• Current Assignee: Photon-X, Inc.
• Original Assignee: Photon-X, Inc.
• Inventors: Barbour, Blair Allen, Ackerson, David Scott
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Tarko, Asmamaw G
• Application Number:
  US12/962,345
• Publication Number:
  US 20110134220A1
• Time in Patent Office: 1,267 Days
• Field of Search: 348/44
• US Class Current: 348/44
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: H04N 13/207 using a single 2D image sensor

Abstract

In accordance with one aspect, the present invention provides a real-time 3D visualization system. The system includes means for conveying electromagnetic energy emanating from one or more 3D surfaces including a scene and means for sensing spatial phase characteristics of the electromagnetic energy as the configuration of the 3D surfaces relative to the system changes. The system includes means for creating a 3D scene model utilizing the spatial phase characteristics sensed in a plurality of configurations and means for displaying the 3D scene model in real-time. The means for displaying includes means for synthesizing depth cues.

Claims

1. A real-time 3D visualization system included in a real 3D scene, the scene includes one or more moving surfaces and electromagnetic energy, the energy scatters from the surfaces, the system includes:

a. means for conveying energy, the energy is received from the surfaces,

b. means for sensing at video sampling rates a plurality of polarization characteristics of the energy, thereby creating sensed data, the energy is received from the means for conveying,

c. means for modeling one or more surface elements, thereby creating a scene model, the means for modeling uses sensed data, the surface elements represent one or more regions on the surfaces, the means for modeling further includes means for modeling changing polarization of the energy as it interacts with the surface elements in a plurality of scattering modes from a plurality of directions at a plurality of times, and

d. means for displaying the scene in real-time, thereby evoking a 3D notion of the scene, the means for displaying uses the scene model, the means for displaying further includes means for displaying a plurality of depth cues.

2. The system of claim 1 wherein the means for modeling further includes means to represent the surface elements in one or more of the following morphologies:

rigid, deformable and particle.

3. The system of claim 1 wherein the scene model is solid, spatially sorted and hierarchical.

4. The system of claim 1 wherein the means for conveying includes one or more of a refractive element, a reflective element, a diffractive element, a lens, a mirror, a fiber optic element, a microlens array, a baffle, a micromirror array, a catoptric element, a holographic optical element, a diffractive optical element, a beam steering mechanism, an element including metamaterials or an element including birefringents.

5. The system of claim 1 wherein the means for sensing includes one or more of a focal plane array, a direct image sensor, a multi-sampling sensor, a photon counter, a frequency selective surface, a plasmonic crystal, a micropolarizer array, quantum dots, a form birefringent, an antenna-coupled metal-oxide-metal diode, a liquid crystal, a nematic liquid crystal, a ferroelectric liquid crystal, a linear polarizer, a wave plate, a beam splitter or a light emitting diode.

6. The system of claim 1 wherein the means for sensing includes one or more on-chip means to increase dynamic range, including one or more of an active pixel sensor, a delta-sigma converter and a subtraction technique under which sets of adjacent pixels are subtracted to form auxiliary on-chip differences that can be used to compute the orientations of surface elements.

7. The system of claim 1 which further includes means for sensing one or more spectral characteristics of the electromagnetic energy in one or more spectral bands, the means for modeling utilizing the spectral characteristics.

8. The system of claim 1 which further includes means for sensing one or more range characteristics to the surfaces, the means for modeling utilizing the range characteristics.

9. The system of claim 1 which further includes means for sensing one or more location characteristics of the system or the surfaces, the means for modeling utilizing the location characteristics.

10. The system of claim 1 wherein the plurality of scattering modes includes at least two of specular reflection, diffuse reflection, micro facet reflection, retro-reflection, transmission and emission.

11. The system of claim 1 which further includes means for determining from a plurality of polarization characteristics one or more of a tag location or information encoded into the tag including a 3D image, the means for modeling utilizing the location or the information encoded into the tag.

12. The system of claim 1 wherein means for processing the scene model and means for displaying the scene model includes means for image compression and decompression.

13. The system of claim 1 wherein said means for displaying further includes a means for tracking a display observer'"'"'s head, the tracking is used to display a motion parallax depth cues.

14. The system of claim 1 which further includes means for providing functionality including one of automatic audio, manual audio, autofocus, manual focus, automatic exposure, manual exposure, automatic white balance, manual white balance, headphone jack, external microphone, filter rings, lens adapters, digital zoom, optical zoom, playback and record controls, rechargeable batteries, synchronization with other the apparatus and image stabilization.

15. The system of claim 1 which further includes means for extracting information about of the surface elements using the scene model, thereby creating real-time information about the surface elements.

16. The system of claim 1 which further includes means for refining the scene model by communicating with an operator or an automated control system to locate the system at one or more desired locations.

17. The system of claim 1 wherein the information extracted by the means for extracting real-time information is one or more of a 3D video, a compressed 3D video, a noise suppressed 3D video, a route, a descriptions, an anomaly, a change, a feature, a shape, sizes, poses, dimensions, motions, speeds, velocities, accelerations, expressions, gestures, emotions, deception, postures, activities, behaviors, faces, lips, ears, eyes, irises, veins, moles, wounds, birthmarks, freckles, scars, wrinkles, fingerprints, thumbprints, palm prints, warts, categories, identities, instances, surfaces of internal organs, breasts, skin tumors, skin cancers, dysmorphologies, abnormalities, teeth, gums, facial expressions, facial macro expressions, facial micro expressions, facial subtle expressions, head gestures, hand gestures, arm gestures, gaits, body gestures, wagging tails, athletic motions, fighting positions, lip reading, crawling, talking, screaming, barking, breathing, running, galloping, eating, gun raising, axe swinging, phone talking, guitar playing, crowd behavior, health, mental state, range of motion, performance, weight, volume and concealed objects.