System for combining virtual and real-time environments

• US 20070035561A1
• Filed: 04/11/2005
• Published: 02/15/2007
• Est. Priority Date: 04/11/2005
• Status: Active Grant

• Current Assignee: Systems Technology Incorporated
• Original Assignee: Systems Technology Incorporated
• Inventors: Brickman, Noah, Bachelder, Edward
• Granted Patent: US 7,479,967 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

The present invention relates to a method and an apparatus for combining virtual reality and real-time environment. The present invention provides a system that combines captured real-time video data and real-time 3D environment rendering to create a fused (combined) environment. The system captures video imagery and processes it to determine which areas should be made transparent (or have other color modifications made), based on sensed cultural features and/or sensor line-of-sight. Sensed features can include electromagnetic radiation characteristics (i.e. color, infra-red, ultra-violet light). Cultural features can include patterns of these characteristics (i.e. object recognition using edge detection). This processed image is then overlaid on a 3D environment to combine the two data sources into a single scene. This creates an effect where a user can look through ‘windows’ in the video image into a 3D simulated world, and/or see other enhanced or reprocessed features of the captured image.

Claims

1. A virtual reality system comprising:

a camera adapted to receive images of a physical environment and to produce a frame of the physical environment that contains pixels representing a plurality of colors;

a predetermined target color;

a means for processing the frame and rendering pixels of the target color of the frame to be transparent pixels;

a means for generating a virtual image;

a means for combining the virtual image with the frame of the physical environment to form a combined image whereby the virtual image is visible at all positions of the transparent pixels; and

,

a means for displaying the combined image.

2. The system of claim 1 wherein the target color is magenta.

3. The system of claim 1 further including a means for comparing the pixels of the frame to the target color under different predetermined lighting conditions to determine a match with the target color exists at any of the predetermined lighting conditions.

4. The system of claim 1 further including a means for tracking locations of a physical object moving within a predetermined distance from the camera to produce a tracked object image and for replacing the tracked object image with a virtual image when the physical object moves beyond the predetermined distance from the camera.