System for combining virtual and real-time environments
First Claim
1. A virtual reality system comprising:
- a camera adapted to receive images of a physical environment and to produce a frame of the physical environment that contains pixels representing a plurality of colors;
a predetermined target color;
a means for processing the frame and rendering pixels of the target color of the frame to be transparent pixels;
a means for generating a virtual image;
a means for combining the virtual image with the frame of the physical environment to form a combined image whereby the virtual image is visible at all positions of the transparent pixels; and
,a means for displaying the combined image.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention relates to a method and an apparatus for combining virtual reality and real-time environment. The present invention provides a system that combines captured real-time video data and real-time 3D environment rendering to create a fused (combined) environment. The system captures video imagery and processes it to determine which areas should be made transparent (or have other color modifications made), based on sensed cultural features and/or sensor line-of-sight. Sensed features can include electromagnetic radiation characteristics (i.e. color, infra-red, ultra-violet light). Cultural features can include patterns of these characteristics (i.e. object recognition using edge detection). This processed image is then overlaid on a 3D environment to combine the two data sources into a single scene. This creates an effect where a user can look through ‘windows’ in the video image into a 3D simulated world, and/or see other enhanced or reprocessed features of the captured image.
-
Citations
20 Claims
-
1. A virtual reality system comprising:
-
a camera adapted to receive images of a physical environment and to produce a frame of the physical environment that contains pixels representing a plurality of colors; a predetermined target color; a means for processing the frame and rendering pixels of the target color of the frame to be transparent pixels; a means for generating a virtual image; a means for combining the virtual image with the frame of the physical environment to form a combined image whereby the virtual image is visible at all positions of the transparent pixels; and
,a means for displaying the combined image. - View Dependent Claims (2, 3, 4, 12, 13, 14, 15, 16, 17, 18)
-
-
5. A method for combining virtual reality environments and real-time environments for a user comprising:
-
identifying a target masking color; providing a real-time environment having a predesignated area to be overlaid with virtual reality images and having the target masking color covering the predesignated area; providing the virtual reality images; providing the user with a head mounted display; providing a camera and mounting the camera at a location on the head of the user and from which the camera has a view substantially similar to the view of the user'"'"'s eyes; providing 3D spatial information about the location and direction of the user'"'"'s head; operating the camera to provide a real video image of the real-time environment; capturing the real video image in digital pixels; identifying all areas of the real video image to be overlaid with virtual video images by comparing the color of all of the pixels to the target color; making transparent all pixels whose color matches the target color; overlaying the real video images onto the virtual video images to form a combined image; and
,providing to the head mounted display the combined display.
-
-
6. A computer-implemented system for combining a virtual reality environment and a physical environment for a user comprising:
-
a computer; a camera operatively connected to the computer and adapted to provide to the computer real-time physical environment video images in digital pixels and adapted to be mounted on the head of a user at a location from which the camera has a view substantially similar to the view of the user'"'"'s eyes; a physical object operatively connected to the computer, and adapted to interact with the user and to provide input to the computer in response to interaction with the user; a virtual image generator operatively connected to the computer and adapted to provide to the computer virtual video images in digital pixels; a position detector operatively connected to the computer, and adapted to be mounted on the head of the user and to provide to the computer 3D spatial information about the location and direction of the user'"'"'s head; an image display operatively connected to the computer, adapted to be mounted on the head of the user and adapted to receive video images from the computer; a real-time physical environment; a pre-determined target masking color covering a predesignated area of the real-time physical environment; all areas of the real-time physical environment video images in which the color of the pixels of the real-time physical environment video images match the target color being transparent; the computer adapted to change the virtual video images in response to the input to the computer from the physical object; the real-time physical environment video images overlaid onto the virtual video images to form combined video images; and
,the combined video images provided to the image display. - View Dependent Claims (7, 8, 9, 10, 11, 19, 20)
-
Specification