Apparatus and method for dynamic reflecting car mirrors in virtual reality applications in head mounted displays

• US 10,453,262 B1
• Filed: 07/20/2017
• Issued: 10/22/2019
• Est. Priority Date: 07/21/2016
• Status: Active Grant

• Current Assignee: RelayCars, LLC
• Original Assignee: RelayCars, LLC
• Inventors: Falstrup, David, Denick, Thom, McClain, Alex, Johnson, Aaron, Peterson, Seth, Hadnagy, Pat
• Primary Examiner(s): Wu, Yanna
• Application Number:
  US15/655,415
• Time in Patent Office: 824 Days
• Field of Search: None
• CPC Class Codes: G06T 15/80 Shading
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 3/16 Spatio-temporal transformations, e.g. video cubism
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An apparatus and method for the creation of dynamically reflecting car mirrors in a virtual application using a proprietary layered panorama method. This approach utilizes the panoramic background as a source image for the reflection and specially produced masks for the mirrors in each stereo panoramic vehicle image.

Claims

1. A method comprising:

a. outlining mirror areas in a vehicle;

b. outlining a rear view mirror, a driver'"'"'s side mirror and a passenger'"'"'s side mirror;

c. filling said outlined rear view mirror with a white mask, filling said outlined driver'"'"'s side mirror with a white mask, and filling said outlined passenger'"'"'s side mirror with a white mask;

d. filling each white mask with an image simulating a reflected mirror image of at least one environment at least behind said vehicle;

e. separating each image as viewed from a reflection from each of said rear view mirror, said driver'"'"'s side mirror and said passenger'"'"'s side mirror;

f. creating a separate environmental panoramic image from each image obtained from step “

e”

by using a camera to create a reflected panoramic image from each of said mirrors, including, for each of said mirrors;

i. placing the camera facing toward a rear of the vehicle to obtain the reflected panoramic image,

ii. inverting the reflected panoramic image obtained from step (f)(i) to create an inverted panoramic image,

iii. darkening the inverted panoramic image from step (f)(ii); and

iv. projecting the darkened inverted panoramic image obtained from step (f)(iii) into a shader program to create an illusion that each of said rear view mirror, driver'"'"'s side mirror and passenger'"'"'s side mirror are reflecting said at least one environment in reverse.

2. The method in accordance with claim 1, wherein each said at least one environment in step “

d”

is created from a computer graphics application.

3. The method in accordance with claim 1, wherein the creating a separate environmental panoramic image includes using the shader program to separate each reflected panoramic image from said rear view mirror, said driver'"'"'s side mirror and said passenger'"'"'s side mirror.

4. A method comprising:

a. outlining mirror areas of a vehicle;

b. filling the outlined mirror areas with a white mask;

c. filling said white mask with an image simulating a reflected mirror image of an environment behind said vehicle;

d. separating the reflected mirror image as viewed as a reflection from each of a rear view mirror, a driver'"'"'s side mirror and a passenger'"'"'s side mirror associated with said mirror areas;

e. creating separate environmental panoramic images from each said separated reflected mirror image from the rear view mirror, the driver'"'"'s side mirror and the passenger'"'"'s side mirror obtained from step “

d”

by using a camera to create a reflected panoramic image from each of the rear view mirror, the driver'"'"'s side mirror and the passenger'"'"'s side mirror, including, for each of the rear view mirror, the driver'"'"'s side mirror and the passenger'"'"'s side mirror;

i. placing the camera facing toward a rear of the vehicle to obtain the reflected panoramic image,

ii. inverting the reflected panoramic image obtained from step (e)(i) to create an inverted reflected panoramic image,

iii. darkening said inverted reflected panoramic image from step (e)(ii) and

iv. projecting said darkened inverted reflected panoramic image obtained from step (e)(iii) into a shader to create an illusion that said mirror is reflecting said environment in reverse.

5. The method in accordance with claim 4, wherein said environment in step “

c”

is created within a game engine software application.

6. The method in accordance with claim 4, further comprising:

using a computer graphics shader program to create the separate environmental panoramic images.