DRIVING SIMULATION APPARATUS, WIDE-ANGLE CAMERA VIDEO SIMULATION APPARATUS, AND IMAGE DEFORMING/COMPOSITING APPARATUS

• US 20100136507A1
• Filed: 09/29/2009
• Published: 06/03/2010
• Est. Priority Date: 12/01/2008
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Fujita, Takushi, Miyata, Ryosuke
• Granted Patent: US 8,907,950 B2
• Time in Patent Office: Days
• US Class Current: 434/29
• CPC Class Codes: G09B 9/04 for teaching control of land vehicles
• CPC Class Codes: G09B 9/05 the view from a vehicle being simulated G09B9/052, G09B9/058 take precedence

Abstract

A driving simulation apparatus includes an input unit that receives an input of an operation instruction, a vehicle simulation unit that calculates a position and an orientation of a vehicle in a virtual environment in real-time, a camera state calculation unit that calculates a viewpoint position and a viewing direction of a on-vehicle wide-angle camera based on the position and the orientation of the vehicle, and a wide-angle camera simulation unit. The wide-angle camera simulation unit includes an intermediate image rendering unit that generates multiple intermediate images by, for multiple intermediate projection planes that cover a given field of view from the on-vehicle wide-angle camera, performing perspective projection on the virtual environment from the viewpoint position and the viewing direction of the on-vehicle wide-angle camera, and a video rendering unit that generates a on-vehicle wide-angle camera video by deforming and compositing the intermediate images based on a parameter generated based on a lens characteristic of the on-vehicle wide-angle camera that has been recorded in advance.

Claims

1. A driving simulation apparatus comprising;

an input unit that receives an input of an operation instruction with respect to a vehicle in a virtual environment;

a vehicle simulation unit that calculates a position and an orientation of the vehicle in the virtual environment in real-time in accordance with the input of the operation instruction;

a camera state calculation unit that obtains a viewpoint position and a viewing direction of a on-vehicle wide-angle camera based on the position and the orientation of the vehicle; and

a wide-angle camera simulation unit including;

an intermediate image rendering unit that generates a plurality of intermediate images by, for a plurality of intermediate projection planes that cover a given field of view from the on-vehicle wide-angle camera, performing perspective projection on the virtual environment from both the viewpoint position and the viewing direction of the on-vehicle wide-angle camera that are calculated by the camera state calculation unit, and

a video rendering unit that generates a on-vehicle wide-angle camera video by generating a parameter relating to deforming and compositing intermediate images based on a lens characteristic of the on-vehicle wide-angle camera that has been recorded in advance, and deforming and compositing the plurality of intermediate images with use of the parameter.

2. The driving simulation apparatus according to claim 1,

wherein the video rendering unit generates the on-vehicle wide-angle camera video by generating, as the parameter, data expressing a correspondence relationship between the plurality of intermediate projection planes and a capturing plane of the on-vehicle wide-angle camera, and mapping textures of the plurality of intermediate images to the capturing plane with use of the data expressing the correspondence relationship.

3. The driving simulation apparatus according to claim 1,

wherein the intermediate image rendering unit causes hardware to execute processing in which the plurality of intermediate images are generated by performing perspective projection from the viewpoint position and the viewing direction of the on-vehicle wide-angle camera, and

the video rendering unit causes hardware to execute processing in which the on-vehicle wide-angle camera video is generated by deforming and compositing the plurality of intermediate images with use of the parameter obtained based on the lens characteristic of the on-vehicle wide-angle camera.

4. The driving simulation apparatus according to claim 1,

wherein the intermediate image rendering unit generates the plurality of intermediate images by performing perspective projection on the virtual environment from the viewpoint position and the viewing direction of the on-vehicle wide-angle camera, with use of, as the plurality of intermediate projection planes, at least two planes among six planes of a cube in which the viewpoint of the on-vehicle camera is included,

the driving simulation apparatus further comprises a direction vector attachment unit that, for each node of a two-dimensional mesh arranged on a screen of the on-vehicle wide-angle camera video to be generated, calculates a direction vector that associates the node and a position on an intermediate projection plane from among the plurality of intermediate projection planes, based on the parameter obtained based on the lens characteristic of the on-vehicle wide-angle camera,

wherein the video rendering unit generates the on-vehicle wide-angle camera video by mapping textures of the plurality of intermediate images to the two-dimensional mesh with use of the direction vectors.

5. The driving simulation apparatus according to claim 1,

wherein the intermediate image rendering unit generates the plurality of intermediate images by performing perspective projection on the virtual environment from the viewpoint position and the viewing direction of the on-vehicle wide-angle camera, with use of, as the plurality of intermediate projection planes, at least two planes among six planes of a cuboid in which the viewpoint of the on-vehicle camera is included,

the driving simulation apparatus further comprises

a direction vector attachment unit that, for each node of a two-dimensional mesh arranged on a screen of the on-vehicle wide-angle camera video, calculates a direction vector that associates the node and a position on an intermediate projection plane from among the plurality of intermediate projection planes, based on the parameter obtained based on the lens characteristic of the on-vehicle wide-angle camera, and

a vector conversion unit that, based on a positional relationship between the cuboid and the viewpoint of the on-vehicle wide-angle camera, converts the direction vectors calculated by the direction vector attachment unit into vectors that, for each node of the two-dimensional mesh, associate the node and a position on a plane of a post-deformation cube obtained when the cuboid is deformed into a cube whose center is the camera viewpoint,

wherein the video rendering unit generates the on-vehicle wide-angle camera video by mapping textures of the plurality of intermediate images to the two-dimensional mesh with use of the direction vectors.

6. The driving simulation apparatus according to claim 4,

wherein a cube mapping function is used as processing in which the textures of the plurality of intermediate images are mapped to the two-dimensional mesh with use of the direction vectors.

7. The driving simulation apparatus according to claim 5,

wherein a cube mapping function is used as processing in which the textures of the plurality of intermediate images are mapped to the two-dimensional mesh with use of the direction vectors.

8. The driving simulation apparatus according to claim 1, further comprising a screen coordinate conversion unit that converts a screen coordinate in accordance with a video deformation pattern that has been recorded in advance.

9. The driving simulation apparatus according to claim 4,

wherein partition lines of the two-dimensional mesh arranged on the screen of the on-vehicle wide-angle camera video to be generated have been set so as to overlap with, among boundary lines of the plurality of intermediate projection planes, boundary lines that are to be straight lines on the screen of the on-vehicle wide-angle camera video to be generated.

10. The driving simulation apparatus according to claim 5,

wherein partition lines of the two-dimensional mesh arranged on the screen of the on-vehicle wide-angle camera video to be generated have been set so as to overlap with, among boundary lines of the plurality of intermediate projection planes, boundary lines that are to be straight lines on the screen of the on-vehicle wide-angle camera video to be generated.

11. A wide-angle camera video simulation apparatus comprising:

a camera information input unit that receives an input of data expressing a viewpoint position and a viewing direction of a wide-angle camera in a virtual environment;

an intermediate image rendering unit that generates a plurality of intermediate images by, for a plurality of intermediate projection planes that cover a given field of view from the wide-angle camera, performing perspective projection on the virtual environment from both the viewpoint position and the viewing direction of the on-vehicle wide-angle camera that are input by the camera information input unit; and

a video rendering unit that generates a wide-angle camera video by generating a parameter relating to deforming and compositing intermediate images based on a lens characteristic of the wide-angle camera that has been recorded in advance, and deforming and compositing the plurality of intermediate images with use of the parameter.

12. A computer-readable recording media that stores a program for simulating driving, the program causing a computer to execute:

input processing in which an input of an operation instruction with respect to a vehicle in a virtual environment is received;

vehicle state calculation processing in which a position and an orientation of the vehicle in the virtual environment are calculated in real-time in accordance with the input of the operation instruction;

camera state calculation processing in which a viewpoint position and a viewing direction of a on-vehicle wide-angle camera are obtained based on the position and the orientation of the vehicle;

intermediate image rendering processing in which a plurality of intermediate images are generated by, for a plurality of intermediate projection planes that cover a given field of view from the on-vehicle wide-angle camera, performing perspective projection on the virtual environment from both the viewpoint position and the viewing direction of the on-vehicle wide-angle camera that are calculated in the camera state calculation processing; and

video rendering processing in which a on-vehicle wide-angle camera video is generated by generating a parameter relating to deforming and compositing intermediate images based on a lens characteristic of the on-vehicle wide-angle camera that has been recorded in advance, and deforming and compositing the plurality of intermediate images.

13. A computer-readable recording media that stores a program for simulating a wide-angle camera video or deforming and compositing images, the program causing a computer to execute:

camera information input processing in which an input of data expressing a viewpoint position and a viewing direction of a wide-angle camera in a virtual environment is received;

intermediate image rendering processing in which a plurality of intermediate images are generated by, for a plurality of intermediate projection planes that cover a given field of view from the wide-angle camera, performing perspective projection on the virtual environment from both the viewpoint position and the viewing direction of the wide-angle camera that are input in the camera information input processing; and

video rendering processing in which a wide-angle camera video is generated by generating a parameter relating to deforming and compositing intermediate images based on a lens characteristic of the wide-angle camera that has been recorded in advance, and deforming and compositing the plurality of intermediate images.

14. A method for causing a computer to simulate driving, comprising:

receiving an input of an operation instruction with respect to a vehicle in a virtual environment;

calculating a position and an orientation of the vehicle in the virtual environment in real-time in accordance with the input of the operation instruction;

obtaining, with use of mounting position information of a on-vehicle wide-angle camera of the vehicle that has been recorded in advance, a viewpoint position and a viewing direction of the on-vehicle wide-angle camera based on the position and the orientation of the vehicle;

generating a plurality of intermediate images by, for a plurality of intermediate projection planes that cover a given field of view from the on-vehicle wide-angle camera, performing perspective projection on the virtual environment from both the viewpoint position and the viewing direction of the on-vehicle wide-angle camera that are calculated in the camera state calculating step; and

generating a on-vehicle wide-angle camera video by generating a parameter relating to deforming and compositing intermediate images based on a lens characteristic of the on-vehicle wide-angle camera that has been recorded in advance, and deforming and compositing the plurality of intermediate images with use of the parameter.

15. A method for causing a computer to simulate a wide-angle camera video or deforming and compositing images, comprising:

inputting data expressing a viewpoint position and a viewing direction of a wide-angle camera in a virtual environment;

generating a plurality of intermediate images by, for a plurality of intermediate projection planes that cover a given field of view from the wide-angle camera, performing perspective projection on the virtual environment from both the viewpoint position and the viewing direction of the wide-angle camera that are input in the camera information inputting step; and

generating a wide-angle camera video by generating a parameter relating to deforming and compositing intermediate images based on a lens characteristic of the wide-angle camera that has been recorded in advance, and deforming and compositing the plurality of intermediate images.