IMAGE GENERATION APPARATUS

• US 20160212384A1
• Filed: 12/22/2015
• Published: 07/21/2016
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Inventors: SAWADA, Yasuyoshi
• Granted Patent: US 10,499,014 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image generation apparatus that is used in a vehicle, the image generation apparatus includes: an image generator that temporally continuously generates a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras; and a controller configured to control the image generator to: (i) temporally continuously change a position of the virtual viewpoint so that the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle; (ii) acquire a detection result from a detector that detects an object existing in the vicinity of the vehicle; and (iii) determine a rotation direction of the moving image based on a position of the object detected by the detector.

Claims

1. An image generation apparatus that is used in a vehicle, the image generation apparatus comprising:

an image generator that temporally continuously generates a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras; and

a controller configured to control the image generator to;

(i) temporally continuously change a position of the virtual viewpoint so that the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle;

(ii) acquire a detection result from a detector that detects an object existing in the vicinity of the vehicle; and

(iii) determine a rotation direction of the moving image based on a position of the object detected by the detector.

2. The image generation apparatus according to claim 1, wherein the controller is further configured to:

(iv) derive a risk level of each of a plurality of objects in a case where the detector has detected the plurality of objects, the risk level being a degree of possibility of each of the plurality of objects to come into contact with the vehicle, and

the controller determines the rotation direction of the moving image based on a position of one object of the plurality of objects having a greatest risk level.

3. The image generation apparatus according to claim 2, wherein

the controller derives the risk level of each of the plurality of objects based on a position of each of the plurality of objects.

4. The image generation apparatus according to claim 3, wherein

the controller derives the risk level of each of the plurality of objects based on (1) a prediction path in which the vehicle is expected to run and (2) the position of each of the plurality of objects.

5. The image generation apparatus according to claim 1, wherein

the controller causes a rotation speed of the moving image to be slower than a normal rotation speed in a case where a sight line of the virtual viewpoint is directed toward the object, and

the controller causes the rotation speed of the moving image to be faster than the normal rotation speed in a case where the sight line of the virtual viewpoint is directed toward a region not including the object.

6. The image generation apparatus according to claim 1, wherein

the controller changes the position of the virtual viewpoint so as to direct a sight line of the virtual viewpoint only toward the object by skipping a region not including the object in the vicinity of the vehicle.

7. The image generation apparatus according to claim 1, further comprising:

a display that displays the moving image.

8. An image generation method that is used in a vehicle, the image generation method comprising the steps of:

(a) temporally continuously generating, by an image generator, a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras;

(b) temporally continuously changing, by a controller that controls the image generator, a position of the virtual viewpoint so that the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle;

(c) acquiring, by the controller, a detection result from a detector that detects an object existing in the vicinity of the vehicle; and

(d) determining, by the controller, a rotation direction of the moving image based on a position of the object detected by the detector.

9. The image generation method according to claim 8, further comprising the step of:

(e) deriving, by the controller, a risk level of each of a plurality of objects in a case where the detector has detected the plurality of objects, the risk level being a degree of possibility of each of the plurality of objects to come into contact with the vehicle, wherein

the step (d) determines the rotation direction of the moving image based on a position of one object of the plurality of objects having a greatest risk level.

10. The image generation method according to claim 9, wherein

the step (e) derives the risk level of each of the plurality of objects based on a position of each of the plurality of objects.

11. The image generation method according to claim 10, wherein

the step (e) derives the risk level of each of the plurality of objects based on (1) a prediction path in which the vehicle is expected to run and (2) the position of each of the plurality of objects.

12. The image generation method according to claim 8, wherein

the step (b) causes a rotation speed of the moving image to be slower than a normal rotation speed in a case where a sight line of the virtual viewpoint is directed toward the object, and

the step (b) causes the rotation speed of the moving image to be faster than the normal rotation speed in a case where the sight line of the virtual viewpoint is directed toward a region not including the object.

13. The image generation method according to claim 8, wherein

the step (b) changes the position of the virtual viewpoint so as to direct a sight line of the virtual viewpoint only toward the object by skipping a region not including the object in the vicinity of the vehicle.

14. A non-transitory computer-readable recording medium that stores a program to be executed by a computer that is used in a vehicle, the program causing the computer to execute the steps of:

(a) temporally continuously generating a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras;

(b) temporally continuously changing a position of the virtual viewpoint so that the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle;

(c) acquiring a detection result from a detector that detects an object existing in the vicinity of the vehicle; and

(d) determining a rotation direction of the moving image based on a position of the object detected by the detector.

15. The recording medium according to claim 14, wherein the program further causes the computer to execute the step of:

(e) deriving a risk level of each of a plurality of objects in a case where the detector has detected the plurality of objects, the risk level being a degree of possibility of each of the plurality of objects to come into contact with the vehicle, wherein

the step (d) determines the rotation direction of the moving image based on a position of one object of the plurality of objects having a greatest risk level.

16. The recording medium according to claim 15, wherein

the step (e) derives the risk level of each of the plurality of objects based on a position of each of the plurality of objects.

17. The recording medium according to claim 16, wherein

the step (e) derives the risk level of each of the plurality of objects based on (1) a prediction path in which the vehicle is expected to run and (2) the position of each of the plurality of objects.

18. The recording medium according to claim 14, wherein

the step (b) causes a rotation speed of the moving image to be slower than a normal rotation speed in a case where a sight line of the virtual viewpoint is directed toward the object, and

the step (b) causes the rotation speed of the moving image to be faster than the normal rotation speed in a case where the sight line of the virtual viewpoint is directed toward a region not including the object.

19. The recording medium according to claim 14, wherein

the step (b) changes the position of the virtual viewpoint so as to direct a sight line of the virtual viewpoint only toward the object by skipping a region not including the object in the vicinity of the vehicle.