Image generation apparatus

• US 10,499,014 B2
• Filed: 12/22/2015
• Issued: 12/03/2019
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: DENSO Ten Limited (f/k/a Fujitsu Ten Ltd.) (DENSO Corporation)
• Inventors: Sawada, Yasuyoshi
• Primary Examiner(s): Kwan, Matthew K
• Application Number:
  US14/978,196
• Publication Number:
  US 20160212384A1
• Time in Patent Office: 1,442 Days
• Field of Search: 348159
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B62D 15/029 Steering assistants using warnings or proposing actions to the driver without influencing the steering system
• CPC Class Codes: B62D 15/0295 by overlaying a vehicle path based on present steering angle over an image without processing that image
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image generation apparatus that is used in a vehicle, the image generation apparatus includes: an image generator that temporally continuously generates a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras; and a controller configured to control the image generator to: (i) temporally continuously change a position of the virtual viewpoint so that the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle; (ii) acquire a detection result from a detector that detects an object existing in the vicinity of the vehicle; and (iii) determine a rotation direction of the moving image based on a position of the object detected by the detector.

Claims

1. An image generation apparatus that is used in a vehicle, the image generation apparatus comprising:

an image generator circuit that temporally continuously generates a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras; and

a microcomputer configured to control the image generator circuit to;

(i) temporally continuously change a position of a center point of the virtual viewpoint around the vehicle so that each of the plurality of virtual viewpoint images is from a different virtual viewpoint around the vehicle and the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle;

(ii) acquire a detection result from a detector that detects an object existing in the vicinity of the vehicle; and

(iii) based on an object position of the object detected by the detector, determine a rotation direction of the center point of the virtual viewpoint around the vehicle to be either a clockwise direction or a counter-clockwise direction depending on which of the clockwise and counterclockwise directions causes the center point of the virtual viewpoint to approach the object position of the object detected by the detector in a quicker manner.

2. The image generation apparatus according to claim 1, wherein the microcomputer is further

configured to derive a risk level of each of a plurality of objects in a case where the detector has detected the plurality of objects, the risk level being a degree of possibility of each of the plurality of objects to come into contact with the vehicle;

wherein the object position is

a position of one object of the plurality of objects having a greatest risk level.

3. The image generation apparatus according to claim 2, wherein

the microcomputer derives the risk level of each of the plurality of objects based on a position of each of the plurality of objects.

4. The image generation apparatus according to claim 3, wherein

the microcomputer derives the risk level of each of the plurality of objects based on (1) a prediction path in which the vehicle is expected to run and (2) the position of each of the plurality of objects.

5. The image generation apparatus according to claim 1, wherein

the microcomputer causes a rotation speed of the center point of the virtual viewpoint around the vehicle to be slower than a normal rotation speed in a case where a sight line of the virtual viewpoint is directed toward the object, and

the microcomputer causes the rotation speed of the center point of the virtual viewpoint around the vehicle to be faster than the normal rotation speed in a case where the sight line of the virtual viewpoint is directed toward a region not including the object.

6. The image generation apparatus according to claim 1, wherein

the microcomputer changes the position of the virtual viewpoint so as to direct a sight line of the virtual viewpoint only toward the object by skipping a region not including the object in the vicinity of the vehicle.

7. The image generation apparatus according to claim 1, further comprising:

a display that displays the moving image.

8. An image generation method that is used in a vehicle, the image generation method comprising the steps of:

(a) temporally continuously generating, by an image generator circuit, a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras;

(b) temporally continuously changing, by a microcomputer that controls the image generator circuit, a position of a center point of the virtual viewpoint around the vehicle so that the plurality of virtual viewpoint images is from a different viewpoint around the vehicle and the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle;

(c) acquiring, by the microcomputer, a detection result from a detector that detects an object existing in the vicinity of the vehicle; and

(d) based on an object position of the object detected by the detector, determining, by the microcomputer, a rotation direction of the center point of the virtual viewpoint around the vehicle to be either a clockwise direction or a counter-clockwise direction depending on which of the clockwise and counterclockwise directions causes the center point of the virtual viewpoint to approach the object position of the object detected by the detector in a quicker manner.

9. The image generation method according to claim 8, further comprising the step of:

(e) deriving, by the microcomputer, a risk level of each of a plurality of objects in a case where the detector has detected the plurality of objects, the risk level being a degree of possibility of each of the plurality of objects to come into contact with the vehicle,

wherein

on the object position is a position of one object of the plurality of objects having a greatest risk level.

10. The image generation method according to claim 9, wherein

the step (e) derives the risk level of each of the plurality of objects based on a position of each of the plurality of objects.

11. The image generation method according to claim 10, wherein

the step (e) derives the risk level of each of the plurality of objects based on (1) a prediction path in which the vehicle is expected to run and (2) the position of each of the plurality of objects.

12. The image generation method according to claim 8, wherein

the step (b) causes a rotation speed of the center point of the virtual viewpoint around the vehicle to be slower than a normal rotation speed in a case where a sight line of the virtual viewpoint is directed toward the object, and

the step (b) causes the rotation speed of the center point of the virtual viewpoint around the vehicle to be faster than the normal rotation speed in a case where the sight line of the virtual viewpoint is directed toward a region not including the object.

13. The image generation method according to claim 8, wherein

the step (b) changes the position of the virtual viewpoint so as to direct a sight line of the virtual viewpoint only toward the object by skipping a region not including the object in the vicinity of the vehicle.

14. A non-transitory computer-readable recording medium that stores a program to be executed by a computer that is used in a vehicle, the program causing the computer to execute the steps of:

(a) temporally continuously generating a plurality of virtual viewpoint images showing a vicinity of the vehicle viewed from a virtual viewpoint, using a plurality of captured images acquired by a plurality of cameras;

(b) temporally continuously changing a position of a center point of the virtual viewpoint around the vehicle so that the plurality of virtual viewpoint images is from a different virtual viewpoint around the vehicle and the plurality of virtual viewpoint images appear to be a moving image showing the vicinity of the vehicle in such a way as to go around the vicinity of the vehicle;

(c) acquiring a detection result from a detector that detects an object existing in the vicinity of the vehicle; and

(d) based on an object position of the object detected by the detector, determining a rotation direction of the center point of the virtual viewpoint around the vehicle to be either a clockwise direction or a counter-clockwise direction depending on which of the clockwise and counterclockwise directions causes the center point of the virtual viewpoint to approach the object position of the object detected by the detector in a quicker manner.

15. The recording medium according to claim 14, wherein the program further causes the computer to execute the step of:

(e) deriving a risk level of each of a plurality of objects in a case where the detector has detected the plurality of objects, the risk level being a degree of possibility of each of the plurality of objects to come into contact with the vehicle,

wherein

the object position is a position of one object of the plurality of objects having a greatest risk level.

16. The recording medium according to claim 15, wherein

the step (e) derives the risk level of each of the plurality of objects based on a position of each of the plurality of objects.

17. The recording medium according to claim 16, wherein

the step (e) derives the risk level of each of the plurality of objects based on (1) a prediction path in which the vehicle is expected to run and (2) the position of each of the plurality of objects.

18. The recording medium according to claim 14, wherein

the step (b) causes a rotation speed of the center point of the virtual viewpoint around the vehicle to be slower than a normal rotation speed in a case where a sight line of the virtual viewpoint is directed toward the object, and

the step (b) causes the rotation speed of the center point of the virtual viewpoint around the vehicle to be faster than the normal rotation speed in a case where the sight line of the virtual viewpoint is directed toward a region not including the object.

19. The recording medium according to claim 14, wherein

the step (b) changes the position of the virtual viewpoint so as to direct a sight line of the virtual viewpoint only toward the object by skipping a region not including the object in the vicinity of the vehicle.