Image processing device

• US 10,540,807 B2
• Filed: 08/21/2018
• Issued: 01/21/2020
• Est. Priority Date: 09/15/2017
• Status: Active Grant

• Current Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Original Assignee: Aisin Seiki Co., Ltd. (Aisin Corp.)
• Inventors: Yamamoto, Kinji
• Primary Examiner(s): Alcon, Fernando
• Application Number:
  US16/107,146
• Publication Number:
  US 20190088006A1
• Time in Patent Office: 518 Days
• Field of Search: None
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image processing device includes: an acquisition unit that acquires an image by imaging surroundings of a vehicle; a generation unit that projects the image onto a three-dimensional projection plane including the vehicle at a first time point and a first projection area rising above the vehicle, and generates a composite image seen on the plane from a virtual viewpoint; a calculation unit that calculates a moving route of the vehicle from the first time point to a second time point on the plane; a change unit that changes a projection destination of at least a part of the moving route in the first projection area to a second projection area; and a display control unit that displays the composite image on the plane that includes the vehicle at the first time point, and the first and second projection areas seen from the virtual viewpoint on a display unit.

Claims

1. An image processing device comprising:

at least one processor configured to

acquire a captured image obtained by imaging surroundings of a vehicle by at least one camera mounted on the vehicle;

project the captured image onto a three-dimensional projection plane including the vehicle at a first time point and a first projection area rising above the vehicle, and generate a composite image seen on the three-dimensional projection plane from a virtual viewpoint;

calculate a moving route of the vehicle from the first time point to a second time point after the first time point on the three-dimensional projection plane according to a travel state of the vehicle;

change a projection destination of at least a part of the moving route in the first projection area to a second projection area that is horizontal with respect to the vehicle, while holding an area in the first projection area other than the moving route in the first projection area; and

display the composite image on the three-dimensional projection plane that includes the vehicle at the first time point, the first projection area, and the second projection seen from the virtual viewpoint on a display, the composite image being a result of the changing of the projection destination of the at least the part of the moving route.

2. The image processing device according to claim 1,

wherein the first projection area is positioned around the second projection area,

the second projection area includes the vehicle at the first time point, and

the second projection area is partially expanded such that at least a part of the moving route included in the first projection area is included in the second projection area.

3. The image processing device according to claim 2, wherein the at least one processor is further configured to:

detect a projection position of a three-dimensional object existing around the moving route in the second projection area,

wherein the second projection area is reduced such that the projection position is included in the first projection area.

4. The image processing device according to claim 1,

wherein the moving route of the vehicle under an autonomous driving control is calculated.

5. The image processing device according to claim 1, further comprising:

a storage configured to store a first map which associates coordinates of the three-dimensional projection plane with texture coordinates of the captured image projected on the coordinates of the three-dimensional projection plane, and a second map which associates coordinates of a two-dimensional projection plane including only a projection plane horizontal to the vehicle with the texture coordinates projected on the coordinates of the two-dimensional projection plane,

wherein, if the autonomous driving control is performed, a map is overwritten in the first map, which associates the coordinates of the moving route in the three-dimensional projection plane with the texture coordinates of the captured image projected on the coordinates, with the second map which associates the coordinates of the moving route in the two-dimensional projection plane with the texture coordinates of the captured image projected on the coordinates, and

the captured image is projected onto the three-dimensional projection plane according to the first map.