Image processing device, image processing system, and image processing method

• US 9,706,175 B2
• Filed: 12/22/2010
• Issued: 07/11/2017
• Est. Priority Date: 12/24/2009
• Status: Active Grant

• Current Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Original Assignee: Fujitsu Ten Limited (DENSO Corporation)
• Inventors: Kinoshita, Kohtaroh, Koharazawa, Masahiro, Imu, Sunja
• Primary Examiner(s): BECKER, JOSEPH W
• Application Number:
  US13/517,118
• Publication Number:
  US 20120257058A1
• Time in Patent Office: 2,393 Days
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

In an image processing device configured to be installed in a vehicle, an image acquirer acquires camera images captured by cameras provided on the vehicle. A synthetic image generator generates synthetic images, which include downward views of the vehicle and a periphery thereof viewed from viewpoints different from one another, based on the camera images. A display image provider outputs, to the display device installed in the vehicle, information corresponding to a display image in which the synthetic images are continuously reproduced.

Claims

1. An image processing device configured to be installed in a vehicle, comprising:

an image acquirer configured to acquire camera images captured by cameras provided on the vehicle, the camera images containing information showing the whole periphery of the vehicle at the time of capturing the camera images;

a synthetic image generator configured to generate synthetic images based on the camera images, the synthetic images including first downward views of the vehicle and a periphery thereof viewed from virtual viewpoints different from one another; and

a display image provider configured to output, to a display device installed in the vehicle, information corresponding to a display image in which the synthetic images are continuously reproduced so as to perform an animated representation that shows orbiting around the vehicle, wherein the animated representation is started under a predetermined condition and a display mode is continuously changed in a predetermined order,

wherein the display mode includes;

a surrounding confirmation mode in which the information corresponding to the display image is output to the display device so as to perform the animated representation;

a front mode in which information corresponding to an image including a front of the vehicle is output to the display device;

a navigation mode in which information corresponding to a map image for a navigation guide is output to the display device; and

a back mode in which information corresponding to an image including a rear of the vehicle is output to the display device, and

wherein in the predetermined order is an order in which the surrounding confirmation mode is switched to the front mode when a predetermined time elapses after the surrounding confirmation mode starts, the front mode is switched to the navigation mode when a speed of the vehicle becomes equal to or more than a predetermined speed after the front mode starts, and the navigation mode is switched to the back mode when a shift lever of the vehicle is switched to a reverse position after the navigation mode starts.

2. The image processing device according to claim 1,

wherein the synthetic images includes a second downward view of the vehicle and the periphery thereof viewed from right above of the vehicle;

and

wherein the second downward view is reproduced at least once while the first downward views are reproduced, as the display image.

3. The image processing device according to claim 1, wherein each of the first downward views shows the vehicle at a center thereof.

4. The image processing device according to claim 2, wherein each of the first downward views shows the vehicle at a center thereof.

5. The image processing device according to claim 1, wherein the synthetic image generator is configured to generate the synthetic images when the image processing device is activated.

6. The image processing device according to claim 1, wherein the synthetic image generator is configured to eliminate a part of the camera images having a brightness lower than a reference value when the synthetic image is generated.

9. The image processing device according to claim 1, wherein the display image provider outputs the information corresponding to the display image in which the synthetic images are continuously reproduced so as to perform the animated representation that shows orbiting around the vehicle when a predetermined operation is performed when the vehicle is in a stopped state.

12. The image processing device according to claim 1, wherein the predetermined condition for starting the animated representation includes at least one of (i) an activation of the image processing device and (ii) a predetermined input operation by a user while the vehicle is in a stopped state.

15. The image processing device according to claim 1, wherein the animated representation shows orbiting around an entire periphery of the vehicle.

7. An image processing system comprising:

a plurality of cameras configured to be provided on a vehicle; and

an image processing device configured to be installed in the vehicle, comprising;

an image acquirer configured to acquire camera images captured by the cameras, the camera images containing information showing the whole periphery of the vehicle at the time of capturing the camera images;

a synthetic image generator configured to generate synthetic images based on the camera images, the synthetic images respectively being downward views of the vehicle and a periphery thereof viewed from virtual viewpoints different from one another; and

a display image provider configured to output, to a display device installed in the vehicle, information corresponding to a display image in which the synthetic images are continuously reproduced so as to perform an animated representation that shows orbiting around the vehicle, wherein the animated representation is started under a predetermined condition and a display mode is continuously changed in a predetermined order,

wherein the display mode includes;

a surrounding confirmation mode in which the information corresponding to the display image is output to the display device so as to perform the animated representation;

a front mode in which information corresponding to an image including a front of the vehicle is output to the display device;

a navigation mode in which information corresponding to a map image for a navigation guide is output to the display device; and

a back mode in which information corresponding to an image including a rear of the vehicle is output to the display device, and

wherein in the predetermined order is an order in which the surrounding confirmation mode is switched to the front mode when a predetermined time elapses after the surrounding confirmation mode starts, the from mode is switched to the navigation mode when a speed of the vehicle becomes equal to or more than a predetermined speed after the front mode starts, and the navigation mode is switched to the back mode when a shift lever of the vehicle is switched to a reverse position after the navigation mode starts.

10. The image processing system according to claim 7, wherein the display image provider outputs the information corresponding to the display image in which the synthetic images are continuously reproduced so as to perform the animated representation that shows orbiting around the vehicle when a predetermined operation is performed when the vehicle is in a stopped state.

13. The image processing system according to claim 7, wherein the predetermined condition for starting the animated representation includes at least one of (i) an activation of the image processing device and (ii) a predetermined input operation by a user while the vehicle is in a stopped state.

16. The image processing system according to claim 7, wherein the animated representation shows orbiting around an entire periphery of the vehicle.

8. An image processing method, comprising:

acquiring camera images captured by cameras provided on a vehicle, the camera images containing information showing the whole periphery of the vehicle at the time of capturing the camera images;

generating synthetic images based on the camera images, the synthetic images respectively being downward views of the vehicle and a periphery thereof viewed from virtual viewpoints different from one another; and

outputting, to a display device installed in the vehicle, information corresponding to a display image in which the synthetic images are continuously reproduced so as to perform an animated representation that shows orbiting around the vehicle, wherein the animated representation is started under a predetermined condition and a display mode is continuously changed in a predetermined order,

wherein the display mode includes;

a surrounding confirmation mode in which the information corresponding to the display image is output to the display device so as to perform the animated representation;

a front mode in which information corresponding to an image including a front of the vehicle is output to the display device;

a navigation mode in which information corresponding to a map image for a navigation guide is output to the display device; and

a back mode in which information corresponding to an image including a rear of the vehicle is output to the display device, and

wherein in the predetermined order is an order in which the surrounding confirmation mode is switched to the front mode when a predetermined time elapses after the surrounding confirmation mode starts, the front mode is switched to the navigation mode when a speed of the vehicle becomes equal to or more than a predetermined speed after the front mode starts, and the navigation mode is switched to the back mode when a shift lever of the vehicle is switched to a reverse position after the navigation mode starts.

11. The image processing method according to claim 8, wherein the information corresponding to the display image in which the synthetic images are continuously reproduced so as to perform the animated representation that shows orbiting around the vehicle is output when a predetermined operation is performed when the vehicle is in a stopped state.

14. The image processing method according to claim 8, wherein the predetermined condition for starting the animated representation includes at least one of (i) an activation of an image processing device and (ii) a predetermined input operation by a user while the vehicle is in a stopped state.

17. The image processing method according to claim 8, wherein the animated representation shows orbiting around an entire periphery of the vehicle.