VEHICLE SURROUNDING IMAGE DISPLAY CONTROL DEVICE, VEHICLE SURROUNDING IMAGE DISPLAY CONTROL METHOD, NON-TRANSITORY TANGIBLE COMPUTER-READABLE MEDIUM COMPRISING COMMAND INCLUDING THE METHOD, AND IMAGE PROCESSING METHOD EXECUTING TOP VIEW CONVERSION AND DISPLAY OF IMAGE OF VEHICLE SURROUNDINGS

• US 20150070394A1
• Filed: 03/08/2013
• Published: 03/12/2015
• Est. Priority Date: 05/23/2012
• Status: Abandoned Application

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Yanagawa, Hirohiko, Takeichi, Masakazu, Wang, Bingchen
• Application Number:
  US14/395,514
• Publication Number:
  US 20150070394A1
• Time in Patent Office: Days
• US Class Current: 345/649
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space

Abstract

A vehicle surrounding image display control device includes: a vehicle surrounding image acquisition device; a bird'"'"'s-eye view image creating device; a division storage device storing A and B bird'"'"'s-eye view images; a shadow determination device; a vehicle movement amount calculation device; a first history image configuration device updating a C bird'"'"'s-eye view image with the B bird'"'"'s-eye view image, and updating a B history bird'"'"'s-eye view image with the A bird'"'"'s-eye view image when the shadow is not present in the B bird'"'"'s-eye view image; a second history image configuration device updating the C bird'"'"'s-eye view image using the B history bird'"'"'s-eye view image, and updating the B history bird'"'"'s-eye view image using the A bird'"'"'s-eye view image when the shadow is present in the B bird'"'"'s-eye view image; and a display control device.

Claims

1. A vehicle surrounding image display control device comprising:

an acquisition device that repetitively acquires a taken image around a vehicle from an in-vehicle camera mounted in the vehicle;

a bird'"'"'s-eye view conversion device that sequentially executes a bird'"'"'s-eye view conversion of the taken image and creates a bird'"'"'s-eye view image;

a division storage device that divides the bird'"'"'s-eye view image along a front-back direction of the vehicle to create a A bird'"'"'s-eye view image in a predetermined A area farther from the vehicle and a B bird'"'"'s-eye view image in a predetermined B area closer to the vehicle, stores the A bird'"'"'s-eye view image in a real area A of a memory, and stores the B bird'"'"'s-eye view image in a real area B of the memory;

a shadow determination device that determines whether a shadow is present in the B bird'"'"'s-eye view image stored in a part or all of the B real area;

a movement calculation device that calculates an amount of movement of the vehicle based on vehicle behavior information input from the vehicle;

a first history image configuration device that;

configures a C bird'"'"'s-eye view image in a C history area of the memory using the B bird'"'"'s-eye view image stored in the B real area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area of the memory for storing a bird'"'"'s-eye view image in an C area outside of a photographing area of the in-vehicle camera reflects a present surrounding layout of the vehicle; and

configures a B history bird'"'"'s-eye view image in a B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B bird'"'"'s-eye view image in the B history area of the memory for storing a bird'"'"'s-eye view image in a same area as the B real area reflects the present surrounding layout of the vehicle,

when the shadow determination device determines that the shadow is not present in the B bird'"'"'s-eye view image;

a second history image configuration device that;

configures the C bird'"'"'s-eye view image in the C history area using the B history bird'"'"'s-eye view image stored in the B history area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area of the memory reflects the present surrounding layout of the vehicle; and

configures the B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B bird'"'"'s-eye view image in the B history area reflects the present surrounding layout of the vehicle,

when the shadow determination device determines that the shadow is present in the B bird'"'"'s-eye view image; and

a display control device that controls the image display device to display the A bird'"'"'s-eye view image in the A real area, the C bird'"'"'s-eye view image in the C history area, and the B bird'"'"'s-eye view image in the B real area or the B history bird'"'"'s-eye view image in the B history area.

2. The vehicle surrounding image display control device according to claim 1, further comprising:

a first display control device that controls the image display device to display the A bird'"'"'s-eye view image in the A real area, the B bird'"'"'s-eye view image in the B real area, and the C bird'"'"'s-eye view image in the C history area when the shadow determination device determines that the shadow is not present in the B bird'"'"'s-eye view image; and

a second display control device that controls the image display device to display the A bird'"'"'s-eye view image in the A real area, the B history bird'"'"'s-eye view image in the B history area, and the C bird'"'"'s-eye view image in the C history area when the shadow determination device determines that the shadow is present in the B bird'"'"'s-eye view image.

3. The vehicle surrounding image display control device according to claim 1, further comprising:

a data amount determination device that determines whether at least a predetermined amount of B history bird'"'"'s-eye view image data is present in the B history area,

wherein the first history image configuration device;

configures the C bird'"'"'s-eye view image in the C history area using the B bird'"'"'s-eye view image stored in the B real area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area reflects the present surrounding layout of the vehicle; and

configures the B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B bird'"'"'s-eye view image in the B history area reflects the present surrounding layout of the vehicle,

when the shadow determination device determines that the shadow is not present in the B bird'"'"'s-eye view image, or when the shadow determination device determines that the shadow is present in the B bird'"'"'s-eye view image, and the data amount determination device determines that an amount of B history bird'"'"'s-eye view image data is not equal to or larger than a predetermined amount, and

wherein the second history image configuration device;

configures the C bird'"'"'s-eye view image in the C history area using the B history bird'"'"'s-eye view image stored in the B history area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area reflects the present surrounding layout of the vehicle; and

configures the B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B history bird'"'"'s-eye view image in the B history area reflects the present surrounding layout of the vehicle,

when the shadow determination device determines that the shadow is present in the B bird'"'"'s-eye view image, and the data amount determination device determines that an amount of B history bird'"'"'s-eye view image data is equal to or larger than a predetermined amount.

4. The vehicle surrounding image display control device according to claim 1,

wherein the first history image configuration device:

sequentially updates the A real area, the B real area, and the C history area so that the bird'"'"'s-eye view image moves in a joining area that joins an A area, which is a display area of the A bird'"'"'s-eye view image, an B area, which is a display area of the B bird'"'"'s-eye view image, and an C area, which is a display area of the C bird'"'"'s-eye view image, according to the amount of movement of the vehicle; and

sequentially updates the B history area and the A real area so that the bird'"'"'s-eye view image moves in a joining area that joins a B area, which is a display area of the B history bird'"'"'s-eye view image, and an A area, which is a display area of the A bird'"'"'s-eye view image,

when the shadow determination device determines that the shadow is not present in the B bird'"'"'s-eye view image, and

wherein the second history image configuration device;

sequentially updates the A real area, the B history area, and the C history area so that the bird'"'"'s-eye view image moves in a joining area that joins an A area, which is a display area of the A bird'"'"'s-eye view image, a B area, which is a display area of the B history bird'"'"'s-eye view image, and a C area, which is a display area of the C bird'"'"'s-eye view image, according to the amount of movement of the vehicle,

when the shadow determination device determines that the shadow is present in the B bird'"'"'s-eye view image.

5. A vehicle surrounding image display method comprising:

repetitively acquiring a taken image around a vehicle from an in-vehicle camera mounted in a vehicle;

sequentially executing a bird'"'"'s-eye view conversion of the taken image to create a bird'"'"'s-eye view image;

dividing the bird'"'"'s-eye view image along a front-back direction of the vehicle to create a A bird'"'"'s-eye view image in a predetermined A area farther from the vehicle, and a B bird'"'"'s-eye view image in a predetermined B area closer to the vehicle, storing the A bird'"'"'s-eye view image in a A real area of a memory, and storing the B bird'"'"'s-eye view image in a B real area of the memory;

determining whether a shadow is present in the bird'"'"'s-eye view image stored in a part or all of the B real area;

calculating an amount of movement of the vehicle based on vehicle behavior information input from the vehicle;

configuring a C bird'"'"'s-eye view image in a C history area using the B bird'"'"'s-eye view image stored in the B real area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area of the memory for storing a bird'"'"'s-eye view image in an C area outside of a photographing area of the in-vehicle camera reflects a present surrounding layout of the vehicle, and configuring a B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B history bird'"'"'s-eye view image in the B history area of the memory for storing a bird'"'"'s-eye view image in a same area as the B real area reflects the present surrounding layout of the vehicle, when the shadow is not present in the B bird'"'"'s-eye view image;

configuring the C bird'"'"'s-eye view image in the C history area using the B history bird'"'"'s-eye view image stored in the B history area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area of the memory reflects a present surrounding layout of the vehicle, and configuring the B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B history bird'"'"'s-eye view image in the B history area reflects the present surrounding layout of the vehicle, when the shadow is present in the B bird'"'"'s-eye view image; and

controlling the image display device to display the A bird'"'"'s-eye view image in the A real area, the C bird'"'"'s-eye view image in the C history area, and the B bird'"'"'s-eye view image in the B real area or the B history bird'"'"'s-eye view image in the B history area.

6. A non-transitory tangible computer-readable medium comprising instructions being executed by a computer, the instructions including a computer-implemented method for controlling to display a vehicle surrounding image, the instructions including:

repetitively acquiring a taken image around a vehicle from an in-vehicle camera mounted in a vehicle;

sequentially executing a bird'"'"'s-eye view conversion of the taken image to create a bird'"'"'s-eye view image;

dividing the bird'"'"'s-eye view image along a front-back direction of the vehicle to create a A bird'"'"'s-eye view image in a predetermined A area farther from the vehicle, and a B bird'"'"'s-eye view image in a predetermined B area closer to the vehicle, storing the A bird'"'"'s-eye view image in a A real area of a memory, and storing the B bird'"'"'s-eye view image in a B real area of the memory;

determining whether a shadow is present in the bird'"'"'s-eye view image stored in a part or all of the B real area;

calculating an amount of movement of the vehicle based on vehicle behavior information input from the vehicle;

configuring a C bird'"'"'s-eye view image in a C history area using the B bird'"'"'s-eye view image stored in the B real area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area of the memory for storing a bird'"'"'s-eye view image in an C area outside of a photographing area of the in-vehicle camera reflects a present surrounding layout of the vehicle, and configuring a B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B history bird'"'"'s-eye view image in the B history area of the memory for storing a bird'"'"'s-eye view image in a same area as the B real area reflects the present surrounding layout of the vehicle, when the shadow is not present in the B bird'"'"'s-eye view image;

configuring the C bird'"'"'s-eye view image in the C history area using the B history bird'"'"'s-eye view image stored in the B history area according to the amount of movement of the vehicle so that the C bird'"'"'s-eye view image in the C history area of the memory reflects the present surrounding layout of the vehicle, and configuring the B history bird'"'"'s-eye view image in the B history area using the A bird'"'"'s-eye view image stored in the A real area according to the amount of movement of the vehicle so that the B history bird'"'"'s-eye view image in the B history area reflects the present surrounding layout of the vehicle, when the shadow is present in the B bird'"'"'s-eye view image; and

controlling the image display device to display the A bird'"'"'s-eye view image in the A real area, the C bird'"'"'s-eye view image in the C history area, and the B bird'"'"'s-eye view image in the B real area or the B history bird'"'"'s-eye view image in the B history area.

7. An image processing method of sequentially acquiring an image around a vehicle from an in-vehicle camera mounted in the vehicle, executing a top view conversion of the image, and displaying the image on an image display device mounted in the vehicle, the image processing method comprising:

dividing the image after executing the top view conversion into two image pieces;

determining whether a shadow of the vehicle is included in a short-range image closer to the vehicle in divided image pieces; and

replacing a present short-range image with a history image of a long-range image farther from the vehicle, which is taken before a predetermined time when the shadow of the vehicle is included in the short-range image.

8. A vehicle surrounding image display control device that acquires an image around a vehicle from an in-vehicle camera mounted in the vehicle, executes a top view conversion of the image to store the image in a memory, and displays the image on an image display device, the vehicle surrounding image display control device comprising:

an acquisition device that repetitively acquires a taken image around the vehicle;

a top view conversion device that sequentially executes the top view conversion of the taken image to create a series of top view image group;

a division storage device that;

divides a latest top view image along a front-back direction of the vehicle when the latest top view image in a created series of top view image group is stored in the memory;

stores a long-range top view image farther from the vehicle in an A real area; and

stores a short-range top view image closer to the vehicle in a B real area;

a shadow determination device that determines whether a shadow of the vehicle is reflected on the latest short-range top view image stored in the B real area);

a movement calculation device that calculates an amount of movement of the vehicle based on vehicle behavior information input from the vehicle;

a first history image configuration device that;

sets a C history area, which stores the history image outside of a photographing area of the in-vehicle camera and corresponding to a surrounding layout of the vehicle, and a B history area, which stores a short-range top view image in a same area as the B real area in the memory; and

overwrites the C history area with the short-range top view image stored in the B real area, and overwrites the B history area with the long-range top view image stored in the A real area, according to the amount of movement of the vehicle, when the shadow determination device determines that the shadow is not present;

a second history image configuration device that overwrites the C history area with the short-range top view image stored in the B history area, and overwrites the B history area with the long-range top view image stored in the A real area, according to the amount of movement of the vehicle, when the shadow determination device determines that the shadow is present; and

a display control device that controls the image display device to display the long-range top view image in the A real area, the history image in the C history area, and the short-range top view image in the B real area or in the B history area.

9. A vehicle surrounding image display control device comprising:

an acquisition device that repetitively acquires a taken image around a vehicle from an in-vehicle camera mounted in the vehicle;

a bird'"'"'s-eye view conversion device that sequentially executes a bird'"'"'s-eye view conversion of the taken image and creates a bird'"'"'s-eye view image;

a movement calculation device that calculates an amount of movement of the vehicle based on vehicle behavior information input from the vehicle;

a memory that includes a B real area for storing a B bird'"'"'s-eye view image in a predetermined B area around the vehicle, an A real area for storing an A bird'"'"'s-eye view image A in a predetermined A area farther from the vehicle than the B area, a B history area for storing as a B history bird'"'"'s-eye view image, which is prepared by moving a relative position of the A bird'"'"'s-eye view image stored in the A real area with respect to the vehicle according to the amount of movement of the vehicle calculated by the movement calculation device, and a C history area for storing as a bird'"'"'s-eye view image C, which is prepared by moving a relative position of the B history bird'"'"'s-eye view image stored in the B history area or the B bird'"'"'s-eye view image stored in the B real area with respect to the vehicle according to the amount of movement of the vehicle calculated by the movement calculation device;

a division storage device that divides the bird'"'"'s-eye view image along a front-back direction of the vehicle to create the A bird'"'"'s-eye view image and the B bird'"'"'s-eye view image, stores the A bird'"'"'s-eye view image in the A real area, and stores the B bird'"'"'s-eye view image in the B real area;

a shadow determination device that determines whether a shadow is present in the B bird'"'"'s-eye view image stored in a part or all of the B real area;

a first history image configuration device that configures the C bird'"'"'s-eye view image in the C history area using the B bird'"'"'s-eye view image stored in the B real area when the shadow determination device determines that the shadow is not presented in the B bird'"'"'s-eye view image;

a second history image configuration device that configures the C bird'"'"'s-eye view image in the C history area using the B history bird'"'"'s-eye view image stored in the B history area when the shadow determination device determines that the shadow is present in the B bird'"'"'s-eye view image; and

a display control device that controls the image display device to display the A bird'"'"'s-eye view image in the A real area, the C bird'"'"'s-eye view image in the C history area, and the B bird'"'"'s-eye view image in the B real area or the B history bird'"'"'s-eye view image in the B history area.