Vehicle device, calibration result determination method, and non-transitory storage medium

• US 10,861,192 B2
• Filed: 02/11/2020
• Issued: 12/08/2020
• Est. Priority Date: 09/12/2017
• Status: Active Grant

• Current Assignee: JVC Kenwood Corporation
• Original Assignee: JVC Kenwood Corporation
• Inventors: Nagai, Katsuyuki, Arase, Takayuki
• Primary Examiner(s): Aghevli, Reza
• Application Number:
  US16/787,078
• Publication Number:
  US 20200175722A1
• Time in Patent Office: 301 Days
• Field of Search: None
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/304 using merged images, e.g. merging camera image with stored images
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A vehicle device includes a video acquisition unit configured to acquire videos captured by cameras configured to capture surroundings of a vehicle; a light emission controller configured to control light emission of a light beam pattern which is emitted by light emitting devices and with which the ground is irradiated at certain positions with respect to the vehicle; an image processing unit 24 configured to generate, using the videos acquired, a bird'"'"'s-eye view video including the light beam pattern; a superimposing image generation unit configured to generate a superimposing image at a position corresponding to the light beam pattern included in the bird'"'"'s-eye view video; and an image determination processing unit 27 configured to detect, in the bird'"'"'s-eye view video, an overlapping portion of the superimposing image and an image of the light beam pattern and determine whether camera parameters for generating the bird'"'"'s-eye view video are accurately calibrated.

Claims

1. A vehicle device, comprising:

a video acquisition unit configured to acquire videos captured by cameras configured to capture surroundings of a vehicle;

a light emission controller configured to control light emission of a light beam pattern which is emitted by light emitting devices and with which the ground or a floor surface is irradiated at predetermined positions with respect to the vehicle;

an image processing unit configured to generate, using the videos acquired by the video acquisition unit, a bird'"'"'s-eye view video including the light beam pattern the light emission of which is controlled by the light emission controller;

a superimposing image generation unit configured to generate a superimposing image at a position corresponding to the light beam pattern included in the bird'"'"'s-eye view video generated by the image processing unit; and

an image determination processing unit configured to detect, in the bird'"'"'s-eye view video on which the superimposing image generated by the superimposing image generation unit is superimposed, an overlapping portion of the superimposing image and an image of the light beam pattern to determine whether camera parameters for generating the bird'"'"'s-eye view video are accurately calibrated.

2. The vehicle device according to claim 1, wherein the light emission controller is further configured to perform a control such that a color of the light emission of the light beam pattern becomes a complementary color of a color of the ground or a color of the floor surface, or becomes a color similar to the complementary color.

3. The vehicle device according to claim 1, wherein the light emission controller is further configured to control an intensity of the light emission of the light beam pattern based on the bird'"'"'s-eye view video generated by the image processing unit.

4. The vehicle device according to claim 1, wherein the light emission controller is further configured to acquire information about a brightness of the surroundings of the vehicle to control the intensity of the light emission of the light beam pattern based on the information about the brightness of the surroundings of the vehicle.

5. The vehicle device according to claim 1, wherein the superimposing image generation unit is further configured to determine a color and a transparency level of the superimposing image based on the color of the light emission of the light beam pattern included in the bird'"'"'s-eye view video generated by the image processing unit.

6. A calibration result determination method that determines a calibration result of camera parameters for generating a bird'"'"'s-eye view video of a vehicle, the calibration result determination method comprising:

controlling light emission of a light beam pattern which is emitted by light emitting devices and with which the ground or a floor surface is irradiated at predetermined positions with respect to the vehicle;

generating, using videos captured by cameras configured to capture surroundings of the vehicle, a bird'"'"'s-eye view video including the light beam pattern the light emission of which is controlled at the controlling;

generating a superimposing image at a position corresponding to the light beam pattern included in the bird'"'"'s-eye view video generated at the generating of the bird'"'"'s-eye view video; and

detecting, in the bird'"'"'s-eye view video on which the superimposing image generated at the generating of the superimposing image is superimposed, an overlapping portion of the superimposing image and an image of the light beam pattern to determine whether the camera parameters for generating the bird'"'"'s-eye view video are accurately calibrated.

7. A non-transitory storage medium that stores a computer program that causes a computer mounted on a vehicle to perform processing comprising:

controlling light emission of a light beam pattern which is emitted by light emitting devices and with which the ground or a floor surface is irradiated at predetermined positions with respect to the vehicle;

generating, using videos captured by cameras configured to capture surroundings of the vehicle, a bird'"'"'s-eye view video including the light beam pattern the light emission of which is controlled at the controlling;

generating a superimposing image at a position corresponding to the light beam pattern included in the bird'"'"'s-eye view video generated at the generating of the bird'"'"'s-eye view video; and

detecting, in the bird'"'"'s-eye view video on which the superimposing image generated at the generating of the superimposing image is superimposed, an overlapping portion of the superimposing image and an image of the light beam pattern to determine whether the camera parameters for generating the bird'"'"'s-eye view video are accurately calibrated.