Image processor and monitoring system

• US 20040184638A1
• Filed: 03/15/2004
• Published: 09/23/2004
• Est. Priority Date: 04/28/2000
• Status: Active Grant

• Current Assignee: Hirofumi Ishii, Kunio Nobori, Masamichi Nakagawa, Shusaku Okamoto
• Original Assignee: Hirofumi Ishii, Kunio Nobori, Masamichi Nakagawa, Shusaku Okamoto
• Inventors: Nobori, Kunio, Ishii, Hirofumi, Nakagawa, Masamichi, Okamoto, Shusaku
• Granted Patent: US 7,714,887 B2
• Time in Patent Office: Days
• US Class Current: 382/104
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/20228 Disparity calculation for image-based rendering
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The image processor of the present invention generates a more natural synthesized image from camera images taken with a plurality of cameras capturing the surroundings of a vehicle. A parameter storage section stores a plurality of image synthesis parameter groups representing the correspondence between the camera images and the synthesized image and having different spatial or temporal resolution relations. A parameter selection section selects the image synthesis parameter group according to the output of a vehicle motion detection section for detecting the motion of the vehicle such as the driving speed and direction. An image synthesis section generates the synthesized image from the camera images according to the selected image synthesis parameter group.

Claims

1. An image processor for receiving camera images taken with a plurality of cameras capturing the surroundings of a vehicle and generating a synthesized image from the camera images, the image processor comprising:

a parameter generation section constructed to be able to generate a plurality of image synthesis parameter groups each representing the correspondence between the camera images and the synthesized image and having different spatial or temporal resolution relations,

wherein the synthesized image is generated from the camera images according to the image synthesis parameter group generated by the parameter generation section, and

the parameter generation section switches the image synthesis parameter group to be generated according to an output of a vehicle motion detection section for detecting the motion of the vehicle.

2. The image processor of claim 1, wherein the parameter generation section comprises:

a parameter storage section for storing the plurality of image synthesis parameter groups, and

a parameter selection section for selecting at least one among the plurality of image synthesis parameter groups stored in the parameter storage section according to the output of the vehicle motion detection section.

3. The image processor of claim 1, wherein the camera images are interlaced images, and the plurality of image synthesis parameter groups include at least a frame-base image synthesis parameter group and a field-base image synthesis parameter group.

4. The image processor of claim 3, wherein the parameter generation section generates the field-base image synthesis parameter group when the motion of the vehicle detected by the vehicle motion detection section is relatively fast, and generates the frame-base image synthesis parameter group when the motion of the vehicle is relatively slow.

5. The image processor of claim 1, wherein the vehicle motion detection section detects the motion of the vehicle from the camera images.

6. The image processor of claim 1, wherein the plurality of cameras are constructed to be able to switch a capture pattern according to an input switch signal, and

the parameter generation section sends the switch signal to the cameras, together with generating the image synthesis parameter group, according to the output of the vehicle motion detection section, to switch the capture patterns of the cameras.

7. The image processor of claim 6, wherein the parameter generation section selects the image synthesis parameter group and switches the capture patterns of the cameras according to an output of a vehicle status detection section for detecting brightness of the surroundings of the vehicle, in addition to the output of the vehicle motion detection section.

8. The image processor of claim 1, wherein the parameter generation section generates the image synthesis parameter group according to an output of a vehicle status detection section for detecting the vehicle status such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle, in addition to the output of the vehicle motion detection section.

9. A monitoring system comprising:

a plurality of cameras for capturing the surroundings of a vehicle;

an image processing section for receiving camera images from the plurality of cameras and generating a synthesized image from the camera images; and

a display section for displaying the synthesized image generated by the image processing section,

wherein the image processing section comprises a parameter generation section constructed to be able to generate a plurality of image synthesis parameter groups each representing the correspondence between the camera images and the synthesized image and having different spatial or temporal resolution relations,

the synthesized image is generated from the camera images according to the image synthesis parameter group generated by the parameter generation section, and

the parameter generation section generates at least one of the plurality of image synthesis parameter groups according to an output of a vehicle motion detection section for detecting the motion of the vehicle.

10. An image processor for receiving camera images taken with a plurality of cameras capturing the surroundings of a vehicle and generating a synthesized image from the camera images, the image processor comprising:

a parameter storage section for storing a plurality of sets of an image synthesis parameter group representing the correspondence between the camera images and the synthesized image and a filter parameter group corresponding to the image synthesis parameter group;

a parameter selection section for selecting at least one among the plurality of sets of the image synthesis parameter group and the filter parameter group stored by the parameter storage section according to an output of a vehicle motion detection section for detecting the motion of the vehicle and an output of a vehicle status detection section for detecting the status of the vehicle such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle; and

a filtering section for performing frequency band limitation filtering for the camera images according to the filter parameter group of the set selected by the parameter selection section,

wherein the synthesized image is generated from the camera images filtered by the filtering section according to the image synthesis parameter group of the set selected by the parameter selection section.

11. The image processor of claim 10, wherein the filter parameter group includes filtering setting data for each pixel position of the camera image.

12. A monitoring system comprising:

a plurality of cameras for capturing the surroundings of a vehicle;

an image processing section for receiving camera images from the plurality of cameras and generating a synthesized image from the camera images; and

a display section for displaying the synthesized image generated by the image processing section,

wherein the image processing section comprises;

a parameter storage section for storing a plurality of sets of an image synthesis parameter group representing the correspondence between the camera images and the synthesized image and a filter parameter group corresponding to the image synthesis parameter group;

a parameter selection section for selecting at least one among the plurality of sets of the image synthesis parameter group and the filter parameter group stored by the parameter storage section according to an output of a vehicle motion detection section for detecting the motion of the vehicle and an output of a vehicle status detection section for detecting the status of the vehicle such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle; and

a filtering section for performing frequency band limitation filtering for the camera images according to the filter parameter group of the set selected by the parameter selection section,

wherein the synthesized image is generated from the camera images filtered by the filtering section according to the image synthesis parameter group of the set selected by the parameter selection section.

13. An image processor for receiving camera images taken with a plurality of cameras capturing the surroundings of a vehicle and generating a synthesized image from the camera images, the image processor comprising:

a brightness correction parameter calculation section for calculating brightness correction parameters for correcting the brightness and tint of the camera images; and

a brightness correction section for correcting the brightness and tint of the camera images using the brightness correction parameters calculated by the brightness correction parameter calculation section,

wherein the synthesized image is generated from the plurality of camera images subjected to brightness correction by the brightness correction section according to an image synthesis parameter group representing the correspondence between the camera images and the synthesized image,

the image synthesis parameter group includes overlap area data for an overlap area on the synthesized image in which coverages of the plurality of cameras overlap, the overlap area data indicating coordinates of pixels in camera images corresponding to the overlap area, and

the brightness correction parameter calculation section receives the overlap area data and calculates the brightness correction parameters using brightness and tint data for the pixels in the camera images corresponding to the overlap area indicated by the overlap area data.

14. The image processor of claim 13, wherein the brightness correction parameter calculation section performs statistical processing on the brightness in the overlap area for the camera images corresponding to the overlap area, and calculates the brightness correction parameters based on the processing results.

15. The image processor of claim 13, wherein, when a plurality of overlap areas exist, the brightness correction parameter calculation section sets priorities to the overlap areas to be considered during the calculation of the brightness correction parameters according to an output of a vehicle motion detection section for detecting the motion of the vehicle.

16. The image processor of claim 13, wherein the brightness correction section is incorporated in the cameras.

17. A monitoring system comprising:

a plurality of cameras for capturing the surroundings of a vehicle;

an image processing section for receiving camera images from the plurality of cameras and generating a synthesized image from the camera images; and

a display section for displaying the synthesized image generated by the image processing section,

wherein the image processing section comprises;

a brightness correction parameter calculation section for calculating brightness correction parameters for correcting the brightness and tint of the camera images; and

a brightness correction section for correcting the brightness and tint of the camera images using the brightness correction parameters calculated by the brightness correction parameter calculation section,

wherein the synthesized image is generated from the plurality of camera images subjected to brightness correction by the brightness correction section according to an image synthesis parameter group representing the correspondence between the camera images and the synthesized image,

the image synthesis parameter group includes overlap area data for an overlap area on the synthesized image in which coverages of the plurality of cameras overlap, the overlap area data indicating coordinates of pixels in camera images corresponding to the overlap area, and

the brightness correction parameter calculation section receives the overlap area data and calculates the brightness correction parameters using brightness and tint data for the pixels in the camera images corresponding to the overlap area indicated by the overlap area data.

18. An image processor for receiving camera images taken with a plurality of cameras capturing the surroundings of a vehicle and generating a synthesized image from the camera images,

wherein, in an overlap area in which coverages of a plurality of cameras overlap on the synthesized image, a camera image used for generation of the synthesized image is selected among camera images from the plurality of cameras according to an output of a vehicle motion detection section for detecting the motion of the vehicle or an output of a vehicle status detection section for detecting the status of the vehicle such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle.

19. An image processor for receiving camera images taken with a plurality of cameras capturing the surroundings of a vehicle and generating a synthesized image from the camera images,

wherein, in an overlap area in which coverages of a plurality of cameras overlap on the synthesized image, weights to camera images from the plurality of cameras are set according to an output of a vehicle motion detection section for detecting the motion of the vehicle or an output of a vehicle status detection section for detecting the status of the vehicle such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle.

20. The image processor of claim 19, wherein the image processor includes:

an image synthesis parameter group associating pixels in the synthesized image with pixels in the camera images; and

a table representing the correspondence between a weight reference number and a combination of sets of weighting information, wherein a portion of the image synthesis parameter group corresponding to the overlap area holds any of the weight reference numbers shown in the table.

21. A monitoring system comprising:

a plurality of cameras for capturing the surroundings of a vehicle;

an image processing section for receiving camera images from the plurality of cameras and generating a synthesized image from the camera images; and

a display section for displaying the synthesized image generated by the image processing section,

wherein, in an overlap area in which coverages of a plurality of cameras overlap on the synthesized image, a camera image used for generation of the synthesized image is selected among camera images from the plurality of cameras according to an output of a vehicle motion detection section for detecting the motion of the vehicle or an output of a vehicle status detection section for detecting the status of the vehicle such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle.

22. A monitoring system comprising:

a plurality of cameras for capturing the surroundings of a vehicle;

an image processing section for receiving camera images from the plurality of cameras and generating a synthesized image from the camera images; and

a display section for displaying the synthesized image generated by the image processing section,

wherein, in an overlap area in which coverages of a plurality of cameras overlap on the synthesized image, weights to camera images from the plurality of cameras are set according to an output of a vehicle motion detection section for detecting the motion of the vehicle or an output of a vehicle status detection section for detecting the status of the vehicle such as an operation by a driver of the vehicle or whether or not an obstacle exists in the surroundings of the vehicle.

23. An image processor for receiving camera images taken with a plurality of cameras capturing the surroundings of a vehicle and generating a synthesized image from the camera images,

wherein the image processor includes an image synthesis parameter group associating pixels in the synthesized image with pixels in the camera images, and

in an overlap area in which coverages of the plurality of cameras overlap on the synthesized image, the image synthesis parameter group includes weights set to camera images from the plurality of cameras represented by a dithering method.