In-Vehicle Image Processing Device and Semiconductor Device

• US 20150145997A1
• Filed: 10/25/2014
• Published: 05/28/2015
• Est. Priority Date: 11/22/2013
• Status: Active Grant

• Current Assignee: Renesas Electronics Corporation
• Original Assignee: Renesas Electronics Corporation
• Inventors: TERASHIMA, Kazuaki
• Granted Patent: US 9,946,938 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/605 the adjustment being automatic
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

An in-vehicle image processing device capable of appropriately monitoring areas forward of, around, and rearward of a vehicle is provided at low cost. The device is for mounting on a vehicle and includes a camera, an image processing unit, and a determination unit. With a reflector provided in front of the camera, the camera can image, for display in a frame at a time, a first area forward of the vehicle and a second area, e.g. an area around the vehicle. In the image processing unit supplied with such image data from the camera, either the first-area image or the second-area image is appropriately processed whereas image processing is omitted for the other image. Alternatively, both images are subjected to a same image processing. The determination unit supplied with vehicle speed information supplies appropriate control instruction information based on the current vehicle speed to the image processing unit.

Claims

1. An in-vehicle image processing device for mounting on a vehicle, comprising a camera, an image recognition unit and a determination unit,

wherein the determination unit is configured to be able to receive speed information on the vehicle and supply control instruction information based on the speed information to the image recognition unit, and

wherein the image recognition unit is configured to be able to:

receive image data from the camera, the image data including image data for a first image area to show an image of an area forward of the vehicle and image data for a second image area to show an image of an area in a different direction, the first image area and the second image area both being simultaneously included in a frame;

subject the image data for both the first image area and the second image area to image processing based on the control instruction information; and

supply results of the image processing to the determination unit.

2. The in-vehicle image processing device according to claim 1,

wherein the second image area shows an image of an area around the vehicle, and

wherein the determination unit is configured to be able to supply control instruction information to the image recognition unit, the control instruction information being for, when the speed information indicates that the vehicle is traveling at high speed, having the image data for the first image area processed in a manner suitable for a distant view while having image processing omitted for the image data for the second image area and being for, when the speed information indicates that the vehicle is traveling at low speed, having the image data for the second image area processed in a manner suitable for a near view while having image processing omitted for the image data for the first image area.

3. The in-vehicle image processing device according to claim 2,

wherein the determination unit is configured to be able to supply control instruction information to the image recognition unit, the control instruction information being for having both the image data for the first image area and the image data for the second image area processed in a manner suitable for an intermediate distance view relative to image processing suitable for a distant view and image processing suitable for a near view.

4. The in-vehicle image processing device according to claim 1,

wherein the image recognition unit further has a function for calculating, based on the image data, speed information on the vehicle and supplying the speed information to the determination unit.

5. The in-vehicle image processing device according to claim 1, wherein the camera is used as a first camera and the image data is processed as first image data, the in-vehicle image processing device further comprising a second camera which, along with the first camera, makes up a pair of right and left cameras,

wherein the second camera is configured to be able to supply second image data to the image recognition unit, the second image data representing an image of an area forward of the vehicle to be shown in a frame, and

wherein the image recognition unit is configured to be able to perform image recognition processing based on a parallax between the image data for the first image area out of the first image data and the second image data.

6. The in-vehicle image processing device according to claim 5,

wherein the second image data includes image data for a third image area to show an image of an area forward of the vehicle and image data for a fourth image area to show an image of an area around the vehicle, the third image area and the fourth image area being simultaneously included in a frame,

wherein the image recognition unit is configured to be able to perform image recognition processing based on a parallax between the image data for the first image area included in the first image data and the image data for the third image area included in the second image data, and

wherein the determination unit is configured to be able to supply control instruction information to the image recognition unit, the control instruction information being for, when the speed information indicates that the vehicle is traveling at high speed, having the image data for the first image area and the image data for the third image area processed in a manner suitable for a distant view while having image processing omitted for the image data for the second image area and the image data for the fourth image area and being for, when the speed information indicates that the vehicle is traveling at low speed, having the image data for the second image area and the image data for the fourth image area processed in a manner suitable for a near view while having image processing omitted for the image data for the first image area and the image data for the third image area.

7. The in-vehicle image processing device according to claim 1,

wherein the image recognition unit further has a function to convert, out of the image data, image data for each image area into image data for each image area based on a prescribed viewpoint and synthesize an image using the image data for each image area based on the prescribed viewpoint.

8. The in-vehicle image processing device according to claim 7,

wherein the image recognition unit synthesizes an image of a spot, image data on which is included in the image data for plural image areas, using the image data for plural image areas in a mutually complementary manner based on parallaxes therebetween while, for a spot, no image data on which is included in the image data for any of the plural image areas, generating an image by presumption based on image data on a past frame or based on image data for an edge portion of the plural image areas or outputting prescribed image data for indicating that image data on the spot cannot be generated by presumption.

9. A semiconductor device comprising an image recognition unit and a determination unit, the semiconductor device being capable of, by being coupled with a camera for mounting on a vehicle, configuring an in-vehicle image processing device,

wherein the determination unit is configured to be able to receive speed information on the vehicle and supply control instruction information based on the speed information to the image recognition unit, and

wherein the image recognition unit is configured to be able to receive image data from the camera, the image data including image data for a first image area to show an image of an area forward of the vehicle and image data for a second image area to show an image of an area in a different direction, the first image area and the second image area both being simultaneously included in a frame;

subject the image data for both the first image area and the second image area to image processing based on the control instruction information; and

supply results of the image processing to the determination unit.

10. The semiconductor device according to claim 9,

wherein the second image area shows an image of an area around the vehicle, and

wherein the determination unit is configured to be able to supply control instruction information to the image recognition unit, the control instruction information being for, when the speed information indicates that the vehicle is traveling at high speed, having the image data for the first image area processed in a manner suitable for a distant view while having image processing omitted for the image data for the second image area and being for, when the speed information indicates that the vehicle is traveling at low speed, having the image data for the second image area processed in a manner suitable for a near view while having image processing omitted for the image data for the first image area.

11. The semiconductor device according to claim 10,

wherein the determination unit is configured to be able to supply control instruction information to the image recognition unit, the control instruction information being for having both the image data for the first image area and the image data for the second image area processed in a manner suitable for an intermediate distance view relative to image processing suitable for a distant view and image processing suitable for a near view.

12. The semiconductor device according to claim 9,

wherein the image recognition unit further has a function for calculating, based on the image data, speed information on the vehicle and supplying the speed information to the determination unit.

13. The semiconductor device according to claim 9, wherein the camera is used as a first camera and the image data is processed as first image data, the in-vehicle image processing device further comprising a second camera which, along with the first camera, makes up a pair of right and left cameras,

wherein the second camera is configured to be able to supply second image data to the image recognition unit of the semiconductor device, the second image data representing an image of an area forward of the vehicle to be shown in a frame, and

wherein the image recognition unit is configured to be able to perform image recognition processing based on a parallax between the image data for the first image area out of the first image data and the second image data.

14. The semiconductor device according to claim 13,

wherein the second image data includes image data for a third image area to show an image of an area forward of the vehicle and image data for a fourth image area to show an image of an area around the vehicle, the third image area and the fourth image area being simultaneously included in a frame,

wherein the image recognition unit is configured to be able to perform image recognition processing based on a parallax between the image data for the first image area included in the first image data and the image data for the third image area included in the second image data, and

wherein the determination unit is configured to be able to supply control instruction information to the image recognition unit, the control instruction information being for, when the speed information indicates that the vehicle is traveling at high speed, having the image data for the first image area and the image data for the third image area processed in a manner suitable for a distant view while having image processing omitted for the image data for the second image area and the image data for the fourth image area and being for, when the speed information indicates that the vehicle is traveling at low speed, having the image data for the second image area and the image data for the fourth image area processed in a manner suitable for a near view while having image processing omitted for the image data for the first image area and the image data for the third image area.

15. The semiconductor device according to claim 9,

wherein the image recognition unit further has a function to convert, out of the image data, image data for each image area into image data for each image area based on a prescribed viewpoint and synthesize an image using the image data for each image area based on the prescribed viewpoint.

16. The semiconductor device according to claim 15,

wherein the image recognition unit synthesizes an image of a spot, image data on which is included in the image data for plural image areas, using the image data for plural image areas in a mutually complementary manner based on parallaxes therebetween while, for a spot, no image data on which is included in the image data for any of the plural image areas, generating an image by presumption based on image data on a past frame or based on image data for an edge portion of the plural image areas or outputting prescribed image data for indicating that image data on the spot cannot be generated by presumption.

17. The semiconductor device according to claim 9 formed over a single semiconductor substrate.