Image processing apparatus, image processing method, and vehicle control system to determine the presence of an object from an image of a peripheral area of a moving body

• US 10,769,951 B2
• Filed: 08/30/2016
• Issued: 09/08/2020
• Est. Priority Date: 09/30/2015
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Hirasawa, Yasutaka, Oyaizu, Hideki, Kondo, Yuhi, Aoki, Suguru, Akama, Taketo
• Primary Examiner(s): Refai, Ramsey
• Application Number:
  US15/761,878
• Publication Number:
  US 20180301032A1
• Time in Patent Office: 1,470 Days
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An image acquisition unit 341-1 acquires a polarization image and a non-polarization image indicating a peripheral area of a moving body, such as the peripheral area of a vehicle. A discrimination information generation unit 342-1 uses the polarization image acquired by the image acquisition unit 341-1 and generates analysis object discrimination information indicating a road surface or the like. An image analysis unit 344-1 uses an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit 342-1 with respect to the non-polarization image acquired by the image acquisition unit 341-1, and performs a discrimination of an object, such as an obstacle on the road surface. It is possible to efficiently perform a determination of the presence of the object from the non-polarization image of the peripheral area of the moving body.

Claims

1. An image processing apparatus comprising:

a discrimination information generation unit configured to generate analysis object discrimination information by using a polarization image indicating a peripheral area of a moving body; and

an image analysis unit configured to perform discrimination of an object by using an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit to a non-polarization image indicating the peripheral area of the moving body, wherein

the discrimination information generation unit generates, as the analysis object discrimination information, information related to a braking distance of the moving body in the peripheral area of the moving body by using the polarization image, and

the image analysis unit sets an area within the braking distance as the image analysis area.

2. The image processing apparatus according to claim 1, wherein

the discrimination information generation unit generates the analysis object discrimination information indicating a road surface in the peripheral area of the moving body on the basis of the polarization image, and

the image analysis unit sets the image analysis area in accordance with an area of the road surface.

3. The image processing apparatus according to claim 2, wherein

the discrimination information generation unit sets, to an area of the road surface, a continuous image area in which a direction of a normal line calculated from the polarization image is a vertical direction of the moving body.

4. The image processing apparatus according to claim 1, wherein,

on the basis of a distance in the non-polarization image and the braking distance calculated from a roughness on a traveling surface and a moving speed, the discrimination information generation unit generates, as the analysis object discrimination information, information indicating an area within the braking distance in the non-polarization image.

5. The image processing apparatus according to claim 1, wherein

the image analysis unit is provided in plurality, and

the plurality of image analysis units share an area in which the non-polarization image is divided on the basis of the analysis object discrimination information and perform discrimination of the object.

6. The image processing apparatus according to claim 5, wherein

the analysis object discrimination information is information related to the braking distance of the moving body in the peripheral area of the moving body and information indicating the road surface in the peripheral area of the moving body, and

the plurality of image analysis units perform discrimination of the object in an area within the braking distance by using many image analysis units as compared with other image areas.

7. The image processing apparatus according to claim 1, wherein

an image transfer unit that performs a transfer of the non-polarization image is provided,

the image analysis unit is connected to the image transfer unit via a communication network, and

the image transfer unit controls a transfer of the non-polarization image to the image analysis unit on the basis of the analysis object discrimination information generated by the discrimination information generation unit.

8. The image processing apparatus according to claim 7, wherein

the image transfer unit transfers an image of the image analysis area.

9. The image processing apparatus according to claim 7, wherein

the discrimination information generation unit generates the analysis object discrimination information by using the polarization image in each capturing direction, and

the image transfer unit selects the non-polarization image transferred to the image analysis unit on the basis of the analysis object discrimination information generated by the discrimination information generation unit and transfers the selected non-polarization image along with the analysis object discrimination information.

10. The image processing apparatus according to claim 7, wherein

the discrimination information generation unit generates the analysis object discrimination information by using the polarization image in each capturing direction, and

the image transfer unit selects the non-polarization image transferred to the image analysis unit on the basis of the analysis object discrimination information generated by the discrimination information generation unit and transfers an image of the image analysis area in the selected non-polarization image along with the analysis object discrimination information.

11. The image processing apparatus according to claim 7, wherein

an image analysis control unit and the plurality of image analysis units are provided, and

the image analysis control unit generates image analysis control information to control that how discrimination of the object to the non-polarization image transferred to the image analysis unit from the image transfer unit is performed by the plurality of image analysis units on the basis of the analysis object discrimination information generated by the discrimination information generation unit and transmits the image analysis control information to the image analysis unit along with the non-polarization image.

12. An image processing method comprising:

generating analysis object discrimination information by a discrimination information generation unit by using a polarization image indicating a peripheral area of a moving body; and

performing discrimination of an object by an image analysis unit by using an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit to a non-polarization image indicating the peripheral area of the moving body, wherein generating analysis object discrimination information includes generating, as the analysis object discrimination information, information related to a braking distance of the moving body in the peripheral area of the moving body by using the polarization image, further including setting, by the image analysis unit, an area within the braking distance as the image analysis area.

13. A vehicle control system comprising:

a discrimination information generation unit configured to generate analysis object discrimination information by using a polarization image indicating a peripheral area of a vehicle;

an image analysis unit configured to perform discrimination of an object by using an image of an image analysis area set on the basis of the analysis object discrimination information generated by the discrimination information generation unit to a non-polarization image indicating the peripheral area of the vehicle; and

a driving system control unit configured to perform control of the vehicle on the basis of a discrimination result of the object in the image analysis unit, wherein

the discrimination information generation unit generates, as the analysis object discrimination information, information related to a braking distance of the vehicle in the peripheral area of the vehicle by using the polarization image, and

the image analysis unit sets an area within the braking distance as the image analysis area.