METHOD AND APPARATUS FOR DETERMINING ABNORMAL OBJECT

• US 20180150701A1
• Filed: 05/01/2017
• Published: 05/31/2018
• Est. Priority Date: 11/29/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: KANG, Hyoa, KIM, Changhyun
• Granted Patent: US 10,546,201 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/126 Evolutionary algorithms, e.g. genetic algorithms or genetic programming
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06V 10/451 with interaction between the filter responses, e.g. cortical complex cells
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Disclosed is a method and apparatus for determining an abnormal object, the method including selecting a candidate object from target objects extracted from a two-dimensional (2D) image of a front view captured from a host vehicle, generating a three-dimensional (3D) model of the candidate object, determining, based on the 3D model, whether the candidate object corresponds to an abnormal object that interferes with driving of the host vehicle, and outputting the abnormal object, in response to the candidate object corresponding to the abnormal object.

Claims

1. A method of determining an abnormal object, the method comprising:

selecting a candidate object from target objects extracted from a two-dimensional (2D) image of a front view captured from a host vehicle;

generating a three-dimensional (3D) model of the candidate object;

determining, based on the 3D model, whether the candidate object corresponds to an abnormal object that interferes with driving of the host vehicle; and

outputting the abnormal object, in response to the candidate object corresponding to the abnormal object.

2. The method of claim 1, wherein in response to the target objects traveling in a same lane as the host vehicle, the selecting of the candidate object comprises selecting the candidate object from the target objects based on portions of the target objects not occluded by another vehicle in the 2D image.

3. The method of claim 2, wherein the selecting of the candidate object comprises selecting a target object of the target objects as the candidate object in response to a difference in portion of the target object not occluded by another vehicle in the 2D image and the portion of the other target objects that is not occluded by other vehicles being greater than or equal to a threshold.

4. The method of claim 2, wherein the selecting of the candidate object comprises selecting the candidate object from the target objects based on any one or any combination of a degree of curve of the lane, a distance from the host vehicle, and types of the nearby vehicles.

5. The method of claim 1, wherein the selecting of the candidate object comprises selecting the candidate object from the target objects using a neural network trained to output a candidate object in response to an input of the 2D image.

6. The method of claim 5, wherein the neural network is adaptively trained based the determining of whether the candidate object corresponds to the abnormal object.

7. The method of claim 1, wherein the selecting of the candidate object comprises selecting the candidate object from the target objects based on an abnormal score from a previous image captured earlier than the 2D image, and

the abnormal score determines whether a target object corresponds to the abnormal object.

8. The method of claim 1, wherein the selecting of the candidate object comprises selecting the candidate object from the target objects based on driving patterns of the target objects acquired from the 2D image and a previous image captured earlier than the 2D image.

9. The method of claim 1, wherein the determining of whether the candidate object corresponds to the abnormal object comprises determining whether the candidate object corresponds to the abnormal object based on any one or any combination of whether the candidate object is travelling in a lane identical or adjacent to that of the host vehicle, a moving direction, a moving speed, a distance from the host vehicle, and a size of the candidate object.

10. The method of claim 9, wherein the determining of whether the candidate object corresponds to the abnormal object comprises determining whether the candidate object corresponds to the abnormal object based any one or any combination of a shape of a road on which the host vehicle is travelling, a traffic accident frequency, a traffic condition, and a driving pattern of a user of the host vehicle.

11. The method of claim 1, wherein, in response to the candidate object being a person, the determining of whether the candidate object corresponds to the abnormal object comprises determining whether the candidate object corresponds to the abnormal object using any one or any combination of a moving direction, a moving speed, a distance from the host vehicle, a pose, a direction of a face, and a height of the person.

12. The method of claim 1, wherein the outputting of the abnormal object comprises any one or any combination of displaying the abnormal object on a display device in the host vehicle, replaying a warning sound indicating a presence of the abnormal object through a speaker device in the host vehicle, delivering tactile feedback on the abnormal object by a steering handle of the host vehicle, delivering information on the abnormal object by a vibration of a seat of the host vehicle, and providing the abnormal object to a driving system that controls or guides the driving of the host vehicle.

13. The method of claim 1, wherein the selecting of the candidate object comprises selecting a target object travelling in a pattern different than remaining target objects.

14. A non-transitory computer-readable medium storing instructions that, when executed by a processor, causes the processor to perform the method of claim 1.

15. An apparatus for determining an abnormal object, the apparatus comprising:

an image sensor configured to capture a front image from a host vehicle and to generate a two-dimensional (2D) image; and

a processor configured to

select a candidate object from target objects extracted from the 2D image,

generate a 3D model of the candidate object,

determine, based on the 3D model, whether the candidate object corresponds to an abnormal object that interferes with driving of the host vehicle, and

output the abnormal object, in response to the candidate object corresponding to the abnormal object.

16. The apparatus of claim 15, wherein, in response to the target objects traveling in a same lane as the host vehicle, the processor is further configured to select the candidate object from the target objects based on portions of the target objects not occluded by another vehicle in the 2D image.

17. The apparatus of claim 15, wherein the processor is further configured to select the candidate object from the target objects using a neural network trained to output a candidate object in response to an input of the 2D image.

18. The apparatus of claim 17, wherein the neural network is adaptively trained based a determination of whether the candidate object corresponds to the abnormal object.

19. The apparatus of claim 15, wherein the processor is further configured to select the candidate object from the target objects based on an abnormal score from a previous image captured earlier than the 2D image, and

the abnormal score determines whether a target object corresponds to the abnormal object.

20. The apparatus of claim 15, wherein the processor is further configured to determine whether the candidate object corresponds to the abnormal object based on any one or any combination of whether the candidate object is travelling in a lane identical or adjacent to that of the host vehicle, a moving direction, a moving speed, a distance from the host vehicle, and a size of the candidate object.

21. The apparatus of claim 20, wherein the processor is further configured to determine whether the candidate object corresponds to the abnormal object based on any one or any combination of a shape of a road on which the host vehicle is travelling, a traffic accident frequency, a traffic condition, and a driving pattern of a user of the host vehicle.