Detection of lane-splitting motorcycles

• US 9,598,076 B1
• Filed: 10/22/2015
• Issued: 03/21/2017
• Est. Priority Date: 10/22/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Jain, Jinesh J., Banvait, Harpreetsingh, Han, Kyu Jeong
• Primary Examiner(s): NGUYEN, TAN QUANG
• Application Number:
  US14/920,369
• Time in Patent Office: 516 Days
• Field of Search: 701/28, 701/301, 348/148, 340/435
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2710/18 Braking system
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 2720/106 Longitudinal acceleration
• CPC Class Codes: B60W 2754/00 Output or target parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 15/931 of land vehicles
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 3/802 Systems for determining direction or deviation from predetermined direction
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A controller for an autonomous vehicle received audio streams from one or more microphones and one or more image streams from one or more cameras. The audio streams are processed to identify audio features that potentially correspond to a motorcycle. The video streams are likewise processed to identify image objects that have movement and/or visual attributes corresponding to a motorcycle. The audio features and image objects are correlated. If a correlation is identified among the audio features and image objects, then a potentially lane-splitting motorcycle may be determined to be proximate. Actions may be taken to avoid collision with the lane-splitting motorcycle, such as avoiding lane changes, signaling longer, or performing lane changes more slowly.

Claims

1. A controller for an autonomous vehicle comprising:

one or more processing devices programmed to

receive one or more image streams;

receive one or more audio streams;

identify a plurality of vehicle images in the one or more image streams;

identify a plurality of audible features in the one or more audio streams; and

identify a potential lane-splitting motorcycle by correlating the plurality of vehicle images and the plurality of audible features.

2. The controller of claim 1, wherein the one or more processing devices are further programmed to receive environmental information from one or more of one or more cameras, RADAR sensors, LIDAR sensors, and SONAR sensors.

3. The controller of claim 2, wherein the one or more processing devices are further programmed to:

in response to identifying the potential lane-splitting motorcycle, avoid causing the autonomous vehicle to make lane changes.

4. The controller of claim 1, wherein the one or more processing devices are further programmed to:

identify one or more audible features in the one or more audio streams, the one or more audible features having a motorcycle-specific sound pattern;

identify among the plurality of vehicle images, motorcycle-specific image patterns;

attempt to correlate the one or more audible features and the one or more image patterns;

if the one or more audible features and the one or more image patterns are correlated, augment a confidence score that the potential lane-splitting motorcycle is a potential obstacle.

5. The controller of claim 4, wherein the one or more processing devices are programmed to attempt to correlate the one or more audible features and the one or more image patterns by inputting the one or more audible features and one or more image patterns to a machine learning algorithm.

6. The controller of claim 5, wherein the one or more processing devices are further programmed to identify among the plurality of vehicle images, motorcycle-specific image patterns comprises identifying objects exhibiting motorcycle-specific movement among the plurality of vehicle images.

7. The controller of claim 5, wherein the one or more processing devices are further programmed to identify the one or more audible features by filtering the one or more audio streams to obtain a filtered signal including the one or more audible features.

8. The controller of claim 1, further comprising a microphone array coupled thereto, the microphone array configure to generate the one or more audio streams.

9. The controller of claim 1, further comprising at least one microphone coupled configured to generate the one or more audio streams, the at least one microphone located within a cabin of the autonomous vehicle.

10. The controller of claim 1, further comprising one or more cameras configured to generate the one or more images streams, the one or more cameras coupled to the autonomous vehicle and oriented rearwardly relative to the autonomous vehicle.

11. A method for collision avoidance comprising:

receiving, by a controller of an autonomous vehicle, one or more image streams;

receiving, by the controller, one or more audio streams;

identifying, by the controller, a plurality of vehicle images in the one or more image streams;

identifying, by the controller, a plurality of audible features in the one or more audio streams; and

identifying, by the controller, a potential lane-splitting motorcycle by correlating the plurality of vehicle images and the plurality of audible features.

12. The method of claim 11, further comprising:

identifying, by the controller, a set of potential obstacles among the plurality of vehicle images, the set of potential obstacles including the potential lane splitting motorcycle;

evaluating, by the controller, possible collisions between the autonomous vehicle and the set of potential obstacles; and

activating, by the controller, at least one of a steering actuator, accelerator actuator, and brake actuator of the autonomous vehicle effective to avoid collisions with the set of potential obstacles.

13. The method of claim 12, further comprising:

in response to identifying the potential lane-splitting motorcycle, avoiding, by the controller, causing the autonomous vehicle to make lane changes.

14. The method of claim 11, further comprising:

(a) identifying, by the controller, one or more audible features in the one or more audio streams, the one or more audible features having motorcycle-specific sound patterns;

(b) identifying, by the controller, among the plurality of vehicle images, image patterns characteristic of a motorcycle;

(c) attempting, by the controller, to correlate the one or more audible features and the one or more image patterns;

in response to the attempt of (c) being successful, augmenting, by the controller, a confidence score that the potential lane-splitting motorcycle is a potential obstacle.

15. The method of claim 14, wherein attempting to correlate the one or more audible features and the one or more image patterns comprises inputting the one or more audible features and one or more image patterns to a machine learning algorithm.

16. The method of claim 15, wherein identifying among the plurality of vehicle images, motorcycle-specific image patterns comprises identifying objects exhibiting motorcycle-specific movement among the plurality of vehicle images.

17. The method of claim 15, wherein identifying the one or more audible features comprises filtering the one or more audio streams to obtain a filtered signal including the one or more audible features.

18. The method of claim 11, wherein receiving the one or more audio streams comprises receiving the one or more audio streams from a microphone array.

19. The method of claim 11, wherein receiving the one or more audio streams comprises receiving the one or more audio streams from least one microphone coupled located within a cabin of the autonomous vehicle.

20. The method of claim 11, wherein receiving the one or more images streams comprises receiving the one or more image streams from one or more cameras coupled to the autonomous vehicle and oriented rearwardly relative to the autonomous vehicle.