STIXEL ESTIMATION METHODS AND SYSTEMS

• US 20170147888A1
• Filed: 11/16/2016
• Published: 05/25/2017
• Est. Priority Date: 11/20/2015
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: ZENG, SHUQING
• Granted Patent: US 10,482,331 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G06F 18/2414 Smoothing the distance, e.g. radial basis function networks [RBFN]
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

Methods and systems are provided for detecting an object. In one embodiment, a method includes: receiving, by a processor, image data from an image sensor; receiving, by a processor, radar data from a radar system; processing, by the processor, the image data from the image sensor and the radar data from the radar system using a deep learning method; and detecting, by the processor, an object based on the processing.

Claims

1. A method for detecting an object, comprising:

receiving, by a processor, image data from an image sensor;

receiving, by a processor, radar data from a radar system;

processing, by the processor, the image data from the image sensor and the radar data from the radar system using a deep learning method; and

detecting, by the processor, an object based on the processing.

2. The method of claim 1, wherein the processing comprises processing the image data to determine one or more stixels, and wherein the deep learning method uses the one or more stixels.

3. The method of claim 2, wherein the processing comprises:

forming a region of interest (ROI) window of an image based on the image data; and

moving the ROI window to a plurality of locations, from a left side of the image location to a right side of the image, to sequentially determine a presence of a stixel at each location.

4. The method of claim 2, wherein the processing comprises:

forming a region of interest (ROI) window based on the image data;

determining whether a centerline of the ROI window includes a stixel using a convolution neural network; and

reporting a probability, a location, a height, and a class label when a stixel is determined.

5. The method of claim 1, wherein the processing comprises:

processing the radar data from the radar system to determine one or more presence vectors, and wherein the deep learning method uses the one or more presence vectors.

6. The method of claim 1, wherein the processing comprises fusing stixels of the image data and presence vectors of the radar data using the deep learning methods.

7. The method of claim 6, further comprising determining a velocity for each of the fused stixels and presence vectors, and wherein the detecting the object is based on the velocity of each of the fused stixels and presence vectors.

8. The method of claim 7, further comprising determining a displacement for each of the fused stixels and presence vectors, and wherein the detecting the object is based on the displacement of each of the fused stixels and presence vectors.

9. A system for detecting an object, comprising:

an image sensor that generates image data;

a radar system that generates radar data; and

a computer module that, by a processor, processes the image data and the radar data using a deep learning method, and detects an object based on the processing.

10. The system of claim 9, wherein the computer module processes the image data to determine one or more stixels, and wherein the deep learning method uses the one or more stixels.

11. The system of claim 10, wherein the computer module forms a region of interest (ROI) window of an image based on the image data;

and moves the ROI window to a plurality of locations, from a left side of the image location to a right side of the image, to sequentially determine a presence of a stixel at each location.

12. The system of claim 10, wherein the computer module forms a region of interest (ROI) window based on the image data, determines whether a centerline of the ROI window includes a stixel using a convolution neural network, and reports a probability, a location, a height, and a class label when a stixel is determined.

13. The system of claim 9, wherein the computer module processes the radar data from the radar system to determine one or more presence vectors, and the deep learning method uses the one or more presence vectors.

14. The system of claim 9, wherein the computer module fuses stixels of the image data and presence vectors of the radar data using the deep learning methods.

15. The system of claim 14, wherein the computer module determines a velocity for each of the fused stixels and presence vectors, and detects the object based on the velocity of each of the fused stixels and presence vectors.

16. The system of claim 15, wherein the computer module determines a displacement for each of the fused stixels and presence vectors, and detects the object based on the displacement of each of the fused stixels and presence vectors.

17. The system of claim 9, wherein the image sensor and the radar system are associated with a vehicle, and wherein the control module detects the object in proximity to the vehicle.