System and method for semantic segmentation using hybrid dilated convolution (HDC)

• US 10,147,193 B2
• Filed: 03/10/2017
• Issued: 12/04/2018
• Est. Priority Date: 03/10/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: Tusimple
• Inventors: Huang, Zehua, Chen, Pengfei, Wang, Panqu
• Primary Examiner(s): Chakraborty, Raj
• Application Number:
  US15/456,294
• Publication Number:
  US 20180260956A1
• Time in Patent Office: 634 Days
• Field of Search: 382156
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 30/10 Path keeping cruise control for automatically following a preceding vehicle B60W30/165
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

A system and method for semantic segmentation using hybrid dilated convolution (HDC) are disclosed. A particular embodiment includes: receiving an input image; producing a feature map from the input image; performing a convolution operation on the feature map and producing multiple convolution layers; grouping the multiple convolution layers into a plurality of groups; applying different dilation rates for different convolution layers in a single group of the plurality of groups; and applying a same dilation rate setting across all groups of the plurality of groups.

Claims

1. A system comprising:

a data processor; and

an image processing module, executable by the data processor, the image processing module being configured to perform semantic segmentation using a hybrid dilated convolution (HDC) operation, the HDC operation being configured to;

receive an input image;

produce a feature map from the input image;

perform a convolution operation on the feature map and produce multiple convolution layers;

group the multiple convolution layers into a plurality of groups;

apply different dilation rates for different convolution layers in a single group of the plurality of groups; and

apply a same dilation rate setting across all groups of the plurality of groups

wherein the HDC operation is used by an autonomous control subsystem to control a vehicle without a driver.

2. The system of claim 1 wherein the HDC operation is configured to assign increasing dilation rates to the multiple convolution layers.

3. The system of claim 1 wherein the first dilation rate and the second dilation rate do not have a common factor relationship.

4. The system of claim 1 wherein the HDC operation is configured operate within a fully convolutional network (FCN).

5. The system of claim 1 wherein the HDC operation is performed at an original resolution, thereby enabling pixel-level decoding.

6. A method comprising:

receiving an input image;

producing a feature map from the input image;

performing a convolution operation on the feature map and producing multiple convolution layers;

grouping the multiple convolution layers into a plurality of groups;

applying different dilation rates for different convolution layers in a single group of the plurality of groups; and

applying a same dilation rate setting across all groups of the plurality of groups

wherein the method is used by an autonomous control subsystem to control a vehicle without a driver.

7. The method of claim 6 including assigning increasing dilation rates to the multiple convolution layers.

8. The method of claim 6 wherein the first dilation rate and the second dilation rate do not have a common factor relationship.

9. The method of claim 6 wherein the method operates within a fully convolutional network (FCN).

10. The method of claim 6 wherein the method is performed at an original resolution, thereby enabling pixel-level decoding.

11. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive an input image;

produce a feature map from the input image;

perform a convolution operation on the feature map and produce multiple convolution layers;

group the multiple convolution layers into a plurality of groups;

apply different dilation rates for different convolution layers in a single group of the plurality of groups; and

apply a same dilation rate setting across all groups of the plurality of groups

wherein the instructions are further configured to be used with an autonomous control subsystem to control a vehicle without a driver.

12. The non-transitory machine-useable storage medium of claim 11 wherein the instructions are further configured to assign increasing dilation rates to the multiple convolution layers.

13. The non-transitory machine-useable storage medium of claim 11 wherein the first dilation rate and the second dilation rate do not have a common factor relationship.

14. The non-transitory machine-useable storage medium of claim 11 wherein the instructions are further configured to operate within a fully convolutional network (FCN).

15. The non-transitory machine-useable storage medium of claim 11 wherein the instructions are further configured to perform at an original resolution, thereby enabling pixel-level decoding.