System and method for occluding contour detection

• US 10,067,509 B1
• Filed: 08/31/2017
• Issued: 09/04/2018
• Est. Priority Date: 03/10/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: Tusimple
• Inventors: Wang, Panqu, Chen, Pengfei, Huang, Zehua
• Primary Examiner(s): Brushaber, Frederick M
• Application Number:
  US15/693,446
• Publication Number:
  US 20180259970A1
• Time in Patent Office: 369 Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A system method for occluding contour detection using a fully convolutional neural network is disclosed. A particular embodiment includes: receiving an input image; producing a feature map from the input image by semantic segmentation; applying a Dense Upsampling Convolution (DUC) operation on the feature map to produce contour information of objects and object instances detected in the input image; and applying the contour information onto the input image.

Claims

1. A system comprising:

a data processor; and

an occluding object contour detection processing module, executable by the data processor, the occluding object contour detection processing module being configured to perform an occluding object contour detection operation using a fully convolutional neural network and dense upsampling convolution (DUC), the occluding object contour detection operation being configured to;

receive an input image;

produce a feature map from the input image by semantic segmentation;

apply a DUC operation on the feature map to produce contour information of objects and object instances detected in the input image; and

apply the contour information onto the input image.

2. The system of claim 1 wherein the semantic segmentation and DUC operation are machine learnable.

3. The system of claim 1 wherein the semantic segmentation is performed by a deep convolutional neural network trained on a dataset configured for a traffic environment.

4. The system of claim 1 wherein the DUC operation is configured operate within a fully convolutional network (FCN).

5. The system of claim 1 wherein the contour information is produced without the use of bounding boxes.

6. The system of claim 1 wherein the contour information is used by an autonomous control subsystem to control a vehicle without a driver.

7. A method comprising:

receiving an input image;

producing a feature map from the input image by semantic segmentation;

applying a Dense Upsampling Convolution (DUC) operation on the feature map to produce contour information of objects and object instances detected in the input image; and

applying the contour information onto the input image.

8. The method of claim 7 wherein the semantic segmentation and DUC operation are machine learnable.

9. The method of claim 7 wherein the semantic segmentation is performed by a deep convolutional neural network trained on a dataset configured for a traffic environment.

10. The method of claim 7 wherein the DUC operation is configured operate within a fully convolutional network (FCN).

11. The method of claim 7 wherein the contour information is produced without the use of bounding boxes.

12. The method of claim 7 wherein the contour information is used by an autonomous control subsystem to control a vehicle without a driver.

13. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive an input image;

produce a feature map from the input image by semantic segmentation;

apply a DUC operation on the feature map to produce contour information of objects and object instances detected in the input image; and

apply the contour information onto the input image.

14. The non-transitory machine-useable storage medium of claim 13 wherein the semantic segmentation and DUC operation are machine learnable.

15. The non-transitory machine-useable storage medium of claim 13 wherein the semantic segmentation is performed by a deep convolutional neural network trained on a dataset configured for a traffic environment.

16. The non-transitory machine-useable storage medium of claim 13 wherein the DUC operation is configured operate within a fully convolutional network (FCN).

17. The non-transitory machine-useable storage medium of claim 13 wherein the contour information is produced without the use of bounding boxes.

18. The non-transitory machine-useable storage medium of claim 13 wherein the contour information is used by an autonomous control subsystem to control a vehicle without a driver.