ACTIVATION ZERO-BYPASS AND WEIGHT PRUNING IN NEURAL NETWORKS FOR VEHICLE PERCEPTION SYSTEMS

• US 20200134459A1
• Filed: 10/25/2018
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zeng, Shuqing, Tong, Wei, Wang, Shige, Millett, Roman L.
• Granted Patent: US 11,281,974 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 2300/8046 for replacing a rear-view mirror system
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

In one example implementation according to aspects of the present disclosure, a computer-implemented method includes capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory. The method further includes dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory. The method further includes performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises performing an activation bypass for values below a first threshold, and performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold. The method further includes controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

Claims

1. A computer-implemented method for vehicle perception, the method comprising:

capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory;

dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory;

performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises;

performing an activation bypass for values below a first threshold, and

performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold; and

controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

2. The computer-implemented method of claim 1, wherein performing the weight pruning further comprises:

performing an analysis;

performing weight pruning based on the analysis; and

tuning the weight pruning.

3. The computer-implemented method of claim 2, wherein performing the weight pruning further comprises determining whether to perform additional compression.

4. The computer-implemented method of claim 3, wherein performing the weight pruning further comprises, responsive to determining to perform additional compressing, re-performing the weight pruning and the tuning.

5. The computer-implemented method of claim 3, wherein performing the weight pruning further comprises, responsive to determining not to perform additional compressing, performing a transformation.

6. The computer-implemented method of claim 1, wherein performing the activation bypass avoids performing a multiplication and accumulation (MAC) operation, and wherein the MAC operation is performed for values above the first threshold.

7. The computer-implemented method of claim 1, wherein the first threshold is determined based at least in part on a percent of operations pruned and a prediction error percentage.

8. The computer-implemented method of claim 1, wherein the accelerator further comprises a neural processing unit instruction fetcher, a dispatcher, a level 1 cache, and a level 2 cache.

9. The computer-implemented method of claim 1, wherein the accelerator is communicatively coupled to a host central processing unit via the memory, wherein the accelerator is a special purpose processor and wherein the host central processing unit is a general purpose processor.

10. The computer-implemented method of claim 9, wherein the host central processing unit receives the image data from the camera associated with the vehicle and stores the image data associated with the plurality of images to the memory.

11. A system comprising:

a memory comprising computer readable instructions; and

a processing device for executing the computer readable instructions for performing a method for vehicle perception, the method comprising;

capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory;

dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory;

performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises;

performing an activation bypass for values below a first threshold, and

performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold; and

controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.

12. The system of claim 11, wherein performing the weight pruning further comprises:

performing an analysis;

performing weight pruning based on the analysis; and

tuning the weight pruning.

13. The system of claim 12, wherein performing the weight pruning further comprises determining whether to perform additional compression.

14. The system of claim 13, wherein performing the weight pruning further comprises, responsive to determining to perform additional compressing, re-performing the weight pruning and the tuning.

15. The system of claim 13, wherein performing the weight pruning further comprises, responsive to determining not to perform additional compressing, performing a transformation.

16. The system of claim 11, wherein performing the activation bypass avoids performing a multiplication and accumulation (MAC) operation, and wherein the MAC operation is performed for values above the first threshold.

17. The system of claim 11, wherein the first threshold is determined based at least in part on a percent of operations pruned and a prediction error percentage.

18. The system of claim 11, wherein the accelerator further comprises a neural processing unit instruction fetcher, a dispatcher, a level 1 cache, and a level 2 cache.

19. The system of claim 11, wherein the accelerator is communicatively coupled to a host central processing unit via the memory, wherein the accelerator is a special purpose processor and wherein the host central processing unit is a general purpose processor, and wherein the host central processing unit receives the image data from the camera associated with the vehicle and stores the image data associated with the plurality of images to the memory.

20. A computer program product comprising:

a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processing device to cause the processing device to perform a method for vehicle perception, the method comprising;

capturing a plurality of images at a camera associated with a vehicle and storing image data associated with the plurality of images to a memory;

dispatching vehicle perception tasks to a plurality of processing elements of an accelerator in communication with the memory;

performing, by at least one of the plurality of processing elements, the vehicle perception tasks for the vehicle perception using a neural network, wherein performing the vehicle perception tasks comprises;

performing an activation bypass for values below a first threshold, and

performing weight pruning of synapses and neurons of the neural network based at least in part on a second threshold; and

controlling the vehicle based at least in part on a result of performing the vehicle perception tasks.