Neural network applications in resource constrained environments

• US 9,760,827 B1
• Filed: 01/03/2017
• Issued: 09/12/2017
• Est. Priority Date: 07/22/2016
• Status: Active Grant

• Current Assignee: Alpine Electronics of Silicon Valley, Inc. (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics of Silicon Valley, Inc. (Alps Alpine Co., Ltd.)
• Inventors: Lin, Rocky Chau-Hsiung, Yamasaki, Thomas, Kanda, Koichiro, Risco, Diego Rodriguez, Ryan, Alexander Joseph
• Primary Examiner(s): Chen, Alan
• Application Number:
  US15/397,715
• Time in Patent Office: 252 Days
• Field of Search: None
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Systems and methods are disclosed for applying neural networks in resource-constrained environments. A system may include a sensor located in a resource-constrained environment configured to generate sensor data of the resource-constrained environment. The system may also include a first computing device not located in the resource-constrained environment configured to produce a neural network structure based on the sensor data. The system may further include a second computing device located in the resource-constrained environment configured to provide the sensor data as input to the neural network structure. The second computing device may be further configured to determine a state of the resource-constrained environment based on the input of the sensor data to the neural network structure.

Claims

1. A system comprising:

a sensor located in an automobile, wherein the sensor is configured to produce first sensor data related to an interior of the automobile and second sensor data related to the interior of the automobile, wherein the first sensor data and the second sensor data are images of the interior of the automobile;

a local transceiver device located in the automobile, wherein the local transceiver device is configured to receive the first sensor data from the sensor;

a remote computing device located remote from the automobile, wherein the remote computing device is configured to apply a neural network to the first sensor data in order to produce neural network configuration parameters, wherein the local transceiver device is configured to transmit the first sensor data over a communication link to the remote computing device, and wherein the local transceiver device is configured to receive the neural network configuration parameters over the communication link from the remote computing device;

a local computing device located in the automobile, wherein the local computing device is configured to receive the neural network configuration parameters from the local transceiver device, wherein the local computing device is configured to receive the second sensor data from the sensor, and wherein the local computing device is configured to determine whether a driver of the automobile is in a distracted state based on the neural network configuration parameters and the second sensor data;

an automobile controller located in the automobile, wherein the automobile controller is configured to control a transition of the automobile between a self-driving mode and a non-self-driving mode, wherein the automobile controller is configured to receive an output of the distracted state determination from the local computing device, and wherein the automobile controller is configured to restrict the transition of the automobile between the self-driving mode and the non-self-driving mode based on the output of the distracted state determination; and

a second sensor located in the automobile, wherein the second sensor is configured to produce third sensor data related to an interior of the automobile and fourth sensor data related to the interior of the automobile, wherein the third sensor data and the fourth sensor data are images of the interior of the automobile,

wherein the remote computing device is configured to apply the neural network to the first sensor data and the third sensor data in order to produce the neural network configuration parameters,

wherein the local computing device is configured to receive the fourth sensor data from the second sensor, and wherein the local computing device is configured to determine whether the driver of the automobile is in a distracted state based on the neural network configuration parameters, the second sensor data, and the fourth sensor data,

wherein the remote computing device is configured to combine the first sensor data and the third sensor data into a stereovision image prior to producing the neural network configuration parameters,

wherein the sensor is a camera located adjacent to a rearview mirror in the automobile, and

wherein the second sensor is a camera located adjacent to an A-pillar in the automobile.

2. The system of claim 1, further comprising:

a third sensor located in the automobile, wherein the third sensor is configured to produce fifth sensor data that comprises non-image data related to the automobile and sixth sensor data that comprises non-image data related to the automobile,

wherein the remote computing device is configured to apply the neural network to the first sensor data, the third sensor data, and the fifth sensor data in order to produce the neural network configuration parameters, and

wherein the local computing device is configured to receive the sixth sensor data from the third sensor, and wherein the local computing device is configured to determine whether the driver of the automobile is in a distracted state based on the neural network configuration parameters, the second sensor data, the fourth sensor data, and the sixth sensor data.