Neural network applications in resource constrained environments

• US 9,965,720 B2
• Filed: 09/08/2017
• Issued: 05/08/2018
• Est. Priority Date: 07/22/2016
• Status: Active Grant

• Current Assignee: Alpine Electronics of Silicon Valley, Inc. (Alps Alpine Co., Ltd.)
• Original Assignee: Alpine Electronics of Silicon Valley, Inc. (Alps Alpine Co., Ltd.)
• Inventors: Lin, Rocky Chau-Hsiung, Yamasaki, Thomas, Kanda, Koichiro, Risco, Diego Rodriguez, Ryan, Alexander Joseph
• Primary Examiner(s): Chen, Alan
• Application Number:
  US15/699,298
• Publication Number:
  US 20180025272A1
• Time in Patent Office: 242 Days
• Field of Search: None
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

Systems and methods are disclosed for applying neural networks in resource-constrained environments. A system may include a sensor located in a resource-constrained environment configured to generate sensor data of the resource-constrained environment. The system may also include a first computing device not located in the resource-constrained environment configured to produce a neural network structure based on the sensor data. The system may further include a second computing device located in the resource-constrained environment configured to provide the sensor data as input to the neural network structure. The second computing device may be further configured to determine a state of the resource-constrained environment based on the input of the sensor data to the neural network structure.

Claims

1. A system comprising:

a sensor located in a resource-constrained environment configured to generate first sensor data of the resource-constrained environment and second sensor data of the resource-constrained environment, wherein the first sensor data and the second sensor data are images of the resource-constrained environment;

a first computing device not located in the resource-constrained environment configured to produce a neural network structure based on the first sensor data;

a second computing device located in the resource-constrained environment configured to provide the second sensor data as input to the neural network structure, wherein the second computing device is further configured to determine a state of the resource-constrained environment based on the input of the second sensor data to the neural network structure;

a controller located in the resource-constrained environment configured to control an indicator device in the resource-constrained environment based on the state of the resource-constrained environment determined by the second computing device,

wherein the second computing device is configured to determine the state of the resource-constrained environment at least in part by determining whether an adult is present in the resource-constrained environment based on the input of the second sensor data to the neural network structure, and

wherein the second computing device is configured to determine the state of the resource-constrained environment at least in part by determining whether a child is present in the resource-constrained environment based on the input of the second sensor data to the neural network structure.

2. The system of claim 1,

wherein the resource-constrained environment comprises an interior of an automobile,

wherein the first sensor data and the second sensor data are images of the interior of the automobile, and

wherein the state of the resource-constrained environment comprises the presence or absence of an occupant of the automobile.

3. The system of claim 2,

wherein the controller is further configured to control operation of the automobile based on the determined state of the presence or absence of the occupant of the automobile.

4. The system of claim 1,

wherein the resource-constrained environment comprises an interior of an automobile,

wherein the controller is configured to control the indicator device based on the determination of whether an adult is present in the resource-constrained environment, based on the determination of whether a child is present in the resource-constrained environment, and based on a present gear of the automobile.

5. The system of claim 1,

wherein the second computing device is configured to determine the state of the resource-constrained environment at least in party by determining whether a child is restrained in a child safety seat in the resource-constrained environment based on the input of the second sensor data to the neural network structure.

6. The system of claim 5,

wherein the resource-constrained environment comprises an interior of an automobile,

wherein the controller is configured to control the indicator device based on the determination of whether a child is restrained in a child safety seat in the resource-constrained environment and based on a present gear of the automobile.

7. The system of claim 1,

wherein the resource-constrained environment comprises an interior of an automobile,

wherein the first sensor data and the second sensor data are images of the interior of the automobile,

wherein the second computing device is configured to determine whether an adult is present in the interior of the automobile based on the input of the second sensor data to the neural network structure,

wherein the second computing device is configured to determine whether a child is present in the interior of the automobile based on the input of the second sensor data to the neural network structure, and

wherein the controller is configured to activate the indicator device if the second computing device determines that:

a child is present in the interior of the automobile, and

an adult transitions from being present in the interior of the automobile to being not present in the interior of the automobile.

8. The system of claim 1, wherein the resource-constrained environment comprises an interior of an automobile, and wherein the first sensor data and the second sensor data are images of a driver'"'"'s seat of the automobile, the system further comprising:

a second sensor located in the automobile, wherein the second sensor is configured to produce third sensor data related to the interior of the automobile and fourth sensor data related to the interior of the automobile,

wherein the third sensor data and the fourth sensor data comprise an image of a child safety seat in the interior of the automobile,

wherein the first computing device is configured to produce the neural network structure based on the first sensor data and the third sensor data, and

wherein the second computing device is configured to receive the fourth sensor data from the second sensor, and wherein the second computing device is configured to determine the state of the resource-constrained environment based on input of the second sensor data to the neural network structure and based on input of the fourth sensor data to the neural network structure.

9. The system of claim 1, further comprising:

a local transceiver device located in the resource-constrained environment,

wherein the local transceiver device is configured to receive the first sensor data from the sensor,

wherein the local transceiver device is configured to transmit the first sensor data over a communication link to the first computing device, and

wherein the local transceiver is configured to receive the neural network structure over the communication link from the first computing device.

10. A method comprising:

generating first sensor data and second sensor data of a resource-constrained environment using a sensor located in the resource-constrained environment, wherein the first sensor data and the second sensor data are images of the resource-constrained environment;

producing a neural network structure based on the first sensor data using a first computing device not located in the resource-constrained environment;

inputting the sensor data to the neural network structure using a second computing device located in the resource-constrained environment;

determining a state of the resource-constrained environment using the second computing device based on the input of the second sensor data to the neural network structure; and

controlling an indicator device in the resource-constrained environment using a controller located in the resource-constrained environment based on the state of the resource-constrained environment determined by the second computing device;

determining whether an adult is present in the resource-constrained environment using the second computing device based on the input of the second sensor data to the neural network structure; and

determining whether a child is present in the resource-constrained environment using the second computing device based on the input of the second sensor data to the neural network structure.

11. The method of claim 10,

wherein the resource-constrained environment comprises an interior of an automobile,

wherein the first sensor data and the second sensor data are images of the interior of the automobile, and

wherein the state of the resource-constrained environment comprises the presence or absence of an occupant of the automobile.

12. The method of claim 11, further comprising:

controlling operation of the automobile using the controller based on the determined state of the presence or absence of the occupant of the automobile.

13. The method of claim 10, wherein the resource-constrained environment comprises an interior of an automobile, the method further comprising:

controlling the indicator device using the controller based on the determination of whether an adult is present in the resource-constrained environment, based on the determination of whether a child is present in the resource-constrained environment, and based on a present gear of the automobile.

14. The method of claim 10, wherein the determining the state of the resource-constrained environment comprises:

determining whether a child is restrained in a child safety seat in the resource-constrained environment using the second computing device based on the input of the second sensor data to the neural network structure.

15. The method of claim 14, wherein the resource-constrained environment comprises an interior of an automobile, the method further comprising:

controlling the indicator device using the controller based on the determination of whether a child is restrained in a child safety seat in the resource-constrained environment and based on a present gear of the automobile.

16. The method of claim 10, wherein the resource-constrained environment comprises an interior of an automobile, and wherein the first sensor data and the second sensor data are images of the interior of the automobile, the method further comprising:

determining whether an adult is present in the interior of the automobile using the second computing device based on the input of the second sensor data to the neural network structure,

determining whether a child is present in the interior of the automobile using the second computing device based on the input of the second sensor data to the neural network structure, and

activating the indicator device using the controller based on determining that;

a child is present in the interior of the automobile, and

an adult transitions from being present in the interior of the automobile to being not present in the interior of the automobile.

17. The method of claim 10, wherein the resource-constrained environment comprises an interior of an automobile, and wherein the first sensor data and the second sensor data are images of a driver'"'"'s seat of the automobile, the method further comprising:

generating third sensor data and fourth sensor data of the resource-constrained environment using a second sensor located in the resource-constrained environment, wherein the third sensor data and the fourth sensor data comprise images of a child safety seat in the interior of the automobile;

producing the neural network structure using the first computing device based on the first sensor data and the third sensor data;

receiving the fourth sensor data at the second computing device from the second sensor; and

determining the state of the resource-constrained environment using the second computing device based on input of the second sensor data to the neural network structure and based on input of the fourth sensor data to the neural network structure.

18. The method of claim 10, further comprising:

receiving the first sensor data at a local transceiver device located in the resource-constrained environment from the sensor;

transmitting the first sensor data over a communication link to the first computing device using the local transceiver; and

receiving the neural network structure over the communication link at the local transceiver device from the first computing device.