Methods and systems for validating an autonomous system that includes a dynamic-code module and a static-code module

• US 10,073,965 B2
• Filed: 12/15/2015
• Issued: 09/11/2018
• Est. Priority Date: 12/15/2015
• Status: Active Grant

• Current Assignee: Nagravision SA (Kudelski SA)
• Original Assignee: Nagravision SA (Kudelski SA)
• Inventors: Jantz, Scott, Seltzer, Steven
• Primary Examiner(s): Lee, Jason
• Application Number:
  US14/970,167
• Publication Number:
  US 20170169208A1
• Time in Patent Office: 1,001 Days
• Field of Search: 713155
• CPC Class Codes: G06F 21/44 Program or device authentication
• CPC Class Codes: G06F 21/51 at application loading time, e.g. accepting, rejecting, starting or inhibiting executable software based on integrity or source reliability
• CPC Class Codes: G06F 21/57 Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 9/0861 Generation of secret information including derivation or calculation of cryptographic keys or passwords

Abstract

Disclosed herein are methods and systems for validating an autonomous system that comprises a static-code module and a dynamic-code module, the method including the steps of performing a code-integrity-validation process on the static-code module and performing a behavior-integrity-validation process on the dynamic-code module. In some embodiments, performing the code-integrity-validation process on the static-code module includes performing a signature-verification process on the static-code module. In some embodiments, performing the behavior-integrity-validation process on the dynamic-code module includes using an encrypted-and-signed test vector. In some embodiments, performing the behavior-integrity-validation process on the dynamic-code module includes selecting a test vector from among a plurality of test vectors, generating a modified test vector at least in part by modifying the selected test vector, and performing the behavior-integrity-validation process on the dynamic-code module using the modified test vector.

Claims

1. A method comprising:

an autonomous system performing a code-integrity-validation process on a static-code module of the autonomous system, wherein the static-code module comprises static code, wherein performing the code-integrity-validation process on the static-code module comprises determining whether the static code of the static-code module has or has not been modified;

the autonomous system performing a behavior-integrity-validation process on a dynamic-code module of the autonomous system, wherein performing the behavior-integrity-validation process on the dynamic-code module comprises;

generating an encrypted-and-verified test vector at least in part by verifying a signature of an encrypted-and-signed test vector;

generating a decrypted-and-verified test vector at least in part by decrypting the encrypted-and-verified test vector; and

determining whether an actual output of the dynamic-code module is or is not within a range of stored acceptable outputs for the decrypted-and-verified test vector; and

responsive to both (i) determining that the actual output of the dynamic-code module is within the range of stored acceptable outputs for the decrypted-and-verified test vector and (ii) determining that the static code of the static-code module has not been modified, permitting continued autonomous operation of the autonomous system, and otherwise responsively disabling autonomous operation of the autonomous system.

2. The method of claim 1, determining whether the static code of the static-code module has or has not been modified comprises:

performing a signature-verification process on the static code of the static-code module;

determining that the static code of the static-code module has not been modified if the static code of the static-code module passes the signature-verification process; and

determining that the static code of the static-code module has been modified if the static code of the static-code module fails the signature-verification process.

3. The method of claim 1, further comprising:

receiving both the encrypted-and-signed test vector and the encrypted-and-signed range of acceptable stored outputs as part of a secure update.

4. The method of claim 1, wherein the autonomous system comprises an autonomous driving vehicle, and wherein disabling autonomous operation of the autonomous system comprises continuing to permit manual driving of the autonomous driving vehicle.

5. The method of claim 1, wherein the autonomous system comprises an autonomous driving vehicle, and wherein disabling autonomous operation of the autonomous system comprises disabling all driving of the autonomous driving vehicle.

6. The method of claim 1, wherein disabling autonomous operation of the autonomous system comprises resetting a dynamic code of the dynamic-code module with a default dynamic code.

7. The method of claim 1, wherein performing the behavior-integrity-validation process on the dynamic-code module further comprises:

selecting a test vector from among a plurality of test vectors.

8. The method of claim 7, wherein selecting the test vector comprises randomly selecting the test vector.

9. The method of claim 1, wherein performing the behavior-integrity-validation process on the dynamic-code module further comprises:

selecting a test vector from among a plurality of test vectors; and

generating a modified test vector at least in part by modifying the selected test vector.

10. The method of claim 9, wherein performing the behavior-integrity-validation process on the dynamic-code module further comprises:

generating a modified range of acceptable results by modifying a stored range of acceptable results in accordance with the modified test vector; and

storing an expected output of the dynamic-code module for the modified test vector is the modified range of acceptable results.

11. The method of claim 1, wherein:

the autonomous system comprises an autonomous vehicle; and

the static-code module comprises a behavior arbiter that is configured to receive movement-vector-and-threat-level input and to produce corresponding driving-command output for the autonomous vehicle.

12. The method of claim 1, wherein:

the dynamic-code module comprises a neural network that is configured to receive 3D input data and to produce corresponding clustered-object output data; and

performing the behavior-integrity-validation process on the dynamic-code module further comprises sending test-vector 3D input data to the neural network and receiving corresponding clustered-object actual-output data, wherein;

the actual output of the dynamic-code module is the clustered-object actual-output data; and

a stored expected output of the dynamic-code module for the test-vector 3D input data is stored clustered-object expected-output data corresponding to the test-vector 3D-input data.

13. The method of claim 1, wherein:

the dynamic-code module comprises a classifier module that is configured to receive clustered-object data and to produce corresponding classified-object output data; and

performing the behavior-integrity-validation process on the dynamic-code module further comprises sending test-vector clustered-object data to the classifier module and receiving corresponding classified-object actual-output data, wherein;

the actual output of the dynamic-code module is the classified-object actual-output data; and

a stored expected output of the dynamic-code module for the test-vector clustered-object data stored classified-object expected-output data corresponding to the test-vector clustered-object data.

14. The method of claim 1, wherein:

the dynamic-code module comprises a threat-assessment module that is configured to receive classified-object data and to produce corresponding threat-matrix data; and

performing the behavior-integrity-validation process on the dynamic-code module further comprises sending test-vector classified-object data to the threat-assessment module and receiving corresponding threat-matrix actual-output data, wherein;

the actual output of the dynamic-code module is the classified-object actual-output data; and

a stored expected output of the dynamic-code module for the test-vector classified-object data is stored threat-matrix expected-output data corresponding to the test-vector classified-object data.

15. A secure core processor for an autonomous system secure core, the processor configured to:

perform a code-integrity-validation process on a static-code module of the autonomous system, wherein the static-code module comprises static code, wherein performing the code-integrity-validation process on the static-code module comprises determining whether the static code of the static-code module has or has not been modified;

perform a behavior-integrity-validation process on a dynamic-code module of the autonomous system, wherein performing the behavior-integrity-validation process on the dynamic-code module comprises;

generating an encrypted-and-verified test vector at least in part by verifying a signature of an encrypted-and-signed test vector;

generating a decrypted-and-verified test vector at least in part by decrypting the encrypted-and-verified test vector; and

determining whether an actual output of the dynamic-code module is or is not within a range of stored acceptable outputs for the decrypted-and-verified test vector; and

responsive to both (i) determining that the actual output of the dynamic-code module is within the range of stored acceptable outputs for the decrypted-and-verified test vector and (ii) determining that the static code of the static-code module has not been modified, permit continued autonomous operation of the autonomous system, and otherwise responsively disable autonomous operation of the autonomous system.