Learning and deploying compression of radio signals

• US 10,572,830 B2
• Filed: 04/24/2018
• Issued: 02/25/2020
• Est. Priority Date: 04/24/2017
• Status: Active Grant

• Current Assignee: Virginia Tech Intellectual Properties, Inc. (Virginia Polytechnic Institute and State University)
• Original Assignee: Virginia Tech Intellectual Properties, Inc. (Virginia Polytechnic Institute and State University)
• Inventors: O'Shea, Timothy James
• Primary Examiner(s): Chang, Li Wu
• Application Number:
  US15/961,454
• Publication Number:
  US 20180314985A1
• Time in Patent Office: 672 Days
• Field of Search: None
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 3/086 using evolutionary algorithms, e.g. genetic algorithms or genetic programming
• CPC Class Codes: H04B 17/30 of propagation channels
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training and deploying machine-learned compact representations of radio frequency (RF) signals. One of the methods includes: determining a first RF signal to be compressed; using an encoder machine-learning network to process the first RF signal and generate a compressed signal; calculating a measure of compression in the compressed signal; using a decoder machine-learning network to process the compressed signal and generate a second RF signal that represents a reconstruction of the first RF signal; calculating a measure of distance between the second RF signal and the first RF signal; and updating at least one of the encoder machine-learning network or the decoder machine-learning network based on (i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal.

Claims

1. A method of training at least one machine-learning network to learn compact representations of radio frequency (RF) signals, the method performed by at least one processor executing instructions stored on at least one computer memory coupled to the at least one processor, the method comprising:

determining a first RF signal to be compressed;

using an encoder machine-learning network to process the first RF signal and generate a compressed signal;

using a decoder machine-learning network to process the compressed signal and generate a second RF signal that represents a reconstruction of the first RF signal;

calculating a measure of distance between the second RF signal and the first RF signal;

obtaining a measure of compression in the compressed signal; and

updating at least one of the encoder machine-learning network or the decoder machine-learning network based on (i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal, wherein the updating comprises;

determining an objective function that comprises at least;

(i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal,

calculating a rate of change of the objective function relative to variations in at least one of the encoder machine-learning network or the decoder machine-learning network,

selecting, based on the calculated rate of change of the objective function, at least one of a first variation for the encoder machine-learning network or a second variation for the decoder machine-learning network, and

updating at least one of the encoder machine-learning network or the decoder machine-learning network based on the selected at least one of the first variation or the second variation.

2. The method of claim 1, wherein at least one of the encoding machine-learning network or the decoding machine-learning network comprises at least one of a deep dense neural network (DNN) or a convolutional neural network (CNN) comprising a series of parametric multiplications, additions, and non-linearities.

3. The method of claim 1, wherein the measure of compression in the compressed signal indicates at least one of (i) a relative measure of compression between the first RF signal and the compressed signal or (ii) an absolute measure of complexity in the compressed signal.

4. The method of claim 1, wherein the measure of distance between the second RF signal and the first RF signal comprises a cross-entropy between the second RF signal and the first RF signal.

5. The method of claim 1, wherein using the encoder machine-learning network to process the first RF signal and using the decoder machine-learning network to process the compressed signal comprise:

determining a plurality of basis signals;

using the plurality of basis signals to process the first RF signal by the encoder machine-learning network and generate the compressed signal; and

using the plurality of basis signals to process the compressed signal by the decoder machine-learning network and generate the second RF signal.

6. The method of claim 1, wherein updating at least one of the encoder machine-learning network or the decoder machine-learning network comprises at least one of:

updating at least one encoding network weight or network connectivity in one or more layers of the encoder machine-learning network, or

updating at least one decoding network weight or network connectivity in one or more layers of the decoder machine-learning network.

7. The method of claim 1, wherein updating at least one of the encoder machine-learning network or the decoder machine-learning network comprises either (i) jointly updating the encoder machine-learning network and the decoder machine-learning network, or (ii) separately updating the encoder machine-learning network and the decoder machine-learning network.

8. The method of claim 1, wherein the encoder machine-learning network and the decoder machine-learning network are jointly trained as an auto-encoder to learn compact representations of RF signals, and

wherein the auto-encoder comprises at least one regularization layer that comprises at least one of:

weight regularization on network layer weights, activity regularization on network layer activations, or stochastic impairments on network layer activations or network layer weights.

9. The method of claim 1, wherein the first RF signal and the second RF signal comprise at least one of an acoustic signal or an optical signal.

29. The method of claim 1, wherein the measure of distance between the second RF signal and the first RF signal comprises a geometric distance metric between the second RF signal and the first RF signal.

31. The method of claim 1, wherein obtaining the measure of compression in the compressed signal comprises one of:

calculating the measure of compression in the compressed signal, or accessing a known measure of compression used to generate the compressed signal.

10. A method of deploying at least one machine-learning network that has been trained to learn compact representations of radio frequency (RF) signals, the method performed by at least one processor executing instructions stored on at least one computer memory coupled to the at least one processor, the method comprising:

determining an encoder machine-learning network and a decoder machine-learning network that have been trained to learn compact representations of RF signals;

determining a first RF signal to be compressed;

using the encoder machine-learning network to process the first RF signal and generate a first compressed signal;

obtaining a second compressed signal that comprises the first compressed signal or an alteration thereof; and

using the decoder machine-learning network to process the second compressed signal to generate a second RF signal as a reconstruction of the first RF signal,

wherein at least one of the encoder machine-learning network or the decoder machine-learning network is trained based on (i) a measure of distance between a training RF signal and a reconstruction of the training RF signal, and (ii) a measure of compression in compressing the training RF signal, wherein the training comprises;

determining an objective function that comprises at least;

(i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal,

calculating a rate of change of the objective function relative to variations in at least one of the encoder machine-learning network or the decoder machine-learning network,

selecting, based on the calculated rate of change of the objective function, at least one of a first variation for the encoder machine-learning network or a second variation for the decoder machine-learning network, and

updating at least one of the encoder machine-learning network or the decoder machine-learning network based on the selected at least one of the first variation or the second variation.

11. The method of claim 10, further comprising:

storing the first compressed signal in an information storage device;

accessing, from the information storage device, the second compressed signal as a stored version of the first compressed signal; and

inputting the first compressed signal or the second compressed signal into a system for processing a compact representation of the first RF signal.

12. The method of claim 10, further comprising:

transmitting, by a transmitter, the first compressed signal through a communication channel; and

receiving, by a receiver, the second compressed signal as a received version of the first compressed signal having been processed by transmission through the communication channel.

13. The method of claim 10, further comprising:

determining a measure of distance between the second RF signal and the first RF signal; and

based on the measure of distance between the second RF signal and the first RF signal exceeding a threshold, determining an occurrence of an error or an anomaly.

14. The method of claim 10, wherein the encoder machine-learning network and the decoder machine-learning network have been jointly trained as an auto-encoder to learn compact representations of RF signals, and

wherein the auto-encoder comprises at least one regularization layer that comprises at least one of:

weight regularization on network layer weights, activity regularization on network layer activations, or stochastic impairments on network layer activations or network layer weights.

15. A system comprising:

at least one processor; and

at least one computer memory coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, cause the at least one processor to perform operations comprising;

determining a first RF signal to be compressed;

using an encoder machine-learning network to process the first RF signal and generate a compressed signal;

obtaining a measure of compression in the compressed signal;

using a decoder machine-learning network to process the compressed signal and generate a second RF signal that represents a reconstruction of the first RF signal;

calculating a measure of distance between the second RF signal and the first RF signal; and

updating at least one of the encoder machine-learning network or the decoder machine-learning network based on (i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal, wherein the updating comprises;

determining an objective function that comprises at least;

(i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal,

calculating a rate of change of the objective function relative to variations in at least one of the encoder machine-learning network or the decoder machine-learning network,

selecting, based on the calculated rate of change of the objective function, at least one of a first variation for the encoder machine-learning network or a second variation for the decoder machine-learning network, and

updating at least one of the encoder machine-learning network or the decoder machine-learning network based on the selected at least one of the first variation or the second variation.

16. The system of claim 15, wherein at least one of the encoding machine-learning network or the decoding machine-learning network comprises at least one of a deep dense neural network (DNN) or a convolutional neural network (CNN) comprising a series of parametric multiplications, additions, and non-linearities.

17. The system of claim 15, wherein the measure of compression in the compressed signal indicates at least one of (i) a relative measure of compression between the first RF signal and the compressed signal or (ii) an absolute measure of complexity in the compressed signal.

18. The system of claim 15, wherein the measure of distance between the second RF signal and the first RF signal comprises a cross-entropy between the second RF signal and the first RF signal.

19. The system of claim 15, wherein using the encoder machine-learning network to process the first RF signal and using the decoder machine-learning network to process the compressed signal comprise:

determining a plurality of basis signals;

using the plurality of basis signals to process the first RF signal by the encoder machine-learning network and generate the compressed signal; and

using the plurality of basis signals to process the compressed signal by the decoder machine-learning network and generate the second RF signal.

20. The system of claim 15, wherein updating at least one of the encoder machine-learning network or the decoder machine-learning network comprises at least one of:

updating at least one encoding network weight or network connectivity in one or more layers of the encoder machine-learning network, or

updating at least one decoding network weight or network connectivity in one or more layers of the decoder machine-learning network.

21. The system of claim 15, wherein updating at least one of the encoder machine-learning network or the decoder machine-learning network comprises either (i) jointly updating the encoder machine-learning network and the decoder machine-learning network, or (ii) separately updating the encoder machine-learning network and the decoder machine-learning network.

22. The system of claim 15, wherein the encoder machine-learning network and the decoder machine-learning network are jointly trained as an auto-encoder to learn compact representations of RF signals, and

wherein the auto-encoder comprises at least one regularization layer that comprises at least one of:

weight regularization on network layer weights, activity regularization on network layer activations, or stochastic impairments on network layer activations or network layer weights.

23. The system of claim 15, wherein the first RF signal and the second RF signal comprise at least one of an acoustic signal or an optical signal.

30. The system of claim 15, wherein the measure of distance between the second RF signal and the first RF signal comprises a geometric distance metric between the second RF signal and the first RF signal.

32. The system of claim 15, wherein obtaining the measure of compression in the compressed signal comprises one of:

calculating the measure of compression in the compressed signal, or

accessing a known measure of compression used to generate the compressed signal.

24. A system comprising:

at least one processor; and

at least one computer memory coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, cause the at least one processor to perform operations comprising;

determining an encoder machine-learning network and a decoder machine-learning network that have been trained to compress RF signals;

determining a first RF signal to be compressed;

using the encoder machine-learning network to process the first RF signal and generate a first compressed signal;

obtaining a second compressed signal that comprises the first compressed signal or an alteration thereof; and

using the decoder machine-learning network to process the second compressed signal to generate a second RF signal as a reconstruction of the first RF signal,

wherein at least one of the encoder machine-learning network or the decoder machine-learning network is trained based on (i) a measure of distance between a training RF signal and a reconstruction of the training RF signal, and (ii) a measure of compression in compressing the training RF signal, wherein the training comprises;

determining an objective function that comprises at least;

(i) the measure of distance between the second RF signal and the first RF signal, and (ii) the measure of compression in the compressed signal,

calculating a rate of change of the objective function relative to variations in at least one of the encoder machine-learning network or the decoder machine-learning network,

selecting, based on the calculated rate of change of the objective function, at least one of a first variation for the encoder machine-learning network or a second variation for the decoder machine-learning network, and

updating at least one of the encoder machine-learning network or the decoder machine-learning network based on the selected at least one of the first variation or the second variation.

25. The system of claim 24, wherein the operations further comprise:

storing the first compressed signal in an information storage device;

accessing, from the information storage device, the second compressed signal as a stored version of the first compressed signal; and

inputting the first compressed signal or the second compressed signal into a system for processing a compact representation of the first RF signal.

26. The system of claim 24, wherein the operations further comprise:

transmitting, by a transmitter, the first compressed signal through a communication channel; and

receiving, by a receiver, the second compressed signal as a received version of the first compressed signal having been processed by transmission through the communication channel.

27. The system of claim 24, wherein the operations further comprise:

determining a measure of distance between the second RF signal and the first RF signal; and

based on the measure of distance between the second RF signal and the first RF signal exceeding a threshold, determining an occurrence of an error or an anomaly.

28. The system of claim 24, wherein the encoder machine-learning network and the decoder machine-learning network have been jointly trained as an auto-encoder to learn compact representations of RF signals, and

wherein the auto-encoder comprises at least one regularization layer that comprises at least one of:

weight regularization on network layer weights, activity regularization on network layer activations, or stochastic impairments on network layer activations or network layer weights.