Latency smoothing for teleoperation systems

• US 9,300,430 B2
• Filed: 10/24/2013
• Issued: 03/29/2016
• Est. Priority Date: 10/24/2013
• Status: Active Grant

• Current Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Original Assignee: Harris Corporation (L3Harris Technologies, Inc.)
• Inventors: Summer, Matthew D., Bosscher, Paul M.
• Primary Examiner(s): HSU, ALPUS
• Application Number:
  US14/062,521
• Publication Number:
  US 20150117466A1
• Time in Patent Office: 887 Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/40147 Variable time delay, through internet
• CPC Class Codes: G05B 2219/40149 Local intelligence for global planning, remote intelligence for tuning
• CPC Class Codes: G05B 2219/40151 Time delay, problems caused by time delay between local and remote
• CPC Class Codes: G05B 2219/40174 Robot teleoperation through internet
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: H04L 1/0002 by adapting the transmission rate
• CPC Class Codes: H04L 1/0019 in which mode-switching is based on a statistical approach
• CPC Class Codes: H04L 43/0852 Delays

Abstract

Systems (100) and methods (700) for increasing a predictability of Telematic Operations (“TOs”) of a Teleoperation System (“TS”). The methods involve: measuring an inherent latency of a Communications Link (“CL”) of TS which varies unpredictably over at least a first window of time; analyzing the inherent latency, which was previously measured, to determine a first reference value useful for increasing the predictability of the TOs; using the first reference value to select an amount of controlled latency to be added to CL (120) at each of a plurality of time points (502-518); and adding the amount of controlled latency to CL at each of the plurality of time points so as to increase the predictability of the TOs. In some scenarios, the amount of controlled latency added at a first time point is different than the amount of controlled latency added at a second time point.

Claims

1. A method for increasing a predictability of telematic operations of a teleoperation system for conveying information over a network to remotely control of a slave device, comprising:

measuring, by the slave device or a control unit of the teleoperation system, an inherent latency of a communications link of the teleoperation system which varies unpredictably over at least a first window of time, where the inherent latency comprises a latency between a time a control message is sent from the control unit to a time the control message is received by the slave device;

analyzing, by the slave device or control unit, the inherent latency, which was previously measured, to determine a first reference value that can be used to increase the predictability of the telematic operations;

using, by the slave device or control unit, the first reference value to select an amount of controlled latency to be added to the communications link at each of a plurality of time points; and

adding, by the slave device or control unit, the amount of controlled latency to the communications link at each of the plurality of time points so as to increase the predictability of the telematic operations, where the amount of controlled latency added at a first time point of the plurality of time points is different than the amount of controlled latency added at a second time point of the plurality of time points.

2. The method according to claim 1, wherein the communications link is a forward communications link and the controlled latency is added to the communications link by the slave device.

3. The method according to claim 1, wherein the communications link is a reverse communications link and the controlled latency is added to the communications link by the control unit.

4. The method according to claim 1, wherein the first reference value comprises a peak value of a plurality of measured values for the inherent latency.

5. The method according to claim 1, wherein the first reference value comprises one of the plurality of measured values which falls within a pre-defined threshold range.

6. The method according to claim 1, wherein the first reference value comprises an average of the plurality of measured values for the inherent latency.

7. The method according to claim 1, wherein the amount of controlled latency to be added to the communications link at each time point is selected based on a difference between the first reference value and a measured value for the inherent latency at that time point.

8. The method according to claim 1, wherein the amount of controlled latency to be added to the communications link at each time point is selected to produce a constant overall latency for the communications link.

9. The method according to claim 1, wherein the controlled latency is added to the communications link by time delaying a message received at the control unit or the slave device by an amount determined based on the first reference value and a current level of inherent latency of the communications link.

10. A teleoperation system for electronic remote control of a slave device, comprising:

a control unit communicatively coupled to the slave device which is remotely located from the control unit, where at least one of a computing device of the control unit and a controller of the slave device is configured to;

measure an inherent latency of a communications link of the teleoperation system which varies unpredictably over at least a first window of time, where the inherent latency comprises a latency between a time a control message is sent from the control unit to a time the control message is received by the slave device;

analyze the inherent latency, which was previously measured, to determine a first reference value that can be used to increase a predictability of telematic operations for conveying information over a network to remotely control of the slave device;

use the first reference value to select an amount of controlled latency to be added to the communications link at each of a plurality of time points; and

add the amount of controlled latency to the communications link at each of the plurality of time points so as to increase the predictability of the telematic operations, where the amount of controlled latency added at a first time point of the plurality of time points is different than the amount of controlled latency added at a second time point of the plurality of time points.

11. The teleoperation system according to claim 10, wherein the communications link is a forward communications link.

12. The teleoperation system according to claim 10, wherein the communications link is a reverse communications link.

13. The teleoperation system according to claim 10, wherein the first reference value comprises a peak value of a plurality of measured values for the inherent latency.

14. The teleoperation system according to claim 10, wherein the first reference value comprises one of the plurality of measured values which falls within a pre-defined threshold range.

15. The teleoperation system according to claim 10, wherein the first reference value comprises an average of the plurality of measured values for the inherent latency.

16. The teleoperation system according to claim 10, wherein the amount of controlled latency to be added to the communications link at each time point is selected based on a difference between the first reference value and a measured value for the inherent latency at that time point.

17. The teleoperation system according to claim 10, wherein the amount of controlled latency to be added to the communications link at each time point is selected to produce a constant overall latency for the communications link.

18. The teleoperation system according to claim 10, wherein the controlled latency is added to the communications link by time delaying a message received at the control unit or the slave device by an amount determined based on the first reference value and a current level of inherent latency of the communications link.