System, a method and a computer program product for automated remote control

• US 9,467,518 B2
• Filed: 02/15/2010
• Issued: 10/11/2016
• Est. Priority Date: 02/16/2009
• Status: Expired due to Fees

• Current Assignee: CommuniTake Technologies Ltd.
• Original Assignee: CommuniTake Technologies Ltd.
• Inventors: Perlmutter, Amiram, Karpen, Eran, Lavie, Omri, Holy, Shalev, Sasson, Ronen
• Primary Examiner(s): Blair, Douglas
• Assistant Examiner(s): Moorad, Imran
• Application Number:
  US13/201,406
• Publication Number:
  US 20120191770A1
• Time in Patent Office: 2,430 Days
• Field of Search: 709212-213, 709/216, 709/219, 709/227, 709230-232, 709/246, 709/248, 709/200, 709/201
• US Class Current: 1/1
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images
• CPC Class Codes: G08C 2201/33 Remote control using macros, scripts
• CPC Class Codes: G08C 2201/42 Transmitting or receiving remote control signals via a network
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/34 involving the movement of software or configuration parameters network booting or remote initial program loading [RIPL] G06F9/4416
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04W 4/50 Service provisioning or reconfiguring
• CPC Class Codes: H04W 8/24 Transfer of terminal data

Abstract

A method enables automated remote control of a remote device from a local device connected to the remote device over a network. The method includes executing a set of one or more commands on the remote device by invoking a macro on the local device. Invoking the macro causes the local device to send said the set of commands over the network to the remote device.

Claims

1. A method comprising:

executing a first macro to activate, in relation to a mobile device, a screen capture mode;

starting a clock in response to a sending of a first screen image captured in conjunction with the screen capture mode;

receiving, from the mobile device, the first screen image captured in conjunction with the screen capture mode;

stopping the clock in response to receipt of the first screen image to determine a communication time associated with the first screen image;

processing the first screen image using optical character recognition (OCR) to extract text data from the first screen image;

comparing the text data as extracted from the first screen image with an expected screen configuration;

defining, based on the comparison of the text data text data with the expected screen configuration, a second macro;

in response to an execution of the second macro, receiving a modified part of a second screen image captured based on a determination that a number of images sent by the mobile device is within a defined maximum allowed difference of a number of acknowledgement signals sent with respect to images received from the mobile device, wherein a quality of the second screen image is adjusted based on the communication time associated with the first screen image;

comparing the second screen image with the first screen image to identify one or more differences between the second screen image and the first screen image;

creating one or more regions within the second screen image around each of the one or more differences;

joining the one or more regions into one or more larger regions;

converting each of the one or more larger regions into a defined image format; and

inserting each of the one or more larger regions, as converted, into the first screen image based on one or more coordinates of the one or more larger regions within the second screen image.

2. The method of claim 1, further comprising detecting a problem with respect to the mobile device.

3. The method of claim 1, further comprising combining the first screen image with a previously captured image to generate an image currently displayed at the mobile device.

4. The method of claim 1, wherein receiving a first screen image further comprises receiving one or more coordinates associated with the first screen image.

5. The method of claim 1, further comprising detecting a problem in relation to the mobile device.

6. The method of claim 5, further comprising providing, to the mobile device and based on the detected problem, a first command.

7. The method of claim 6, wherein processing the first screen image comprises processing the first screen image using optical character recognition (OCR) to (a) extract text data from the first screen image and (b) determine a response of the mobile device with respect to the first command.

8. The method of claim 7, wherein defining a second macro comprises defining, based on (a) the comparison of the text data text data with the expected screen configuration and (b) the response of the mobile device with respect to the first command, a second macro.

9. A system comprising:

a processing device configured to;

execute a first macro to activate, in relation to a mobile device, a screen capture mode;

start a clock in response to a sending of a first screen image captured in conjunction with the screen capture mode;

receive, from the mobile device, the first screen image captured in conjunction with the screen capture mode;

stop the clock in response to receipt of the first screen image to determine a communication time associated with the first screen image;

process the first screen image using optical character recognition (OCR) to extract text data from the first screen image;

compare the text data as extracted from the first screen image with an expected screen configuration;

define, based on the comparison of the text data text data with the expected screen configuration, a second macro;

in response to an execution of the second macro, receive a modified part of a second screen image captured based on a determination that a number of images sent by the mobile device is within a defined maximum allowed difference of a number of acknowledgement signals sent with respect to images received from the mobile device, wherein a quality of the second screen image is adjusted based on the communication time associated with the first screen image;

compare the second screen image with the first screen image to identify one or more differences between the second screen image and the first screen image;

create one or more regions within the second screen image around each of the one or more differences;

join the one or more regions into one or more larger regions;

convert each of the one or more larger regions into a defined image format; and

insert each of the one or more larger regions, as converted, into the first screen image based on one or more coordinates of the one or more larger regions within the second screen image.

10. The system of claim 9, wherein the processing device is further configured to detect a problem with respect to the mobile device.

11. The system of claim 9, wherein the processing device is further configured to combine the first screen image with a previously captured image to generate an image currently displayed at the mobile device.

12. The system of claim 9, wherein to receive a first image is to receive one or more coordinates associated with the first screen image.

13. The system of claim 9, wherein the processing device is further configured to detect a problem in relation to the mobile device.

14. The system of claim 13, wherein the processing device is further configured to provide, to the mobile device and based on the detected problem, a first command.

15. The system of claim 14, wherein to process the first screen image the processing device is further to process the first screen image using optical character recognition (OCR) to (a) extract text data from the first screen image and (b) determine a response of the mobile device with respect to the first command.

16. The system of claim 15, wherein to define a second macro the processing device is further configured to define, based on (a) the comparison of the text data text data with the expected screen configuration and (b) the response of the mobile device with respect to the first command, a second macro.

17. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to perform operations comprising:

executing a first macro to activate, in relation to a mobile device, a screen capture mode;

starting a clock in response to a sending of a first screen image captured in conjunction with the screen capture mode;

receiving, from the mobile device, the first screen image, the first screen image captured in conjunction with the screen capture mode;

stopping the clock in response to receipt of the first screen image to determine a communication time associated with the first screen image;

processing the first screen image using optical character recognition (OCR) to extract text data from the first screen image;

comparing the text data as extracted from the first screen image with an expected screen configuration;

defining, based on the comparison of the text data text data with the expected screen configuration, a second macro;

in response to an execution of the second macro, receiving a modified part of a second screen image captured based on a determination that a number of images sent by the mobile device is within a defined maximum allowed difference of a number of acknowledgement signals sent with respect to images received from the mobile device, wherein a quality of the second screen image is adjusted based on the communication time associated with the first screen image;

comparing the second screen image with the first screen image to identify one or more differences between the second screen image and the first screen image;

creating one or more regions within the second screen image around each of the one or more differences;

joining the one or more regions into one or more larger regions;

converting each of the one or more larger regions into a defined image format; and

inserting each of the one or more larger regions, as converted, into the first screen image based on one or more coordinates of the one or more larger regions within the second screen image.

18. The non-transitory computer readable medium of claim 17, further comprising detecting a problem in relation to the mobile device.

19. The non-transitory computer readable medium of claim 18, further comprising providing, to the mobile device and based on the detected problem, a first command.

20. The non-transitory computer readable medium of claim 19, wherein processing the first screen image comprises processing the first screen image using optical character recognition (OCR) to (a) extract text data from the first screen image and (b) determine a response of the mobile device with respect to the first command.