VISUAL REMOTE CONTROL METHOD AND SYSTEM FOR TOUCH-CONTROLLABLE DEVICE, AND RELATED DEVICE

• US 20180173396A1
• Filed: 02/19/2018
• Published: 06/21/2018
• Est. Priority Date: 08/18/2015
• Status: Abandoned Application

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: LIN, Yimin, LIAN, Shiguo
• Application Number:
  US15/898,838
• Publication Number:
  US 20180173396A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: G08C 2201/21 Programming remote control devices via third means
• CPC Class Codes: G08C 2201/30 User interface
• CPC Class Codes: G08C 2201/50 Receiving or transmitting feedback, e.g. replies, status updates, acknowledgements, from the controlled devices
• CPC Class Codes: G08C 2201/70 Device selection
• CPC Class Codes: G08C 2201/91 Remote control based on location and proximity
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Examples of the present disclosure provide a visual remote control method and system for controlling a touch-controllable device. The method includes: obtaining, by a control center device, a video image; determining, by the control center device, an identifier of each of at least one touch-controllable device in the video image; and sending, by the control center device, the video image and the identifier of each touch-controllable device to a remote device, so that the touch-controllable device in the video image is controllable by using the remote device.

Claims

1. A visual remote control method for controlling a touch-controllable device, wherein the method is executed by a control center device in a remote control system, and the method comprises:

obtaining, by the control center device, a video image;

determining, by the control center device, an identifier of each of at least one touch-controllable device in the video image; and

sending, by the control center device, the video image and the identifier of each touch-controllable device to a remote device, so that the touch-controllable device in the video image is controllable by using the remote device.

2. The method according to claim 1, wherein determining, by the control center device, the identifier of each of at least one touch-controllable device in the video image comprises:

performing, by the control center device, local-feature extraction on the video image;

matching an extracted local feature with a visual feature of the touch-controllable device stored in a hardware database, and determining whether the visual feature of the touch-controllable device that matches the local feature exists in the hardware database; and

when the visual feature of the touch-controllable device that matches the local feature exists in the hardware database, determining the identifier of the touch-controllable device that is comprised in the video image is corresponding to the matched visual feature.

3. The method according to claim 2, wherein the hardware database comprises attribute information of N touch-controllable devices and N is a positive integer that is greater than or equal to 1, wherein the attribute information comprises identifiers of the touch-controllable devices, visual features of the touch-controllable devices, hardware application programming interfaces (APIs) of the touch-controllable devices, and control user interfaces (UIs) of the touch-controllable devices.

4. The method according to claim 3, wherein, before determining, by the control center device, the identifier of each of at least one touch-controllable device in the video image, the method further comprises:

obtaining, by the control center device, the attribute information of the N touch-controllable devices; and

storing, by the control center device, the attribute information of the N touch-controllable devices in the hardware database.

5. The method according to claim 4, wherein obtaining, by the control center device, the attribute information of the N touch-controllable devices comprises:

receiving, by the control center device, the attribute information sent by each of the N touch-controllable devices;

or

receiving, by the control center device, the identifier sent by each of the N touch-controllable devices, and obtaining, according to the identifiers of the N touch-controllable devices, the visual features of the N touch-controllable devices, the APIs of the N touch-controllable devices, and the control UIs of the N touch-controllable devices from a server that stores the attribute information of the N touch-controllable devices.

6. The method according to claim 1, wherein determining, by the control center device, the identifier of each of at least one touch-controllable device in the video image comprises:

receiving, by the control center device, the identifier of each touch-controllable device sent by a video capture device.

7. The method according to claim 1, wherein the method further comprises:

receiving, by the control center device, first adjustment information sent by the remote device, wherein the first adjustment information is used to adjust an adjustable running parameter of a first touch-controllable device, and the at least one touch-controllable device comprises the first touch-controllable device; and

sending, by the control center device, the first adjustment information to the first touch-controllable device, so that the first touch-controllable device adjusts the adjustable running parameter according to the first adjustment information.

8. A visual remote control system for controlling a touch-controllable device, wherein the system comprises a control center device, a remote device, and N touch-controllable devices, wherein:

N is a positive integer that is greater than or equal to 1;

the control center device is configured to obtain a video image;

the control center device is further configured to determine an identifier of each touch-controllable device that is in the video image and each touch-controllable device is in the N touch-controllable devices;

the control center device is further configured to send the video image and the identifier of each touch-controllable device to the remote device;

the remote device is configured to receive the video image and the identifier of each touch-controllable device that are sent by the control center device; and

the remote device is further configured to present the video image and the identifier of each touch-controllable device on a display interface.

9. The system according to claim 8, wherein the control center device is configured to:

perform local-feature extraction on the video image;

match an extracted local feature with a visual feature of the touch-controllable device stored in a hardware database to determine whether the visual feature of the touch-controllable device that matches the local feature exists in the hardware database; and

when the visual feature of the touch-controllable device that matches the local feature exists in the hardware database, determine the identifier of the touch-controllable device that is comprised in the video image is corresponding to the matched visual feature.

10. The system according to claim 9, wherein the hardware database comprises attribute information of the N touch-controllable devices, wherein the attribute information comprises identifiers of the touch-controllable devices, visual features of the touch-controllable devices, hardware application programming interfaces (APIs) of the touch-controllable devices, and control user interfaces (UIs) of the touch-controllable devices.

11. The system according to claim 10, wherein the control center device is further configured to obtain the attribute information of the N touch-controllable devices, and store the attribute information of the N touch-controllable devices in the hardware database.

12. The system according to claim 11, wherein:

the control center device is configured to receive the attribute information sent by each of the N touch-controllable devices;

or

the control center device is configured to receive the identifier sent by each of the N touch-controllable devices, and obtain, according to the identifiers of the N touch-controllable devices, the visual features of the N touch-controllable devices, the APIs of the N touch-controllable devices, and the control UIs of the N touch-controllable devices from a server that stores the attribute information of the N touch-controllable devices.

13. The system according to claim 8, wherein the system further comprises a video capture device, wherein:

the video capture device is configured to obtain the video image, and determine the identifier of each touch-controllable device in the video image;

the video capture device is further configured to send the video image and the identifier of each touch-controllable device to the control center device; and

the control center device receives the video image and the identifier of each touch-controllable device that are sent by the video capture device.

14. The system according to claim 8, wherein the remote device is further configured to obtain first input, wherein the first input is used to select a first touch-controllable device, and the touch-controllable device in the video image comprise the first touch-controllable device;

the remote device is further configured to obtain a control UI of the first touch-controllable device and an API of the first touch-controllable device; and

the remote device is further configured to display the control UI of the first touch-controllable device on the display interface.

15. The system according to claim 14, wherein the remote device is further configured to obtain second input that is used to adjust an adjustable running parameter of the first touch-controllable device, and send first adjustment information to the control center device, wherein the first adjustment information is used to adjust the adjustable running parameter of the first touch-controllable device;

the control center device is further configured to send the received first adjustment information to the first touch-controllable device; and

the first touch-controllable device is configured to adjust the corresponding adjustable running parameter according to the first adjustment information.

16. A network device for controlling a touch-controllable device, comprising:

an obtaining circuit, configured to obtain a video image;

a determining circuit, configured to determine an identifier of each of at least one touch-controllable device in the video image; and

a sending circuit, configured to send the video image and the identifier of each touch-controllable device to a remote device, so that the touch-controllable device in the video image is controllable by using the remote device.

17. The network device according to claim 16, wherein the determining circuit is configured to:

perform local-feature extraction on the video image;

match an extracted local feature with a visual feature of a touch-controllable device stored in a hardware database to determine whether a visual feature of a touch-controllable device that matches the local feature exists in the hardware database; and

when the visual feature of the touch-controllable device that matches the local feature exists in the hardware database, determine the identifier of the touch-controllable device that is comprised in the video image is corresponding to the matched visual feature.

18. The network device according to claim 17, wherein the hardware database comprises attribute information of N touch-controllable devices, wherein the attribute information comprises identifiers of the touch-controllable devices, visual features of the touch-controllable devices, hardware application programming interfaces (APIs) of the touch-controllable devices, and control user interfaces (UIs) of the touch-controllable devices.

19. The network device according to claim 18, wherein:

the obtaining circuit is further configured to obtain the attribute information of the N touch-controllable devices; and

the determining circuit is further configured to store the attribute information of the N touch-controllable devices in the hardware database.

20. The network device according to claim 19, wherein:

the obtaining circuit is configured to receive the attribute information sent by each of the N touch-controllable devices;

or

the obtaining circuit is configured to receive the identifier sent by each of the N touch-controllable devices, and obtain, according to the identifiers of the N touch-controllable devices, the visual features of the N touch-controllable devices, the APIs of the N touch-controllable devices, and the control UIs of the N touch-controllable devices from a server that stores the attribute information of the N touch-controllable devices.

21. The network device according to claim 16, wherein the obtaining circuit is configured to receive the identifier of each touch-controllable device sent by a video capture device.

22. The network device according to claim 16, wherein the obtaining circuit is further configured to receive first adjustment information sent by the remote device, wherein the first adjustment information is used to adjust an adjustable running parameter of a first touch-controllable device, and the at least one touch-controllable device comprises the first touch-controllable device;

and

the sending circuit is further configured to send the first adjustment information to the first touch-controllable device, so that the first touch-controllable device adjusts the adjustable running parameter according to the first adjustment information.

23. A device for controlling a touch-controllable device, wherein the device comprises:

a receiving circuit, configured to receive a video image and an identifier of each touch-controllable device that are sent by a control center device; and

a display circuit, configured to present the video image and the identifier of each touch-controllable device on a display interface.

24. The device according to claim 23, wherein:

the device further comprises an obtaining circuit, wherein the obtaining circuit is configured to obtain first input, wherein the first input is used to select a first touch-controllable device, and the touch-controllable device in the video image comprises the first touch-controllable device;

the obtaining circuit is further configured to obtain a control UI of the first touch-controllable device and an API of the first touch-controllable device; and

the display circuit is further configured to display the control UI of the first touch-controllable device on the display interface.

25. The device according to claim 24, wherein:

the obtaining circuit is further configured to obtain second input that is used to adjust an adjustable running parameter of the first touch-controllable device; and

the device further comprises a sending circuit, wherein the sending circuit is configured to send first adjustment information to the control center device, and the first adjustment information is used to adjust the adjustable running parameter of the first touch-controllable device.