User interface device of remote control system for robot device and method using the same

• US 9,757,861 B2
• Filed: 04/06/2015
• Issued: 09/12/2017
• Est. Priority Date: 12/18/2007
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Hong, Soon Hyuk, Kim, Gi Oh, Park, Jong Ho, Park, Ji Hyun, Baek, Seung Hyun
• Primary Examiner(s): Holwerda, Stephen
• Application Number:
  US14/679,127
• Publication Number:
  US 20150209962A1
• Time in Patent Office: 890 Days
• Field of Search: 31856812, 31856816, 31856825, 348162, 348163, 348164, 700245, 700258, 700259
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/40161 Visual display of machining, operation, remote viewing
• CPC Class Codes: G05B 2219/40165 Sensor data to display depends on robot status

Abstract

A user interface device of a remote control system for a robot and a method using the same are provided. The user interface device includes: a radio frequency (RF) unit for receiving, from a remote control robot, camera data and at least one sensor data detecting a distance; a display unit having a main screen and at least one auxiliary screen; and a controller having an environment evaluation module for determining whether the received camera data are in a normal condition, and having a screen display mode change module for displaying, if the received camera data are in a normal condition, the camera data on the main screen and displaying, if the received camera data are in an abnormal condition, the sensor data on the main screen.

Claims

1. A user interface device comprising:

a radio frequency (RF) unit for receiving, from an external device, camera data and sensor data from an ultrasonic sensor and an infrared sensor that output a remote location environment information;

a display unit having a main screen, a first auxiliary screen and a second auxiliary screen; and

a controller configured to analyze the remote location environment information of the external device and select from among camera data, ultrasonic data and infrared data to be displayed on the main screen in accordance with a determination as to whether or not the received camera data are in a normal condition for the remote location environment, and control display of the camera data on the main screen and infrared data on one of the first auxiliary screen or second auxiliary screen when the received camera data are in the normal condition, and control changing of the display on the main screen to display a received sensor data from an ultrasonic sensor on the main screen in response to a determination by the controller that the received camera data are not in the normal condition for the remote location environment when the controller determines the remote location environment is too bright or too dark based on compared light intensity values with one or more reference values,

wherein when the ultrasonic sensor data are displayed on the main screen, in response to an obstacle being undetected by the ultrasonic sensor and detected by the infrared sensor, the controller displays sensor data from the infrared sensor on the main screen, and

the user interface device outputs control instructions to the external device.

2. The user interface device of claim 1, wherein the external device is a remote control robot comprising:

a camera;

an infrared sensor;

a sensor unit that outputs sensor data regarding a detected distance;

a data acquisition module configured to acquire data from the camera and the sensor unit; and

a radio frequency (RF) unit configured to transmit the data acquired by the data acquisition module to the user interface device,

wherein when the received camera data are not in the normal condition, the display of the received camera data is shifted to the first auxiliary screen when the display on the main screen is changed to display the received sensor data in place of camera data.

3. The user interface device of claim 1, wherein the controller includes an environment evaluation module that determines that the received camera data are in the normal condition for the remote location environment information when a brightness of the camera data is within a range between a first reference value and a second reference value for setting a brightness limit.

4. The user interface device of claim 1, wherein the controller includes an environment evaluation module that determines that the received camera data are not in the normal condition when the camera data is an error or when the camera data does not receive data from the external device.

5. The user interface device of claim 2, wherein the controller includes a screen display mode change module that displays infrared data on the main screen when the camera data are in the normal condition for the remote location environment information, and when the camera data are not in the normal condition, ultrasonic sensor data are displayed on the main screen.

6. The user interface device of claim 1, wherein the radio frequency (RF) unit for receiving, from the external device, the camera data and sensor data having the remote location environment information, the received camera data and the sensor data are transmitted through a server or base-station from the external device.

7. A user interface method, comprising:

receiving, from an external device, camera data and sensor data having a remote location environment information;

determining whether the received camera data are in a normal condition for the remote location environment information;

displaying, when the received camera data are in the normal condition, the camera data on a main screen of a display unit having at least two of a main screen, a first auxiliary screen and a second auxiliary screen; and

displaying, when the received camera data are not in the normal condition, ultrasonic sensor data sensed from an ultrasonic sensor on the main screen,

wherein determining whether the received camera data are in the normal condition for the remote location environment information includes evaluating the remote location environment information of the external device, and changing a display on the main screen from the camera data to the ultrasonic sensed data, and

wherein when the ultrasonic sensor data are displayed on the main screen, in response to an obstacle being undetected by the ultrasonic sensor and detected by an infrared sensor, displaying sensor data from the infrared sensor on the main screen, and

outputting control instructions to the external device.

8. The user interface method of claim 7, wherein the external device is a remote control robot comprising:

a camera;

a sensor unit having the ultrasonic sensor for detecting a distance;

a data acquisition module for acquiring data from the camera and the sensor unit; and

an RF unit for transmitting the data acquired by the data acquisition module to a user interface device.

9. The user interface method of claim 7, wherein the normal condition of received camera data for the remote location environment information comprises a range of light intensity between a first reference value for setting a brightness limit and a second reference value for setting a darkness limit.

10. The user interface method of claim 8, wherein the sensor unit further comprises an infrared sensor that outputs infrared sensor data.

11. The user interface method of claim 10, wherein the sensor data further comprises infrared sensor data.

12. The user interface method of claim 11, further comprising:

detecting an obstacle by the infrared sensor; and

displaying, if the camera data are not displayed on the main screen, the infrared sensor data on the main screen when the remote location environment information is determined to be within a predetermined light intensity value range.

13. The user interface method of claim 9, further comprising:

detecting an obstacle by the ultrasonic sensor; and

displaying, if the camera data are not displayed on the main screen, an ultrasonic sensor data on the main screen.

14. The user interface method of claim 8, wherein the received camera data and the ultrasonic sensed data are transmitted through a server or base-station from the external device.

15. A method of providing a user interface comprising:

receiving camera data and sensor data having a remote location environment information from an external device;

determining whether the received camera data are in a normal condition;

determining that the received camera data are not in the normal condition when the camera data are an error or when the camera data do not receive data from the external device; and

displaying the camera data on a screen if the received camera data are in the normal condition, and displaying the received sensor data on the screen comprising an ultrasonic sensed data from an ultrasonic sensor if the received camera data are not in the normal condition or when the camera data are not received from the external device,

wherein the determining whether the received camera data are in the normal condition includes evaluating the remote location environment information of the external device, and wherein displaying the received sensor data includes changing a display on a main screen from the camera data to the ultrasonic sensed data,

wherein when the ultrasonic sensed data are displayed on the main screen, in response to an obstacle being undetected by the ultrasonic sensor and detected by an infrared sensor, displaying sensor data from the infrared sensor on the main screen, and

outputting control instructions to the external device.