Apparatus and method for unmanned surveillance, and robot control device for unmanned surveillance

• US 8,977,409 B2
• Filed: 12/06/2012
• Issued: 03/10/2015
• Est. Priority Date: 01/30/2012
• Status: Expired due to Fees

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Lee, Chang Eun, Cho, Hyun Kyu, Kim, Sung Hoon, Seo, Beom-Su
• Primary Examiner(s): Cheung, Mary
• Assistant Examiner(s): BUTLER, RODNEY ALLEN
• Application Number:
  US13/706,402
• Publication Number:
  US 20130197718A1
• Time in Patent Office: 824 Days
• Field of Search: 701/2, 701 23- 28, 701/439, 701/207, 180/167, 901/1, 348/113, 348/114, 700/245, 700/249, 700/257
• US Class Current: 701/2
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

An unmanned surveillance device includes a robot control terminal configured to be loaded within a remote control robot under a surveillance environment, collect state information and surrounding circumstance information, operate the remote control robot in driving mode or surveillance mode according to a remote control command corresponding to the state information and surrounding circumstance information. Further, the unmanned surveillance device includes a remote control system configured to receive the state information and the surrounding circumstance information of the remote control robot from the robot control terminal, output the received state information and surrounding circumstance information of the remote control robot, and provide the remote control command to the robot control terminal.

Claims

1. A robot control terminal for unmanned surveillance, comprising:

a robot state information processor unit configured to perform a function of creating space infrastructure for driving of a robot, and collect and process the state information of the robot;

a robot autonomous driving unit including instructions executable by a processor to provide location awareness information for autonomous driving of the robot according to a result of processing the state information processed by the robot state information processing unit, and provide an optimal path for the autonomous driving of the robot;

a remote control command processor unit including instructions executable by a processor to process surveillance information for remote control of the robot while being connected with the robot autonomous driving unit, and process behavior control of the robot based on the remote control; and

a user mission performance support unit including instructions executable by a processor to process image information for the remote control of a user while being connected to the remote control command processor unit, and transmit situation awareness information relating to a correctional facility situation to an operation system.

2. The robot control terminal of claim 1, wherein the robot autonomous driving unit provides a function of reasoning and searching for the autonomous driving of the robot.

3. The robot control terminal of claim 1, wherein the robot autonomous driving unit provides a function of modeling and awareness of an environment.

4. The robot control terminal of claim 1, wherein the robot control terminal includes a function of image-based context awareness.

5. An unmanned surveillance device, comprising:

a robot control terminal configured to be loaded within a remote control robot under a surveillance environment, collect state information and surrounding circumstance information, operate the remote control robot in driving mode or surveillance mode according to a remote control command corresponding to the state information and surrounding circumstance information; and

a remote control system configured to receive the state information and the surrounding circumstance information of the remote control robot from the robot control terminal, output a notification relating to an abnormal behavior pattern associated with a correctional environment based on the received state information and surrounding circumstance information of the remote control robot, and provide the remote control command to the robot control terminal;

wherein the robot control terminal comprises a robot autonomous driving unit including instructions executable by a processor to perform a reasoning function responsive to the abnormal behavior pattern associated with the correctional environment.

6. The unmanned surveillance device of claim 5, wherein the driving mode includes at least one of a standby mode, a semiautonomous driving mode, an autonomous driving mode, a return mode, a manual control mode, or an emergency stop mode.

7. A method for unmanned surveillance using a remote control system, the method comprising:

using a robot control terminal loaded in a robot under unmanned surveillance environment to adaptively track an object based on a human context;

further using the robot control terminal to detect the object based on fusion of features after adaptively tracking the object;

controlling a three-dimensional camera for extracting geometric object information after detecting the object based on the fusion of the features;

analyzing and perceiving an abnormal situation context according to a result of the controlling the three dimensional camera; and

generating an alarm regarding an abnormal situation relating to human behavior according to a result of the analyzing and perceiving the abnormal situation context;

wherein the analyzing and perceiving the abnormal situation context comprises performing a real-time analysis of a movement pattern.

8. The method of claim 7, wherein said adaptively tracking the object includes:

tracking the object based on complex features; and

modeling backgrounds based on entropy for changing a background, including lighting.

9. The method of claim 7, wherein said detecting the object based on fusion of features includes:

modeling and matching based on the complex features; and

performing segmentation based on adaptive morphology.

10. The method of claim 7, wherein said controlling the three-dimensional camera includes:

managing a three-dimensional multi-object profile; and

extracting three-dimensional features for extracting geometric information.

11. The method of claim 7, wherein said analyzing and perceiving the abnormal situation context includes:

fusing and analyzing multiple objects feature information and a statistics-based prediction model for dynamic emergency events modeling; and

perceiving abnormal behavior context.

12. The method of claim 7, wherein the analyzing and perceiving the abnormal situation context according to the result of the controlling the three dimensional camera includes:

arriving at a mission point inside of a room;

performing autonomous driving inside of the room according to one of predetermined patterns;

acquiring an inside image of the room based on the autonomous driving; and

based on the acquired inside image, checking whether an abnormal situation exists based on a behavior pattern of at least one prisoner inside the room.

13. The method of claim 8, wherein the complex features include at least one of a correction filter, a color, a context or an edge.