Mobile Security Robot

• US 20160188977A1
• Filed: 11/18/2015
• Published: 06/30/2016
• Est. Priority Date: 12/24/2014
• Status: Abandoned Application

• Current Assignee: iRobot Corporation
• Original Assignee: iRobot Corporation
• Inventors: Kearns, Justin H., Taka, Orjeta
• Application Number:
  US14/944,354
• Publication Number:
  US 20160188977A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 11/002 Manipulators for defensive or military tasks
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0272 comprising means for registering the travel distance, e.g. revolutions of wheels measuring distance traversed on the ground by vehicles, e.g. using odometers G01C22/00
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A method of operating a mobile robot includes receiving a layout map corresponding to a patrolling environment at a computing device and maneuvering the robot in the patrolling environment based on the received layout map. The method further includes receiving imaging data of a scene about the robot when the robot maneuvers in the patrolling environment at the computing device. The imaging data is received from one or more imaging sensors disposed on the robot and in communication with the computing device. The method further includes identifying a person in the scene based on the received imaging data and aiming a field of view of at least one imaging sensor to continuously perceive the identified person in the field of view. The method further includes capturing a human recognizable image of the identified person using the at least one imaging sensor.

Claims

1. A method of operating a mobile robot, the method comprising:

receiving, at a computing device, a layout map corresponding to a patrolling environment;

maneuvering the robot in the patrolling environment based on the received layout map;

receiving, at the computing device, imaging data of a scene about the robot when the robot maneuvers in the patrolling environment, the imaging data received from at least one imaging sensor disposed on the robot and in communication with the computing device;

identifying, by the computing device, a person in the scene based on the received imaging data;

aiming, by the computing device, a field of view of the at least one imaging sensor to continuously perceive the identified person in the field of view based on robot dynamics, person dynamics comprising a movement trajectory of the person, and imaging sensor dynamics of the at least one imaging sensor; and

capturing, by the computing device, a human recognizable image of the identified person using the at least one imaging sensor.

2. The method of claim 1, further comprising:

segmenting, by the computing device, the received imaging data into objects;

filtering, by the computing device, the objects to remove objects greater than a first threshold size comprising a first height of about 8 feet and smaller than a second threshold size comprising a second height of about 3 feet; and

identifying, by the computing device, the person in the scene corresponding to at least a portion of the filtered objects.

3. The method of claim 1, further comprising at least one of:

aiming, by the computing device, the at least one imaging sensor to maintain the corresponding aimed field of view on a facial region of the identified person;

or

commanding, by the computing device, holonomic motion of the robot to maintain the aimed field of view of the at least one imaging sensor on the facial region of the identified person.

4. The method of claim 1, further comprising using, by the computing device, a Kalman filter to track and propagate the movement trajectory of the identified person.

5. The method of claim 4, further comprising commanding, by the computing device, the robot to move in a planar direction with three planar degrees of freedom while maintaining the aimed field of view of the at least one imaging sensor on the identified person associated with the movement trajectory.

6. The method of claim 5, wherein the robot moves in the planar direction at a velocity proportional to the movement trajectory of the identified person.

7. The method of claim 4, further comprising commanding, by the computing device, at least one of panning or tilting the at least one imaging sensor to maintain the aimed field of view of the at least one imaging sensor on the identified person associated with the movement trajectory.

8. The method of claim 1, further comprising:

reviewing, by the computing device, the captured image to determine whether or not the identified person is perceived in a center of the image or the image is clear;

when the identified person is perceived in the center of the image and the image is clear;

storing the captured image in non-transitory memory in communication with the computing device; and

transmitting, by the computing device, the captured image to a security system in communication with the computing device; and

when the identified person is perceived outside the center of the image or the image is blurred;

re-aiming the field of view of the at least one imaging sensor to continuously perceive the identified person in the field of view; and

capturing a subsequent human recognizable image of the identified person using the at least one imaging sensor,

wherein the imaging sensor dynamics comprise a threshold rotational velocity of the imaging sensor relative to an imaging target to capture a clear image of the imaging target.

9. The method of claim 1, further comprising:

applying, by the computing device, a location tag to the captured image associated with a location of the identified person;

applying, by the computing device, a time tag associated with a time the image was captured; and

transmitting a tagged layout map from the computing device to a remote device.

10. The method of claim 9, wherein the location tag defines a location on the layout map.

11. The method of claim 1, wherein the at least one imaging sensor comprises at least one of a still-image camera, a video camera, a stereo camera, or a three-dimensional point cloud imaging sensor.

12. The method of claim 1, wherein the robot dynamics comprise:

a first acceleration/deceleration limit of a drive system of the robot;

a second acceleration/deceleration limit associated with a drive command; and

a deceleration limit associated with a stop command.

13. A robot comprising:

a robot body;

a drive system supporting the robot body and configured to maneuver the robot over a floor surface of a patrolling environment, the drive system having a forward drive direction;

at least one imaging sensor disposed on the robot body; and

a controller in communication with the drive system and the at least one imaging sensor, the controller;

receiving a layout map corresponding to a patrolling environment;

issuing drive commands to the drive system to maneuver the robot in the patrolling environment based on the received layout map;

receiving imaging data from the at least one imaging sensor of a scene about the robot when the robot maneuvers in the patrolling environment;

identifying a moving target in the scene based on the received imaging data;

propagating a movement trajectory of the identified moving target based on the received imaging data;

aiming a field of view of the at least one imaging sensor to continuously perceive the identified moving target in the field of view; and

capturing a human recognizable image of the identified moving target using the at least one imaging sensor.

14. The robot of claim 13, wherein the controller:

segments the received imaging data into objects;

filters the objects to remove objects greater than a first threshold size comprising a first height of about 8 feet and smaller than a second threshold size comprising a second height of about 3 feet; and

identifies a person in the scene as the identified moving target corresponding to at least a portion of the filtered objects.

15. The robot of claim 14, further comprising a rotator and a tilter disposed on the robot body in communication with the controller, the rotator and tilter providing at least one of panning and tilting of the at least one imaging sensor, wherein the controller at least one of:

commands the rotator or tilter to at least one of pan or tilt the at least one imaging sensor to maintain the corresponding aimed field of view on a facial region of the identified person;

or

issues drive commands to the drive system to holonomically move the robot to maintain the aimed field of view of the at least one imaging sensor on the facial region of the identified person.

16. The robot of claim 15, wherein the controller commands the drive system to drive in a planar direction with three planar degrees of freedom at a velocity proportional to the movement trajectory of the identified moving target while maintaining the aimed field of view of the at least one imaging sensor on the identified moving target associated with the movement trajectory.

17. The robot of claim 13, further comprising a rotator and a tilter disposed on the robot body and in communication with the controller, the rotator and tilter providing at least one of panning and tilting of the at least one imaging sensor, wherein the controller commands the rotator or the tilter to at least one of pan or tilt the at least one imaging sensor to maintain the aimed field of view of the at least one imaging sensor on the identified moving target associated with the movement trajectory, wherein the least one of the commanded panning or tilting is at a velocity proportional to the movement trajectory of the identified moving target and proportional to a planar velocity of the robot.

18. The robot of claim 13, wherein the controller reviews the captured image to determine whether the identified moving target is perceived in a center of the image or the image is clear;

when the identified moving target is perceived in the center of the image and the image is clear, the controller;

stores the captured image in non-transitory memory in communication with the controller; and

transmits the captured image to a security system in communication with the controller; and

when the identified moving target is perceived outside the center of the image or the image is blurred, the controller;

re-aims the field of view of the at least one imaging sensor continuously perceive the identified moving target in the field of view; and

captures a subsequent human recognizable image of the identified moving target using the at least one imaging sensor.

19. The robot of claim 13, wherein the controller:

applies a location tag to the captured image associated with a location of the identified moving target, the location tag defining a location on the layout map based on at least one of robot odometry, waypoint navigation, dead-reckoning, or a global positioning system; and

applies a time tag associated with a time the image was captured.

20. The robot of claim 13, wherein the at least one imaging sensor comprises at least one of a still-image camera, a video camera, a stereo camera, or a three-dimensional point cloud imaging sensor.

21. The robot of claim 13, wherein the controller aims the at least one imaging sensor based on acceleration/deceleration limits of the drive system and a latency between sending an image capture request to the at least one imaging sensor and the at least one imaging sensor capturing an image, wherein the acceleration/deceleration limits of the drive system comprise an acceleration/deceleration limit associated with a drive command and a deceleration limit associated with a stop command.

22. The robot of claim 21, wherein the controller determines a movement trajectory of the identified moving target and aims the at least one imaging sensor based on the movement trajectory of the identified moving target.