Interfacing with a mobile telepresence robot

• US 10,399,223 B2
• Filed: 03/01/2017
• Issued: 09/03/2019
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: iRobot Corporation, Jonata Sub Two, Inc. (Teladoc Health, Inc.), Teladoc Health, Inc.
• Original Assignee: Touch Technologies Incorporated
• Inventors: Wang, Yulun, Jordan, Charles S., Wright, Tim, Chan, Michael, Pinter, Marco, Hanrahan, Kevin, Sanchez, Daniel, Ballantyne, James, Herzog, Cody, Whitney, Blair, Lai, Fuji, Temby, Kelton, Rauhut, Eben Christopher, Kearns, Justin H., Wong, Cheuk Wah, Farlow, Timothy Sturtevant
• Primary Examiner(s): Rink, Ryan
• Application Number:
  US15/446,919
• Publication Number:
  US 20170334069A1
• Time in Patent Office: 916 Days
• CPC Class Codes: B25J 11/009 Nursing, e.g. carrying sick persons, pushing wheelchairs, distributing drugs
• CPC Class Codes: B25J 11/0095 Manipulators transporting wafers
• CPC Class Codes: B25J 5/00 Manipulators mounted on wheels or on carriages B25J1/00 takes precedence; programme-controlled manipulators B25J9/00 ; vehicle aspects B60, B62, e.g. remote-controlled steering for motor vehicles B62D1/24; control of position of vehicles G05D1/00
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A telepresence robot may include a drive system, a control system, an imaging system, and a mapping module. The mapping module may access a plan view map of an area and tags associated with the area. In various embodiments, each tag may include tag coordinates and tag information, which may include a tag annotation. A tag identification system may identify tags within a predetermined range of the current position and the control system may execute an action based on an identified tag whose tag information comprises a telepresence robot action modifier. The telepresence robot may rotate an upper portion independent from a lower portion. A remote terminal may allow an operator to control the telepresence robot using any combination of control methods, including by selecting a destination in a live video feed, by selecting a destination on a plan view map, or by using a joystick or other peripheral device.

Claims

1. A telepresence robot comprising:

a drive system configured to move the telepresence robot about a facility according to one or more drive instructions;

a control system in communication with the drive system, the control system configured to generate the one or more drive instructions;

an imaging system in communication with the control system;

a mapping module in communication with the control system, the mapping module configured to access a map data source, the map data source comprising;

a robot map representative of robot-navigable areas of the facility;

at least one tag that identifies a region within the robot-navigable areas of the facility and a behavior to be executed by the robot; and

a destination list including destination locations within the facility, each destination location comprising spatial coordinates locatable relative to the robot map;

a positioning system in communication with the control system configured to provide positioning information associated with a current position;

a communication system configured to provide a video feed from the imaging system to a remote terminal and receive control commands from the remote terminal, and wherein the control system is configured to generate the one or more drive instructions according to a control command received from the remote terminal, wherein the control command is generated by the remote terminal in response to a user input at the remote terminal selecting a movement of the telepresence robot from the current position to a destination location via one of a plurality of options on, the options comprising;

selecting a velocity-based control specifying a direction of the movement of the telepresence robot;

selecting a destination location of the telepresence robot with respect to the destination list;

selecting a destination location of the telepresence robot with respect to the video feed, and,

a navigation system configured to generate a navigation path from the current position to the destination location, wherein the drive system moves the robot along the navigation path and the robot behaves according to the tag behavior when the current position of the robot is within the tag region.

2. The telepresence robot of claim 1, further comprising:

a plurality of sensors configured to identify obstacles in the vicinity of the telepresence robot; and

an obstacle avoidance system in communication with the plurality of sensors and in communication with the control system, wherein the control system is further configured to generate additional drive instructions to avoid obstacles in the vicinity of the telepresence robot.

3. The telepresence robot of claim 2, wherein the plurality of sensors comprises a three-dimensional image sensor that forms a point cloud, including a three-dimensional occupancy of obstacles, and wherein the drive instructions are configured to avoid the three-dimensional occupancy of the obstacles.

4. The telepresence robot of claim 3, further comprising:

a map generation system in communication with the control system, the map generation system configured to autonomously create the robot map, and wherein the control system generates drive instructions to cause the telepresence robot to move throughout the facility and obtain a plurality of measurements, and the map generation system uses the plurality of measurements to generate the robot map.

5. The telepresence robot of claim 2, further comprising a navigation system configured to generate a navigation path, the navigation path comprising a sequence of coordinates from the current position on the robot map to the destination location on the robot map.

6. The telepresence robot of claim 5, wherein the sequence of coordinates forming the navigation path is based at least in part on tag information associated with a tag associated with the robot map.

7. The telepresence robot of claim 5, wherein the navigation system is configured to identify at least one tag relevant to the navigation path of the robot and generate the navigation path by selecting a navigation path from a plurality of potential navigation paths, and wherein the at least one tag relevant to the navigation path of the telepresence robot is associated with at least one of the plurality of potential navigation paths, and wherein the navigation system is configured to select the navigation path based at least in part on the at least one identified relevant tag.

8. The telepresence robot of claim 5, wherein the sequence of coordinates forming the navigation path is transmitted via the communication system to the remote terminal.

9. The telepresence robot of claim 5, wherein the telepresence robot is configured to create a new tag using the sequence of coordinates forming the navigation path, such that the new tag comprises the sequence of coordinates, and tagging information related to the navigation path.

10. The telepresence robot of claim 2, wherein the control system is further configured to transmit coordinates relative to the robot map of a detected obstacle to the remote terminal via the communication system.

11. The telepresence robot of claim 2, wherein the control system is further configured to receive a navigation path from the current position on the robot map to the destination location on the robot map, and wherein the control system is further configured to generate drive instructions to cause the drive system to move the telepresence robot to the destination location based on the navigation path.

12. The telepresence robot of claim 2, wherein the communication system is configured to detect a disruption in communication between the telepresence robot and a remote terminal, and wherein the control system is further configured to continue to generate drive instructions to cause the telepresence robot to autonomously move to the destination location during the disruption in communication.

13. The telepresence robot of claim 6, wherein the tag information relates to at least one of a position, a path, and a volume, and wherein the control system is configured to execute an action relative to the at least one of the position, the path, and the volume.

14. The telepresence robot of claim 1, wherein the map data source is stored remotely, such that the mapping module is configured to access the map data source via the communication system.

15. The telepresence robot of claim 1, wherein the map data source is stored within the telepresence robot, such that the mapping module is configured to access an internal map data source.

16. The telepresence robot of claim 1, wherein the internal map data source is synced with a remotely stored map data source.

17. The telepresence robot of claim 1, wherein the positioning system is further configured to provide a robot pose relative to the robot map.

18. The telepresence robot of claim 6, wherein the telepresence robot is configured to create a new tag by:

associating tag coordinates describing the relative location of the new tag with respect to one of the robot map and the video feed generated by the imaging system; and

associating tag information with the new tag.

19. The telepresence robot of claim 18, wherein the new tag is created in response to the telepresence robot detecting an object in the video feed.

20. The telepresence robot of claim 19, wherein the object is a person and the tag information of the new tag comprises identification information associated with the person.