Remote presence system including a cart that supports a robot face and an overhead camera

• US 10,887,545 B2
• Filed: 03/20/2017
• Issued: 01/05/2021
• Est. Priority Date: 03/04/2010
• Status: Active Grant

• Current Assignee: Teladoc Health, Inc.
• Original Assignee: Teladoc Health, Inc.
• Inventors: Stuart, David, Sanchez, Daniel Steven, Lai, Fuji, Hanrahan, Kevin, Jordan, Charles S., Roe, David, Rosenthal, James, Mangaser, Amante, Whitney, Blair, Walters, Derek
• Primary Examiner(s): Figueroa, Jaime
• Application Number:
  US15/464,164
• Publication Number:
  US 20170195623A1
• Time in Patent Office: 1,387 Days
• Field of Search: 700245, 700257
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/185 from a mobile camera, e.g. for remote control

Abstract

A tele-presence system that includes a cart. The cart includes a robot face that has a robot monitor, a robot camera, a robot speaker, a robot microphone, and an overhead camera. The system also includes a remote station that is coupled to the robot face and the overhead camera. The remote station includes a station monitor, a station camera, a station speaker and a station microphone. The remote station can display video images captured by the robot camera and/or overhead camera. By way of example, the cart can be used in an operating room, wherein the overhead camera can be placed in a sterile field and the robot face can be used in a non-sterile field. The user at the remote station can conduct a teleconference through the robot face and also obtain a view of a medical procedure through the overhead camera.

Claims

1. A cart for use in a surgical mentoring system, the cart comprising:

a first camera;

a second camera;

a speaker;

a microphone; and

an input device,

wherein the second camera is coupled to the cart via a boom that includes a plurality of joints configured to allow a distance between a first end of the boom and a second end of the boom to be varied,

wherein the cart is configured to be coupled to a remote station via a network, the remote station including a camera, a monitor, and microphone, and a speaker, wherein the remote station is configured to display a user interface on the station monitor, the user interface includes a remote view field, a station view field, and a plurality of graphical controls,

wherein the plurality of controls include one or more controls to selectively display video from the first camera or the second camera in the remote view field, one or more controls to display a still image captured by the first camera or the second camera in the remote view field, one or more controls to transfer an image to the cart monitor for display, and one or more controls to annotate an image displayed on the cart monitor from the remote station,

wherein the station view field displays an image captured by the station camera, the station speaker reproduces a sound captured by the cart microphone, the station microphone captures a sound that is transmitted to and reproduced by the cart speaker, and

wherein the orientation of the first camera and the orientation of the second camera can be independently controlled from the remote station.

2. The system of claim 1, wherein the one or more controls includes a switch that allows the user to select simultaneous display of video images from the first and second cameras.

3. The system of claim 2, wherein the video images from both the first camera and the second camera are streamed to the remote station from the cart.

4. The system of claim 1, wherein a height of the first camera is less than a height of the second camera.

5. The system of claim 1, wherein the boom includes a handle that can be used to move the second camera.

6. The system of claim 1, wherein the cart includes a processor and a hard drive.

7. The system of claim 1, wherein cart includes an input panel that allows a user to provide input.

8. The system of claim 1, wherein the user interface includes a location display that provides the location of the cart.

9. The system of claim 1, wherein the system includes a plurality of carts and the user interface includes a graphical input that can be used to select and control a second cart.

10. The system of claim 1, wherein said cart includes a second microphone.

11. The system of claim 3, wherein the one or more controls includes a control that allows a user to switch between the first microphone and the second microphone.

12. A method for surgical mentoring, comprising:

displaying a user interface on a monitor of a remote station, the user interface including a remote view field, a station view field, and a plurality of graphical controls;

establishing a communication session with a cart in proximity to a surgical field, the cart including a first camera, a second camera, a monitor, a microphone, a speaker, and an input device, wherein the second camera is coupled to the cart via a boom that includes a plurality of joints configured to allow a distance between a first end of the boom and a second end of the boom to be varied;

selectively displaying video from the first camera and the second camera in the remote view field in response to user input at the remote station;

displaying a still image captured by one of the first camera or the second camera in the remote view field in response to user input at the remote station;

transferring an image to the cart monitor for display in response to user input at the remote station;

annotating an image displayed on the cart monitor in response to user input at the remote station;

displaying, in the station view field, an image captured by the station camera on the station monitor;

reproducing, by the station speaker, a sound captured by the cart microphone;

capturing, by the station microphone, a sound that is transmitted to and reproduced by the cart speaker; and

controlling the orientation of the first camera and the orientation of the second camera from the remote station.

13. The method of claim 12 further comprising simultaneously displaying video images from the first and second cameras at the remote station.

14. The method of claim 12 further comprising receiving at the remote station video streams from both the first and second cameras.

15. The method of claim 12 further comprising receiving input from an input panel of the cart at the remote station.

16. The method of claim 13, further comprising displaying a location display at the remote station that provides the location of the cart.

17. The method of claim 13, further comprising displaying at the remote station a graphical input that can be selected to select and control a second cart.

18. The method of claim 13, further comprising controlling the zoom of the first camera from the remote station in response to user input.

19. The method of claim 12 further comprising controlling the zoom of the second camera from the remote station in response to user input.

20. A non-transitory computer-readable medium comprising program code that, when executed by one or more processors, causes the one or more processors to perform a method for automated health condition scoring, the method comprising:

displaying a user interface on a monitor of a remote station, the user interface including a remote view field, a station view field, and a plurality of graphical controls;

establishing a communication session with a cart in proximity to a surgical field, the cart including a first camera, a second camera, a monitor, a microphone, a speaker, and an input device, wherein the second camera is coupled to the cart via a boom that includes a plurality of joints configured to allow a distance between a first end of the boom and a second end of the boom to be varied;

selectively displaying video from the first camera and the second camera in the remote view field in response to user input at the remote station;

displaying a still image captured by one of the first camera or the second camera in the remote view field in response to user input at the remote station;

transferring an image to the cart monitor for display in response to user input at the remote station;

annotating an image displayed on the cart monitor in response to user input at the remote station;

displaying, in the station view field, an image captured by the station camera on the station monitor;

reproducing, by the station speaker, a sound captured by the cart microphone;

capturing, by the station microphone, a sound that is transmitted to and reproduced by the cart speaker; and

controlling the orientation of the first camera and the orientation of the second camera from the remote station.