Touchless interface for a medical treatment system

• US 10,067,569 B2
• Filed: 08/14/2015
• Issued: 09/04/2018
• Est. Priority Date: 08/14/2015
• Status: Active Grant

• Current Assignee: Fresenius Medical Care Holdings Incorporated (Fresenius SE & Co. KGaA)
• Original Assignee: Fresenius Medical Care Holdings Incorporated (Fresenius SE & Co. KGaA)
• Inventors: Tarn, Jeffrey, Wang, Fei, Tanenbaum, Lee Daniel
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US14/826,513
• Publication Number:
  US 20170045947A1
• Time in Patent Office: 1,117 Days
• CPC Class Codes: A61M 1/1601 Control or regulation
• CPC Class Codes: A61M 2205/33 Controlling, regulating or measuring
• CPC Class Codes: A61M 2205/502 User interfaces, e.g. screens or keyboards
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G16H 20/40 relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A dialysis machine comprising: one or more processing units configured to transmit control data; a pump configured to pump medical fluid to and from a patient based at least in part on control data received from the processing unit; an electronic panel comprising: a display surface, and at least one panel control unit configured to cause the electronic panel to display at least one user interface element that can be invoked by a user; at least one projector; and at least one camera; wherein the one or more processing units are configured to: process input received by the camera, determine a location of a physical object in a field of view of the camera based on the processed input, determine, based on processed input received on at least one occasion, that the location of the physical object represents an invocation of the at least one user interface element displayed on the electronic panel, and determine the control data based on the processed input.

Claims

1. A dialysis machine comprising:

one or more processing units configured to transmit control data;

a pump configured to pump medical fluid to and from a patient based at least in part on control data received from the processing unit;

an electronic panel comprising;

a display surface, and

at least one panel control unit configured to cause the electronic panel to display at least one user interface element that can be invoked by a user;

at least one projector configured to emit light; and

at least one camera configured to detect the emitted light;

wherein the one or more processing units are configured to;

process input received by the camera,

determine a location of a physical object in a field of view of the camera based on the processed input, including;

identifying a segment of light projected onto the physical object by the projector;

constructing an image that includes a representation of the segment of light, the representation comprising a plurality of pixels each having coordinates in the image, wherein each pixel is defined by at least a u-coordinate value representing a horizontal position in the image and a v-coordinate value representing a vertical position in the image;

averaging the coordinates of the plurality of pixels to determine a location of the physical object in the image; and

determining the location of the physical object in the field of view of the camera based on the determined location of the physical object in the image, comprising calculating an x-coordinate value, a y-coordinate value, and a z-coordinate value of the physical object, wherein the x, y, and z-coordinate values are each determined based on one or more of the following;

one or more of the u-coordinate values, one or more of the v-coordinate values, a focal length of the camera in pixels, and a distance between the projector and the camera;

determine that a width of the physical object is less than a predetermined threshold,

determine, based on the determined location of the physical object in the field of view of the camera and the determination that the width of the physical object is less than the predetermined threshold, that the determined location of the physical object represents an invocation of the at least one user interface element displayed on the electronic panel, and

determine the control data based on the processed input.

2. The dialysis machine of claim 1, wherein the at least one processor is configured to determine that the physical object is a physical object of interest.

3. The dialysis machine of claim 2, wherein the physical object is determined to be a physical object of interest based at least in part on the width of the physical object.

4. The dialysis machine of claim 2, wherein the physical object of interest is a finger of a human hand.

5. The dialysis machine of claim 1, wherein the projector emits a line of light, wherein the length of the line depends on a distance between a point in space and the projector.

6. The dialysis machine of claim 1 comprising four projectors and four cameras, wherein a first projector is positioned above the electronic panel, a second projector is positioned below the electronic panel, a third projector is positioned to a left side of the electronic panel, and a fourth projector is positioned to a right side of the electronic panel.

7. A method performed by one or more processors of a dialysis machine, the method comprising:

processing visual input;

determining a location of a physical object in a field of view based on the processed visual input, including;

identifying a segment of light projected onto the physical object;

constructing an image that includes a representation of the segment of light, the representation comprising a plurality of pixels each having coordinates in the image, wherein each pixel is defined by at least a u-coordinate value representing a horizontal position in the image and a v-coordinate value representing a vertical position in the image;

averaging the coordinates of the plurality of pixels to determine a location of the physical object in the image; and

determining the location of the physical object in the field of view based on the determined location of the physical object in the image, comprising calculating an x-coordinate value, a y-coordinate value, and a z-coordinate value of the physical object, wherein the x, y, and z-coordinate values are each determined based on one or more of the following;

one or more of the u-coordinate values, one or more of the v-coordinate values, a focal length of the camera in pixels, and a distance between the projector and the camera;

determining that a width of the physical object is less than a predetermined threshold; and

determining, based on the determined location of the physical object in the field of view and the determination that the width of the physical object is less than the predetermined threshold, that the determined location of the physical object represents an invocation of at least one invokable user interface element displayed by an electronic panel of the dialysis machine.

8. The method of claim 7, wherein the visual input includes information related to the segment of light that is projected onto the physical object.

9. The method of claim 7, wherein the light comprises infrared light.

10. The method of claim 7 comprising determining that the physical object is a physical object of interest.

11. The method of claim 10, wherein the physical object is determined to be a physical object of interest based at least in part on the width of the physical object.

12. The method of claim 10, wherein the physical object of interest is a finger of a human hand.

13. At least one non-transitory computer-readable medium storing instructions operable to cause one or more computer to perform operations comprising:

processing visual input;

determining a location of a physical object in a field of view based on the processed visual input, including;

identifying a segment of light projected onto the physical object;

constructing an image that includes a representation of the segment of light, the representation comprising a plurality of pixels each having coordinates in the image, wherein each pixel is defined by at least a u-coordinate value representing a horizontal position in the image and a v-coordinate value representing a vertical position in the image;

averaging the coordinates of the plurality of pixels to determine a location of the physical object in the image; and

determining the location of the physical object in the field of view based on the determined location of the physical object in the image, comprising calculating an x-coordinate value, a y-coordinate value, and a z-coordinate value of the physical object, wherein the x, y, and z-coordinate values are each determined based on one or more of the following;

one or more of the u-coordinate values, one or more of the v-coordinate values, a focal length of the camera in pixels, and a distance between the projector and the camera;

determining that a width of the physical object is less than a predetermined threshold; and

determining, based on the determined location of the physical object in the field of view and the determination that the width of the physical object is less than the predetermined threshold, that the determined location of the physical object represents an invocation of at least one invokable user interface element displayed by an electronic panel of a dialysis machine.