Methods and apparatus for documenting a procedure

• US 8,907,914 B2
• Filed: 08/31/2012
• Issued: 12/09/2014
• Est. Priority Date: 08/31/2012
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Venon, Medhi
• Primary Examiner(s): Cheng, Joe H
• Application Number:
  US13/601,790
• Publication Number:
  US 20140062851A1
• Time in Patent Office: 830 Days
• Field of Search: 345/156, 345/173, 178/18.01, 178/18.09, 340/573.1, 715/863
• US Class Current: 345/173
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L

Abstract

Methods and apparatus are disclosed for documenting a procedure in a touchless environment. Example methods disclosed herein include identifying a position of one or more of a user'"'"'s fingertips using a camera of a mobile device and an entropy-based segmentation of an image input, determining a correspondence between user fingertip position and a control interface of the mobile device, including a plurality of functions to be executed based on user input, triggering the mobile device to execute one or more of the plurality of functions based on the user fingertip position during a procedure, and documenting one or more events of the procedure based on the user input for generating a report to be stored in a storage device of the mobile device.

Claims

1. A documenting procedure method comprising:

identifying a position of one or more of a user'"'"'s fingertips in a virtual plane formed apart from a control interface of a mobile device using a camera of the mobile device and an entropy-based segmentation of an image input from the camera;

determining a correspondence between user fingertip position and the control interface of the mobile device, the correspondence forming a user input, the control interface including a plurality of functions to be executed based on the user input;

facilitating execution, via the mobile device, of one or more of the plurality of functions based on the user fingertip position in the virtual plane during a procedure; and

documenting one or more events of the procedure based on the user input to generate a report to be output via the mobile device.

2. A method according to claim 1, wherein the virtual plane is separate from and corresponds to the control interface of the mobile device, and

wherein first coordinates of the virtual plane correspond to second coordinates of the control interface of the mobile device.

3. A method according to claim 1, further comprising detecting a device identification in the image input using the camera of the mobile device;

determining device information based on the device identification; and

documenting the device information in the report as one of the one or more events of the procedure.

4. A method according to claim 1, further comprising storing at least one of an audio recording or video recording of the procedure using at least one of a microphone or the camera based on a user command identified by the user fingertip position;

and

associating the recording of the procedure with the report.

5. A method according to claim 1, wherein the user'"'"'s fingertip position is to be identified at a first time when one or more gloves covering the user'"'"'s fingertips are substantially clear of debris and at a second time when the one or more gloves covering the user'"'"'s fingertips are substantially covered in debris.

6. A method according to claim 1, wherein documenting the one or more events further comprises converting documentation data from the execution of the one or more functions to a format based on at least one of a user setting or data type.

7. A method according to claim 1, wherein the mobile device is at least one of a tablet computer, smartphone, personal digital assistant, personal computer, or laptop computer.

8. An apparatus for documenting procedure comprising:

an image filter to identify a position of one or more of fingertips in a virtual plane formed apart from a control interface of a mobile device using a camera of the mobile device and an entropy-based segmentation of an image input from the camera;

a user command identifier to determine a correspondence between user fingertip position and the control interface of the mobile device, the correspondence forming a user input, the control interface including a plurality of functions to be executed based on the user input;

a controller to facilitate execution, via the mobile device, of one or more of the plurality of functions based on the user fingertip position in the virtual plane during a procedure; and

a documenter to document one or more events of the procedure based on the user input to generate a report to be output via the mobile device.

9. An apparatus according to claim 8, wherein the virtual plane is separate from and corresponds to the control interface of the mobile device, and

wherein first coordinates of the virtual plane correspond to second coordinates of the control interface of the mobile device.

10. An apparatus according to claim 8, further comprising a device identification detector to detect a device identification in the image input using the camera of the mobile device;

and

a device identifier to determine device information based on the device identification,

wherein the documenter documents the device information in the report as one of the one or more events of the procedure.

11. An apparatus according to claim 8, wherein the controller stores at least one of an audio recording or video recording of the procedure using at least one of a microphone or the camera based on a user command identified by the user fingertip position and the documenter associates the recording of the procedure with the report.

12. An apparatus according to claim 8, wherein the user'"'"'s fingertip position is identified at a first time when one or more gloves covering the user'"'"'s fingertips are substantially clear of debris and at a second time when the one or more gloves covering the user'"'"'s fingertips are substantially covered in debris.

13. An apparatus according to claim 8, wherein documenting the one or more events further comprises converting documentation data from the execution of the one or more functions to a format based on at least one of a user setting or data type.

14. An apparatus according to claim 8, wherein the mobile device is at least one of a tablet computer, smartphone, personal digital assistant, personal computer, or laptop computer.

15. A non-transitory machine readable storage medium comprises instructions which, when executed, cause a machine to at least:

identify a position of one or more of a user'"'"'s fingertips in a virtual plane formed apart from a control interface of a mobile device using a camera of the mobile device and an entropy-based segmentation of an image input from the camera;

determine a correspondence between user fingertip position and the control interface of the mobile device, the correspondence forming a user input, the control interface including a plurality of functions to be executed based on the user input;

facilitate execution, via the mobile device, of one or more of the plurality of functions based on the user fingertip position in the virtual plane during a procedure; and

document one or more events of the procedure based on the user input to generate a report to be output via the mobile device.

16. A non-transitory machine readable storage medium according to claim 15, wherein the virtual plane is separate from and corresponds to the control interface of the mobile device, and

wherein first coordinates of the virtual plane correspond to second coordinates of the control interface of the mobile device.

17. A non-transitory machine readable storage medium according to claim 15, wherein the instructions cause the machine to detect a device identification in the image input using the camera of the mobile device;

determine device information based on the device identification; and

document the device information in the report as one of the one or more events of the procedure.

18. A non-transitory machine readable storage medium according to claim 15, wherein the instructions cause the machine to store at least one of an audio recording or video recording of the procedure using at least one of a microphone or the camera based on a user command identified by the user fingertip position;

and

associate the recording of the procedure with the report.

19. A non-transitory machine readable storage medium according to claim 15, wherein the user'"'"'s fingertip position is identified at a first time when one or more gloves covering the user'"'"'s fingertips are substantially clear of debris and at a second time when the one or more gloves covering the user'"'"'s fingertips are substantially covered in debris.

20. A non-transitory machine readable storage medium according to claim 15, wherein documenting the one or more events further comprises converting documentation data from the execution of the one or more functions to a format based on at least one of a user setting or data type.