Medical technical system controlled by vision-detected operator activity

• US 6,175,610 B1
• Filed: 02/08/1999
• Issued: 01/16/2001
• Est. Priority Date: 02/11/1998
• Status: Expired due to Fees

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Peter, Fritz
• Primary Examiner(s): Bruce, David V.
• Assistant Examiner(s): Dunn, Drew A.
• Application Number:
  US09/245,869
• Time in Patent Office: 708 Days
• Field of Search: 378/8, 378/95, 345/435, 345/156, 345/166, 382/100, 250/221, 250/231.18
• US Class Current: 378/8
• CPC Class Codes: A61B 2017/00199 with a console, e.g. a control panel with a display
• CPC Class Codes: A61B 2017/00207 with hand gesture control or hand gesture recognition
• CPC Class Codes: A61B 2017/00367 Details of actuation of instruments, e.g. relations between pushing buttons, or the like, and activation of the tool, working tip, or the like
• CPC Class Codes: A61B 6/00 Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment instruments measuring radiation intensity for application in the field of nuclear medicine, e.g. in vivo counting, G01T1/161; apparatus for taking X-ray photographs G03B42/02
• CPC Class Codes: A61B 6/03 Computed tomography [CT] echo-tomography A61B8/14
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/4423 related to hygiene or sterilisation
• CPC Class Codes: A61B 6/467 characterised by special input means
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A medical system has at least one projection surface onto which a projection unit projects of operating elements. The system is controlled by detecting movements of a hand, preferably a finger, of an operator on the projection surface. An optical detector acquires the position and/or the motion of the finger relative to the projection surface, and a control unit evaluates the output data of the detector in order to control the system.

Claims

1. A medical system comprising:

a plurality of controllable system components;

a projection surface;

a projector which projects an image of at least one operating element for at least one of said system components onto said projection surface;

said projection surface having at least one surface structure and said projector projecting said at least one image of an operating element onto said at least one surface structure;

a visual detector having a field of view encompassing said projection surface for detecting a characteristic of an appendage of an operator on said projection surface selected from the group of characteristics consisting of a position of said appendage relative to said surface structure and a movement of said appendage relative to said surface structure, said detector generating a detector output dependent on said characteristic of said appendage; and

control means, supplied with said detector output, for identifying a control instruction associated with said characteristic of said appendage and for controlling said at least one of said system components according to said control instruction.

2. A medical system as claimed in claim 1 wherein said control instruction comprises a first control instruction, and said medical system further comprising a displaceable control element simulation mounted on said projection surface and displaceable by said operator, and wherein said visual detector detects displacement of said control element simulation and wherein said detector output is additionally dependent on said displacement of said control element simulation, and wherein said control means comprises means for identifying a second control instruction associated with said displacement of said control element simulation, for controlling said at least one of said system components according to said first control instruction and said second control instruction.

3. A medical system as claimed in claim 2 wherein said visual detector additionally detects gestures of said operator and wherein said detector output is additionally dependent on said gestures, and wherein said control means comprises means for identifying a third control instruction associated with said gestures and for controlling said at least one of said system components according to said first, second and third control instructions.

4. A medical system as claimed in claim 3 wherein said gestures have respective amplitudes associated therewith, and wherein said detector output is additionally dependent on said amplitudes of said gestures.

5. A medical system as claimed in claim 3 wherein said visual detector has a field of view which additionally encompasses a head of said operator, and wherein said visual detector additional detects head movements of said operator and wherein said detector output is additionally dependent on said head movements, and wherein said control means comprises means for identifying a fourth control instruction associated with said head movements and for controlling said at least one of said system components according to said first, second, third and fourth control instructions.

6. A medical system as claimed in claim 5 wherein said visual detector detects a facial expression of said operator and wherein said detector output is additionally dependent on said facial expression, and wherein said control means comprises means for identifying a fifth control instruction association with said facial expression for controlling said at least one of said system components according to said first, second, third, fourth and fifth instructions.

7. A medical system as claimed in claim 6 wherein said control means comprises means for checking whether a combination of said first, second, third, fourth and fifth control instructions would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first, second, third, fourth and fifth control instructions would produce said impermissible operation.

8. A medical system as claimed in claim 2 wherein said control means comprises means for checking whether a combination of said first and second control instruction would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

9. A medical system as claimed in claim 1 wherein said control instruction comprises a first control instruction, and wherein said visual detector additionally detects gestures of said operator, and wherein said detector output is additionally dependent on said gestures, and wherein said control means comprises means for identifying a second control instruction associated with said gestures and for controlling said at least one of said system components according to said first and second control instructions.

10. A medical system as claimed in claim 9 wherein said gestures have respective amplitudes associated therewith, and wherein said detector output is additionally dependent on said amplitudes of said gestures.

11. A medical system as claimed in claim 9 wherein said control means comprises means for checking whether a combination of said first and second control instructions would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

12. A medical system as claimed in claim 1 wherein said control instruction comprises a first control instruction and wherein said visual detector has a field of view encompassing a head of said operator and wherein said visual detector detects a head movement of said operator and wherein said detector output is additionally dependent on said head movement, and wherein said control means comprises means for identifying a second control instruction associated with said head movement for controlling said at least one of said system components according to said first and second control instructions.

13. A medical system as claimed in claim 12 wherein said control means comprises means for checking whether a combination of said first and second control instructions would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

14. A medical system as claimed in claim 1 wherein control instruction comprises a first control instruction, and wherein said visual detector has a field of view which encompasses a face of said operator, and wherein said visual detector detects a facial expression of said operator and wherein said detector output is additionally dependent on said facial expression, and wherein said control means comprises means for identifying a second control instruction associated with said facial expression for controlling said at least one of said system components according to said first and second control instructions.

15. A medical system as claimed in claim 14 wherein said control means comprises means for checking whether a combination of said first and second control instructions would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

16. A medical system as claimed in claim 1 wherein said projection surface comprises a first projection surface, and said medical system further comprising a plurality of additional projection surfaces, said first projection surface and said plurality of additional projection surfaces comprising a group of projection surfaces.

17. A medical system as claimed in claim 16 comprising means for selectively projecting said image of at least one operating element on only a selected one of said projection surfaces in said group of projection surfaces or on selected multiple ones of said projection surfaces in said group of projection surfaces.

18. A medical system as claimed in claim 1 further comprising mounting means for said projection surface for selectively orienting said projection surface relative to said operator.

19. A medical system as claimed in claim 1 further comprising means for mounting said projection surface on one of said system components for allowing displacement of said projection surface relative to said one of said system components.

20. A medical system as claimed in claim 1 wherein at least one of said system components generates imaging information, and wherein said imaging information is supplied to said projector and wherein said projector projects said imaging information onto said projection surface.

21. A medical system as claimed in claim 1 wherein at least one of said system components generates system data and wherein said system data are supplied to said projector and wherein said projector projects alphanumeric characters representing said system data on said projection surface.

22. A medical system as claimed in claim 1 wherein said plurality of controllable system components comprise a magnetic resonance imaging apparatus.

23. A medical system as claimed in claim 1 wherein said plurality of controllable system components comprise an X-ray apparatus for producing X-ray shadowgraphs.

24. A medical system as claimed in claim 1 wherein said plurality of controllable system components comprise an ultrasound diagnostic apparatus.

25. A medical system as claimed in claim 1 wherein said plurality of controllable system components comprise an X-ray computed tomography apparatus.

26. A medical system as claimed in claim 25 wherein one of said controllable system components comprises a gantry, and wherein said projection surface is mounted on said gantry.

27. A medical system as claimed in claim 26 wherein said controllable system component comprising said gantry further comprises lateral supports for said gantry, and wherein said projection surface is disposed on one of said lateral supports.

54. A medical system as claimed in claim 1 wherein said plurality of controllable system components comprise a nuclear diagnostic apparatus.

28. A medical system comprising:

a plurality of controllable system components;

a displaceable operating element simulation which simulates an operating element for at least one of said system components and which is displaceable by an operator;

a visual detector having a field of view which encompasses said operating element simulation for detecting displacement of said operating element simulation, said visual detector generating a detector output dependent on said displacement; and

control means, supplied with said detector output, for identifying a control instruction associated with said displacement and for controlling said at least one system component according to said control instruction.

29. A medical system as claimed in claim 28 further comprising:

a projection surface on which said displaceable operating element simulation is disposed;

a projector which projects an image of at least one operating element for at least one of said system components onto said projection surface with said image of said at least one operating element coinciding with said displaceable operating element simulation and wherein said visual detector detects a characteristic of an appendage of said operator on said projection surface selected from the group of characteristics consisting of a position of said appendage relative to said operating element simulation and a motion of said appendage relative to said operating element simulation, and wherein said detector output is additionally dependent on said characteristic of said appendage, and wherein said control instruction comprises a first control instruction and said control means comprises means for identifying a second control instruction associated with said characteristic of said appendage for controlling said at least one of said system components according to said first and second control instructions.

30. A medical system as claimed in claim 29 wherein said projection surface comprises a first projection surface, and said medical system further comprising a plurality of additional projection surfaces, said first projection surface and said plurality of additional projection surfaces comprising a group of projection surfaces.

31. A medical system as claimed in claim 30 comprising means for selectively projecting said image of at least one operating element on only a selected one of said projection surfaces in said group of projection surfaces or on selected multiple ones of said projection surfaces in said group of projection surfaces.

32. A medical system as claimed in claim 29 further comprising mounting means for said projection surface for selectively orienting said projection surface relative to said operator.

33. A medical system as claimed in claim 29 further comprising means for mounting said projection surface on one of said system components for allowing displacement of said projection surface relative to said one of said system components.

34. A medical system as claimed in claim 29 wherein at least one of said system components generates imaging information, and wherein said imaging information is supplied to said projector and wherein said projector projects said imaging information onto said projection surface.

35. A medical system as claimed in claim 29 wherein at least one of said system components generates system data and wherein said system data are supplied to said projector and wherein said projector projects alphanumeric characters representing said system data on said projection surface.

36. A medical system as claimed in claim 29 wherein said plurality of controllable system components comprise an X-ray computed tomography apparatus and wherein one of said controllable system components comprises a gantry, and wherein said projection surface is mounted on said gantry.

37. A medical system as claimed in claim 36 wherein said controllable system component comprising said gantry further comprises lateral supports for said gantry, and wherein said projection surface is disposed on one of said lateral supports.

38. A medical system as claimed in claim 28 wherein said control instruction comprises a first control instruction, and wherein said visual detector additionally detects gestures of said operator, and wherein said detector output is additionally dependent on said gestures, and wherein said control means comprises means for identifying a second control instruction associated with said gestures and for controlling said at least one of said system components according to said first and second control instructions.

39. A medical system as claimed in claim 38 wherein said gestures have respective amplitudes associated therewith, and wherein said detector output is additionally dependent on said amplitudes of said gestures.

40. A medical system as claimed in claim 38 wherein said control means comprises means for checking whether a combination of said first and second control instructions would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

41. A medical system as claimed in claim 28 wherein said control instruction comprises a first control instruction and wherein said visual detector has a field of view encompassing a head of said operator and wherein said visual detector detects a head movement of said operator and wherein said detector output is additionally dependent on said head movement, and wherein said control means comprises means for identifying a second control instruction associated with said head movement for controlling said at least one of said system components according to said first and second control instructions.

42. A medical system as claimed in claim 41 wherein said control means comprises means for checking whether a combination of said first and second control instruction would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

43. A medical system as claimed in claim 28 wherein control instruction comprises a first control instruction, and wherein said visual detector has a field of view which encompasses a face of said operator, and wherein said visual detector detects a facial expression of said operator and wherein said detector output is additionally dependent on said facial expression, and wherein said control means comprises means for identifying a second control instruction associated with said facial expression for controlling said at least one of said system components according to said first and second control instructions.

44. A medical system as claimed in claim 43 wherein said control means comprises means for checking whether a combination of said first and second control instruction would produce an impermissible operation of said at least one of said system components, and for disenabling control of said at least one of said system components if said combination of said first and second control instructions would produce said impermissible operation.

45. A medical system as claimed in claim 28 wherein said plurality of controllable system components comprise a magnetic resonance imaging apparatus.

46. A medical system as claimed in claim 28 wherein said plurality of controllable system components comprise an X-ray apparatus for producing X-ray shadowgraphs.

47. A medical system as claimed in claim 28 wherein said plurality of controllable system components comprise an ultrasound diagnostic apparatus.

48. A medical system as claimed in claim 28 wherein said plurality of controllable system components comprise an X-ray computed tomography apparatus.

55. A medical system as claimed in claim 28 wherein said plurality of controllable system components comprise a nuclear diagnostic apparatus.

49. An X-ray computed tomography apparatus comprising:

a plurality of controllable system components including a tiltable gantry mounted on a pair of lateral supports;

a projection surface mounted on one of said lateral supports;

a projector which projects an image of at least one operating element for at least one of said system components onto said projection surface;

said projection surface having at least one surface structure and said projector projecting said at least one image of an operating element onto said at least one surface structure;

a visual detector having a field of view encompassing said projection surface for detecting a characteristic of an appendage of an operator on said projection surface selected from the group of characteristics consisting of a position of said appendage relative to said surface structure and a movement of said appendage relative to said surface structure, said detector generating a detector output dependent on said characteristic of said appendage; and

control means, supplied with said detector output, for identifying a control instruction associated with said characteristic of said appendage and for controlling said at least one of said system components according to said control instruction.

50. A computed tomography apparatus as claimed in claim 49 further comprising at least one additional projection surface mounted on said gantry, and wherein said projector comprises means for selectively projecting said image of at least one operating element on a selected one of said projection surface and said additional projection surface, or on both of said projection surface and said additional projection surface.

51. A computed tomography apparatus as claimed in claim 49 wherein at least one of said system components generates system information, and wherein said system information is supplied to said projector and wherein said projector projects said system information onto said projection surface in addition to said image of said at least one operating element.

52. A computed tomography apparatus as claimed in claim 49 wherein at least one of said system components produces an image of a patient, and wherein said image of said patient is supplied to said projector, and wherein said projector projects said image of said patient onto said projection surface in addition to said image of said at least one operating element.

53. A computed tomography apparatus as claimed in claim 49 wherein one of said system components produces a text field, and wherein said text field is supplied to said projector and wherein said projector projects said text field onto said projection surface in addition to said image of said at least one operating element.