Method for the interaction of an operator with a model of a technical system

• US 10,642,377 B2
• Filed: 06/12/2017
• Issued: 05/05/2020
• Est. Priority Date: 07/05/2016
• Status: Active Grant

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: MacWilliams, Asa, Sauer, Markus
• Primary Examiner(s): Davis, Tony O
• Application Number:
  US16/315,231
• Publication Number:
  US 20190163287A1
• Time in Patent Office: 1,058 Days
• Field of Search: 345156, 345158, 345690
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts

Abstract

The method provides detection of a plurality of spatial parameters which are assigned to one or both forearms of an operator. Processing markers generated from the spatial parameters are visualized in a virtual scene. The operator activates processing markers and carries out processing operations by changing the position of his or her forearm(s). After the processing operations are complete, the operator deactivates the assignment between processing marker and manipulation marker by rotating a forearm. A particular advantage of the method disclosed herein compared with conventional methods lies in the activation and deactivation using the rotation of a forearm of the operator. A rotation of the wrist may be carried about largely independently of the direction in which the forearm is pointing. In this manner, the operator may use shoulder rotation and elbow flexion to change the position of the forearm in order to change spatial parameters for a processing marker, while processing operations may be deactivated and/or activated by an independent rotation of the forearm.

Claims

1. A method for an interaction of an operator with a model of a technical system, the method comprising:

positionally correct visualization of objects of the model in a virtual scene;

positionally correct visualization of at least one manipulation marker;

detecting a plurality of spatial parameters assigned to at least one forearm of the operator, and positionally correct visualization of at least one processing marker generated from the spatial parameters, in the virtual scene;

activating at least one processing marker as a first processing marker assigned to a first manipulation marker;

carrying out processing operations on the first manipulation marker as a function of a change in spatial parameters configured to be attributed to a change in position of a forearm assigned to the first processing marker; and

enabling the assignment of the first processing marker to the first manipulation marker as a function of a change in spatial parameters configured to be attributed to a rotational movement of the forearm assigned to the first processing marker.

2. The method of claim 1, wherein the processing operations carried out in the model of the technical system are converted into control operations at the technical system.

3. The method of claim 2, wherein the manipulation marker is assigned to at least one object or at least one region of the virtual scene.

4. The method of claim 2, wherein the activation of the first processing marker comprises:

detecting a spatial distance between the at least one processing marker and the at least one manipulation marker; and

activating a processing marker whose spatial distance from the first manipulation marker undershoots a predefinable first threshold value as the first processing marker assigned to the first manipulation marker.

5. The method of claim 4, wherein, when a second threshold value of the spatial distance between the processing marker and the first manipulation marker is undershot, feedback is output to the operator.

6. The method of claim 2, wherein the activation of the first processing marker comprises:

detecting a spatial distance between the at least one processing marker and the at least one manipulation marker;

marking a processing marker whose spatial distance from a first manipulation marker undershoots a predefinable first threshold value as the first processing marker configured to be assigned to the first manipulation marker; and

activating the first processing marker as the first processing marker assigned to the first manipulation marker as the function of the change in the spatial parameters configured to be attributed to the rotational movement of the forearm assigned to the first processing marker.

7. The method of claim 6, wherein, when a second threshold value of the spatial distance between the processing marker and the first manipulation marker is undershot, feedback is output to the operator.

8. The method of claim 2, wherein the rotational movement is a pronation or supination of at least one forearm of the operator.

9. The method of claim 2, wherein the plurality of spatial parameters comprises at least:

one parameter for a shoulder rotation of the operator;

one parameter for an elbow flexion of the operator; and

one parameter for a forearm rotation of the operator.

10. The method of claim 1, wherein the manipulation marker is assigned to at least one object or at least one region of the virtual scene.

11. The method of claim 1, wherein the activation of the first processing marker comprises:

detecting a spatial distance between the at least one processing marker and the at least one manipulation marker; and

activating a processing marker whose spatial distance from the first manipulation marker undershoots a predefinable first threshold value as the first processing marker assigned to the first manipulation marker.

12. The method of claim 11, wherein, when a second threshold value of the spatial distance between the processing marker and the first manipulation marker is undershot, feedback is output to the operator.

13. The method of claim 1, wherein the activation of the first processing marker comprises:

detecting a spatial distance between the at least one processing marker and the at least one manipulation marker;

marking a processing marker whose spatial distance from a first manipulation marker undershoots a predefinable first threshold value as the first processing marker configured to be assigned to the first manipulation marker; and

activating the first processing marker as the first processing marker assigned to the first manipulation marker as the function of the change in the spatial parameters configured to be attributed to the rotational movement of the forearm assigned to the first processing marker.

14. The method of claim 13, wherein, when a second threshold value of the spatial distance between the processing marker and the first manipulation marker is undershot, feedback is output to the operator.

15. The method of claim 1, wherein the rotational movement is a pronation or supination of at least one forearm of the operator.

16. The method of claim 1, wherein the plurality of spatial parameters comprises at least:

one parameter for a shoulder rotation of the operator;

one parameter for an elbow flexion of the operator; and

one parameter for a forearm rotation of the operator.

17. The method of claim 1, wherein the method is used to define an examination region to be delimited on a body of a patient.