Method for tracking and displaying user's spatial position and orientation, a method for representing virtual reality for a user, and systems of embodiment of such methods

• US 6,005,548 A
• Filed: 08/14/1997
• Issued: 12/21/1999
• Est. Priority Date: 08/14/1996
• Status: Expired due to Term

• Current Assignee: Virtoons AG
• Original Assignee: Nurakhmed Nurislamovich Latypov, Nurulla Nurislamovich Latypov
• Inventors: Latypov, Nurulla Nurislamovich, Latypov, Nurakhmed Nurislamovich
• Primary Examiner(s): Shalwala, Bipin H.
• Assistant Examiner(s): Kovalick, Vincent E.
• Application Number:
  US08/910,998
• Time in Patent Office: 859 Days
• Field of Search: 345/156, 345/355, 345/419, 345/420, 345/473, 345/976, 345/331, 345/332, 463/36
• US Class Current: 345/156
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/8005 Athletics
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G01S 5/0247 Determining attitude
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G09B 23/28 for medicine

Abstract

The invention relates to methods and systems for determination of position and orientation of users in a space and display of virtual space, and can be used in interactive computer games, health-strengthening teaching devices, or trainings of the athletic and military purposes. The invention provides an improved accuracy and speed of tracking of the user'"'"'s spatial position and orientation, provides convenience for and naturalness of his/her movements and representation of virtual space in conformity with user'"'"'s movements in the real space. Sensors 4 for tracking angular position of segments 2 of the user'"'"'s locomotor system are attached at places of articulations 3 of segments 2 and at certain segments 2 proper. At least at one of segments 2 arranged are main means 5 for determination of reference directions, which means provide determination of orientation of a given segment relative to the reference directions. Data from sensors 4, means 5 for determination of reference directions and, when necessary, from devices 8 of interaction of a user with the virtual space objects are processed for determination of the user'"'"'s orientation and position as a whole on the basis of particular values of angles between segments, and orientation of a main segment whereat means for determination of reference directions, relative to the reference directions, are arranged. Obtained data are used for displaying user'"'"'s movements in real time, or representation for a user of virtual space that corresponds to his/her viewing field that varies according to movements in real space.

Claims

1. A system for representation of virtual space for a user who moves in the real space, comprising

a plurality of relative rotation angles sensors arranged adjacently to main articulations between the user'"'"'s locomotor system segments, or at segments proper, for outputting, at outputs of said sensors, signals of relative rotation angles of said segments;

means for determination of reference directions, which means are arranged at least at one of the user'"'"'s locomotor system segments, for outputting a reference direction signal at output of said reference directions determination means;

signals analogue-to-digital conversion means, having a first input for receiving signals of said plurality of relative rotation angles sensors, a second input for receiving signals of said at least one means for determination of reference directions, and an output for outputting digital data;

means for processing digital data for construction of a mathematical model of the three-dimensional space and a user in this space, having a first input for receiving said digital data, and a second input for inputting additional data on a user and a space to be displayed, and an output for outputting data on a constructed mathematical model of a user in a space to be simulated;

means for determination of the user'"'"'s viewing field, having an input connected to output of said digital data processing means, and an output;

means for forming signals of image of virtual space corresponding to the user'"'"'s viewing field, having a first input connected to output of said digital data processing means, a second input connected to output of means for determination of the user'"'"'s viewing field, and an output;

means for displaying virtual space for a user, having an input connected to output of means for forming signals of a virtual space image that corresponds to the user'"'"'s viewing field.

2. A system as claimed in claim 1, wherein a strain gauge is used as a relative rotation angle sensor.

3. A system as claimed in claim 1, wherein a fibre-optics sensor is used as a relative rotation angle sensor.

4. A system as claimed in claim 1, wherein means for determination of reference directions comprise a magnetotelluric field sensor and a local vertical sensor.

5. A system as claimed in claim 4, wherein a gravitation sensor is used as a local vertical sensor.

6. A system as claimed in claim 1, wherein a gyrostabilised sensor of reference directions is used as means for determination of reference directions.

7. A system as claimed in claim 1, wherein means for determination of reference directions comprise a sensor of reference directions external source signal.

8. A system as claimed in claim 1, further comprising a manipulator for interaction of a user with objects of displayed real or virtual space, an additional reference directions determination means arranged on a manipulator, having an output connected to input of analogue-to-digital conversion means.

9. A system as claimed in claim 1, further comprising long-term memory means, having an input for receiving digital data from said output of the analogue-to-digital conversion means.

10. A method for representation of virtual space for a user taking into account his/her motion in the real space, including the following steps:

a) attaching, at each of the main articulations of the user'"'"'s locomotor system segments, means for determination of at least one angle between segments that are adjacent to a respective articulation;

b) determination of linear dimensions of the user'"'"'s locomotor system segments;

c) arranging, at least at one of the user'"'"'s locomotor systems segments, means for determining reference directions;

d) determination of orientation of at least one segment, whereat the reference directions determination means are arranged, relative to said reference directions;

e) determination of angles between segments that are adjacent to said main articulations;

f) determination of orientation and position of a user in space as a whole on the basis of obtained values of angles and orientation of said at least one segment, whereat the reference directions determination means are arranged;

g) determination of a reference segment or reference segments of the user'"'"'s locomotor system segments;

h) positioning of said reference segment or segments relative to a reference in the virtual space to be displayed;

i) determination of position and orientation of user'"'"'s head in virtual space for determining the user'"'"'s viewing field in the virtual space;

j) formation and representation for a user of an area of the virtual space corresponding to his/her viewing field.

11. A method as claimed in claim 10, including iteration of steps from (g) to (j) for displaying changes of the virtual space in real time according to user'"'"'s movements and interaction with virtual space objects.

12. A method as claimed in claim 10, further comprising a step of registration of a series of data on the user'"'"'s position and orientation for subsequent use.

13. A method as claimed in claim 10, wherein "North-South" directions of a magnetotelluric field sensor and a local vertical sensor direction are used as reference directions.

14. A method as claimed in claim 10, wherein "North-South" and "West-East" directions of a magnetotelluric field sensor are used as reference directions.

15. A method as claimed in claim 10, wherein directions established by gyrostabilised sensors of reference directions are used as reference directions.

16. A method as claimed in claim 10, wherein directions established by an artificial external source of reference directions are used as reference directions.

17. A method as claimed in claim 10, wherein at least one pair of spatially separated sensors of absolute co-ordinates, which pair is arranged at least at one of the user'"'"'s locomotor system segments, is used as means for determining reference directions.

18. A method as claimed in claim 10, including determination of linear dimensions of the user'"'"'s locomotor system segments on the basis of known human body proportions, using the user'"'"'s height data.

19. A method as claimed in claim 10, further including scaling of values of linear dimensions of the user'"'"'s locomotor system segments for corresponding change of his/her dimensions relative to a space to be displayed.

20. A method as claimed in claim 10, further including steps of

use of a manipulator for interaction of a user with objects of virtual space to be displayed,

determination of position of a manipulator by a position of user'"'"'s hand holding the same, and

determination of exact orientation of a manipulator using, arranged thereon, additional means for determination of said reference directions.

21. A method as claimed in claim 10, wherein the following joints are selected as main articulations between the user'"'"'s locomotor system segments:

hip joint, knee joints, mortis joints, glenohumeral joints, elbow joints, hand joints, articulation of head and shoulders and articulation of thoracic and waist portions of spine.

22. A method as claimed in claim 21, wherein joints of foot and fingers are additionally selected the main articulations.

46. A method as claimed in claim 13, wherein "North-South" directions of magnetotelluric field sensor and directions of a local vertical are used as reference directions.

23. A system of tracking and displaying the user'"'"'s spatial position and orientation, comprising

a plurality of relative rotation angles sensors arranged adjacently to main articulations between the user'"'"'s locomotor system segments, or at said segments proper for outputting signals of relative rotation angles of said segments at said sensors outputs;

means for determination of reference directions, arranged at least at one of the user'"'"'s locomotor system segments, for outputting a reference direction signal at output of said means for determination of reference directions;

means for analogue-to-digital conversion of signals, having a first input for receiving signals of said plurality of relative rotation angles sensors, a second input for receiving signals of said at least one means for determining reference directions, and an output for outputting digital data;

means for processing digital data for constructing a mathematical model of the three-dimensional space and a user in this space, having an input for receiving said digital data, and an output for outputting data on the user'"'"'s position and orientation in the three-dimensional space to be simulated, and

means for displaying the user'"'"'s spatial position and orientation, having an input for receiving data on the user'"'"'s position and orientation in the three-dimensional space to be simulated.

24. A system as claimed in claim 23, further comprising at least one means of interaction of a user with virtual space objects, whereon arranged are additional means for determination of reference directions connected to a respective input of the analogue-to-digital conversion means.

25. A system as claimed in claim 23, wherein a relative rotation angles sensor is implemented as a strain gauge.

26. A system as claimed in claim 23, wherein a relative rotation angles sensor is implemented in the form of a fibre-optics sensor.

27. A system as claimed in claim 23, wherein the reference directions determination means comprises a magnetotelluric field sensor.

28. A system as claimed in claim 27, wherein the reference directions determination means further comprise a local vertical sensor.

29. A system as claimed in claim 28, wherein a gravitation sensor is used as a local vertical sensor.

30. A system as claimed in claim 23, wherein the reference directions determination means comprise a sensor of signal of an artificial external source of reference directions.

31. A system as claimed in claim 23, further comprising means for storing converted measured data for further reproduction of the user'"'"'s spatial position and orientation, connected to a digital data processing means via a transfer bus.

32. A system as claimed in claim 23, further comprising means for inputting additional data on a user and the three-dimensional space to be displayed, connected to a respective input of the digital data processing means.

33. A system as claimed in claim 23, further comprising means for forming an image at a specified viewing angle and amending an image, having an input connected to output of the digital data processing means, and an output connected to input of means for displaying the user'"'"'s spatial position and orientation.

34. A method of tracking and displaying a position and orientation of a user in space, comprising the steps of:

a) attaching means for determining at least one angle between segments adjacent to a respective articulation at each one of main articulations of segments of the user'"'"'s locomotor system;

b) disposing, at least at one of the user'"'"'s locomotor system segments, means for determining reference directions;

c) determining orientation of at least one segment, at which segment the reference direction determination means are disposed, relative to said reference directions;

d) determining angles between segments which are adjacent to said main articulations;

e) determining the user'"'"'s position and orientation as a whole in space on the basis of obtained values of angles, and orientation of said at least one segment, at which segment the reference directions determination means are disposed.

35. A method as claimed in claim 34, comprising iteration of steps from (c) to (e) at a specified periodicity to obtain interrelated series of data that determine temporal changes in the user'"'"'s spatial position and orientation according to real movements of a user.

36. A method as claimed in claim 35, comprising registration of obtained data on the user'"'"'s spatial position and orientation for further use.

37. A method as claimed in claim 34, wherein at step (e) values of linear dimensions of the user'"'"'s locomotor system segments are used, said method further comprising the following steps:

f) determination of a reference segment or segments of the user'"'"'s locomotor system;

g) positioning of said reference segment or segments relative to a reference in a space to be displayed.

38. A method as claimed in claim 37, comprising iteration of steps (c) to (g) at a specified periodicity to obtain and register series of data on the user'"'"'s position and orientation in a space to be displayed, and display the user'"'"'s movements in accordance with the obtained data.

39. A method as claimed in claim 38, comprising display of the user'"'"'s movements in real time.

40. A method as claimed in claim 39, comprising display of the user'"'"'s movements in accordance with a required viewing aspect.

41. A method as claimed in claim 39, further comprising steps of amending an obtained image of the user'"'"'s movements, and displaying the amended image of movements.

42. A method as claimed in claim 38, comprising display of the user'"'"'s movements in accordance with the registered data as necessary at a subsequent instant of time.

43. A method as claimed in claim 37, comprising determination of linear dimensions of the user'"'"'s locomotor system segments on the basis of known human body proportions using data on a user'"'"'s height.

44. A method as claimed in claim 37, further comprising scaling of linear dimensions values of the user'"'"'s locomotor system segments for respective change of his/her dimensions relative to a space to be displayed.

45. A method as claimed in claim 37, further comprising steps of

using a manipulator for interaction of a user with objects of virtual space to be displayed,

determining a position of a manipulator according to a position of a user'"'"'s hand holding the same, and

determining an exact orientation of a manipulator using additional, disposed thereon, means for determination of said reference directions.

47. A method as claimed in claim 34, wherein "North-South" and "West-East" directions of a magnetotelluric field sensor are used as reference directions.

48. A method as claimed in claim 34, wherein directions established by gyrostabilised sensors of reference directions are used as reference directions.

49. A method as claimed in claim 34, wherein directions established by an external source of reference directions are used as reference directions.

50. A method as claimed in claim 34, wherein at least one pair of spatially separated sensors of absolute co-ordinates disposed at least at one of the user'"'"'s locomotor system segments is used as means for determining reference directions.

51. A method as claimed in claim 34, wherein thigh joint, knee joint, mortis joints, glenohumeral joints, elbow joints, hand joints, head and shoulders articulation and articulation of the thoracic and waist portions of spine are selected as main articulations between segments of the user'"'"'s locomotor system.

52. A method as claimed in claim 51, wherein joints of foot and fingers are further selected as main articulations.