Virtual environment manipulation device modelling and control

• US 6,072,466 A
• Filed: 08/01/1997
• Issued: 06/06/2000
• Est. Priority Date: 08/02/1996
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Shah, Jain K., Gallery, Richard D., Heron, Dale R.
• Primary Examiner(s): WU, XIAO MIN
• Application Number:
  US08/904,390
• Time in Patent Office: 1,040 Days
• Field of Search: 345/156, 345/157, 345/158, 345/161, 345/163, 345/7, 345/8, 345/9, 345/169, 345/159, 345/162
• US Class Current: 345/156
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/218 using pressure sensors, e.g. generating a signal proportional to the pressure applied by the player
• CPC Class Codes: A63F 2300/6054 by generating automatically game commands to assist the player, e.g. automatic braking in a driving game
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

A method and apparatus are provided for providing user-directed operation of a virtual manipulator, such as a simulated hand or claw (30) within a computer-generated virtual environment. The environment comprises one or more objects (32) which may be held, released or impelled by the manipulator under user control. Movement of the virtual manipulator is directed by a suitable hand held device operated by a user, with a simple control directing opening or closing of the claw (30). In operation, actions such as throwing, catching or punching an object (32) are partially automated, with the decision as to the action initiated being determined by the separation (D) between object and manipulator, and the current and previous states of each as recorded in the form of respective state variables. A method for the derivation of an applied velocity for the object acted upon is also described.

Claims

1. A method for providing user-directed operation of a virtual manipulator within a computer-generated virtual environment, said environment comprising one or more objects which may be held, released or impelled by the manipulator under user control, the method comprising:

providing user operable positional input means by use of which the user is enabled to move the virtual manipulator about the virtual environment;

providing user operable actuator means in response to operation of which by the user the virtual manipulator is changed from a holding to a releasing orientation, or from a releasing to a holding orientation;

providing automation of manipulation actions on an object by the manipulator when certain predetermined and periodically monitored conditions are found to be true, said conditions comprising at least;

Whether or not the manipulator was previously in a holding orientation;

the separation of the manipulator and the said object;

whether or not the manipulator is currently in a holding orientation; and

whether or not the manipulator previously held the object;

the providing automation includes storing respective data structures for the manipulator and at least one object, for the manipulator the data structure including a state variable having only two states indicating whether the manipulator is displayed in one pre-defined releasing (open) position or one pre-defined holding (closed) position, and for the object a state variable having only two states indicating whether the object is held or not held, said providing automation further including eliminating at least one calculation step of a relative characteristic between the object and the manipulator when the state variable for the object indicates the object is held and the state variable for the manipulator indicates the manipulator is in a holding orientation.

2. A method as claimed in claim 1, wherein the automated manipulation action comprises the manipulator taking hold of an object when the manipulator is changed from a releasing to a holding orientation and the separation of the manipulator and the said object is less than or equal to a first predetermined threshold value, the threshold value corresponding to a space between the manipulator and the object.

3. A method as claimed in claim 1, wherein the automated manipulation action comprises the manipulator releasing a currently held object when the manipulator is changed from a holding to a releasing orientation.

4. A method as claimed in claim 3, wherein the currently held object is given a velocity on release, said velocity being derived from the user directed motion of the manipulator prior to release.

5. A method as claimed in claim 4, wherein a record is maintained of the n most recent locations of the manipulator within the virtual environment and the times at which the manipulator was at these locations, and the derived velocity applied to the object is determined by averaging a calculated velocity for the manipulator at two or more of these locations.

6. A method as claimed in claim 1, wherein the automated manipulation action comprises impelling the object with a velocity derived from the preceding user directed motion of the manipulator, when the separation of the manipulator and the said object is less than or equal to a second predetermined threshold value and the manipulator is in the holding orientation prior to, and during, the determination of separation.

7. A method as claimed in claim 6, wherein a record is maintained of the n most recent locations of the manipulator within the virtual environment and the times at which the manipulator was at these locations, and the derived velocity applied to the object is determined by averaging a calculated velocity for the manipulator at two or more of these locations.

8. Virtual manipulator modelling and control apparatus operable to generate and animate under user direction a representation of a manipulator in a virtual environment and the interaction thereof with representations of at least one object also generated within the virtual environment, the apparatus comprising:

a first data store, holding data defining the virtual environment and each object therein;

a second data store, holding data defining features of the virtual manipulator representation;

a user-operable control device; and

processing means arranged to generate a representation of the virtual environment based on data from the first data store and from a first viewpoint, to generate the manipulator representation within the virtual environment based on data from the second data store, and to periodically modify the generated manipulator representation in response to signals received from the user-operable control means;

characterised in that the apparatus further comprises;

storage means holding respective data structures for the manipulator and for each object, said data structures being periodically updated by the processing means and containing for the manipulator a state variable having only two states indicating whether the manipulator is displayed in one pre-defined releasing (open) orientation or one pre-defined holding (closed) orientation and for each object a state variable having only two states indicating whether or not the object is currently held or not held by the manipulator; and

first position determination means arranged to determine a current separation within the virtual environment between the manipulator representation and an object representation, the processing means autonomously implementing one or more manipulator actions on an object when the said current separation is less than or equal to a predetermined value, and the current and at least one preceding one of the state variables have predetermined settings, said autonomously implementing including eliminating at least one calculation step of a relative characteristic between the object and the manipulator when the state variable for the object indicates the object is held and the state variable for the manipulator indicates the manipulator is in a holding orientation.

9. Apparatus as claimed in claim 8, wherein the user-operable control device is arranged to be held in a users hand, the apparatus including second position determining means operable to detect a location of said control device relative to a predetermined origin position, and said processing means is arranged to periodically modify said generated representation of the manipulator within the virtual environment to reproduce movements of the hand-held control device in said manipulator.

10. Apparatus as claimed in claim 9, wherein the user operable control device comprises a pressure sensor arrange to determine a force exerted by the user, wherein when the exerted force exceeds a first threshold value, the processing means is arranged to modify the generated manipulator representation from an open orientation to a closed orientation, and modify the corresponding stored state variable.

11. Apparatus as claimed in claim 10, wherein the generated manipulator representation is modified from a closed orientation to an open orientation when the exerted force drops below the first threshold when no object is held by the manipulator and below a second threshold value lower than the first when an object is held.

12. Apparatus as claimed in claim 8, wherein the user operable control device comprises a pressure sensor arrange to determine a force exerted by the user, wherein when the exerted force exceeds a first threshold value, the processing means is arranged to modify the generated manipulator representation from an open orientation to a closed orientation, and modify the corresponding stored state variable, and wherein the generated manipulator representation is modified from a closed orientation to an open orientation when the exerted force drops below the first threshold when no object is held by the manipulator and below a second threshold value lower than the first when an object is held.