System and method for constraining a graphical hand from penetrating simulated graphical objects

• US 7,472,047 B2
• Filed: 03/17/2004
• Issued: 12/30/2008
• Est. Priority Date: 05/12/1997
• Status: Expired due to Fees

• Current Assignee: Immersion Corporation
• Original Assignee: Immersion Corporation
• Inventors: Kramer, James F., Ullrich, Christopher
• Primary Examiner(s): Rodriguez; Paul L
• Assistant Examiner(s): Craig; Dwin M
• Application Number:
  US10/801,643
• Publication Number:
  US 20040236541A1
• Time in Patent Office: 1,749 Days
• Field of Search: 703/6, 715/701, 715/702, 715/757, 715/706
• US Class Current: 703/6
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/21 Collision detection, intersection
• CPC Class Codes: G06T 2210/28 Force feedback
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 2219/2021 Shape modification

Abstract

Movement of a graphical hand is constrained when the physical hand controlling the graphical hand does not have a similar physical constraint. An analysis of a revolute-joint-link-spring model in which an uncompressed/unextended spring position represents the corresponding measured joint angle or link position is used. Linear springs , non-linear springs, or the like may be employed to obtain the desired result of allowing a graphical joint or link to deviate from what the corresponding measured joint or link provides. If a graphical hand configuration causes a portion of the hand to penetrate a simulated graphical solid object, a mathematical determination is used to compute modified joint and link positions such that the graphical hand part will no longer penetrate the solid object. Such a constraint technique may include solving a spring model such that the various joint and link springs compress or extend to produce modified joint and link positions.

Claims

1. A system for moving a simulated-multi-articulated structure in relation to the movement of an analogous physical-multi-articulated structure, where said simulated-multi-articulated structure moves in a simulated environment comprising a simulated impediment to free motion, where said physical-multi-articulated structure moves in an environment lacking an analogous physical impediment, said system comprising:

a device for measuring the configuration of said physical-multi-articulated structure and the spatial placement of said physical-multi-articulated structure relative to an inertial reference frame and providing digitized signals associated with the configuration and spatial placement; and

a data processor, comprising data related to the spatial placement of said simulated impediment and constraints of said simulated impediment and said simulated-multi-articulated structure, for receiving said digitized signals and modifying said digitized signals using said data to generate a set of modified signals specifying the configuration and spatial placement of said simulated-multi-articulated structure, such that when said simulated-multi-articulated structure encounters said simulated impediment, the configuration and spatial placement of said simulated-multi-articulated structure is in part determined by the constraints causing said simulated-multi-articulated structure to flex, said flex being manifested in displacement of multiple points of the simulated-multi-articulated structure relative to a non-flex position thereof.

2. A system for moving a simulated hand in relation to the movement of a physical hand, where said simulated hand moves in a simulated environment comprising a simulated impediment to free motion, where said physical hand moves in an environment lacking an analogous physical impediment, said system comprising:

a device for measuring the configuration of said physical hand and the spatial placement of said physical hand relative to an inertial reference frame and providing digitized signals associated with the configuration and spatial placement; and

a data processor, comprising data related to the spatial placement of said simulated impediment and constraints of said simulated impediment and said simulated hand, for receiving said digitized signals and modifying said digitized signals using said data to generate a set of modified signals specifying the configuration and spatial placement of said simulated hand, such that when said simulated hand encounters said simulated impediment, the configuration and spatial placement of said simulated hand is in part determined by the constraints causing said simulated hand to flex, said flex being manifested in displacement of multiple points of the simulated hand relative to a non-flex position thereof.

3. A system according to claim 2, wherein said device comprises goniometers for measuring the angles of the joints of a physical hand, a tracking device for measuring the spatial placement of said physical hand relative to an inertial reference frame and means for mounting said device on said physical hand, said device providing digitized measured signals associated with the angles and the spatial placement;

and

said data processor producing modified signals from said digitized measured signals and said constraints using a simulated spring attached between first and second simulated hands, where the angles and placement, of said first simulated hand uses said digitized measured signals and the angles and placement of said second simulated hand uses said modified signals and said first and second simulated hands are superimposed in the absence of said second simulated hand encountering said simulated impediment;

such that when said second simulated hand encounters said impediment, said second simulated hand flexes and is displaced from said first simulated hand and realigns with said first simulated hand when said impediment is removed.

4. A system for moving a simulated graphical hand in relation to the movement of a physical hand, where said simulated graphical hand moves in a simulated environment comprising a simulated graphical impediment to free motion, where said physical hand moves in an environment lacking an analogous physical impediment, said system comprising:

a device comprising goniometers for measuring the angles of the joints of said physical hand, a tracking device for measuring the spatial placement of said physical hand relative to an inertial reference frame and means for mounting said device on said physical hand, said device providing digitized measured signals associated with the angles and the spatial placement; and

a data processor, comprising data related to the spatial placement of said simulated impediment and constraints of said simulated impediment and said simulated graphical hand, for producing modified signals from said digitized measured signals using said data and a simulated spring mass dashpot attached between first and second simulated hands, where the angles and placement of said first simulated hand uses said digitized measured signals and the angles and placement of said second simulated hand uses said modified signals and said first and second simulated hands are superimposed in the absence of said second simulated hand encountering said simulated graphical impediment, such that when said second simulated hand encounters said simulated graphical impediment, said second simulated hand is flexed and displaced from said first simulated hand and realigns with said first simulated hand when said simulated graphical impediment is removed at a rate regulated by said spring mass dashpot, said second simulated hand being depicted graphically as said simulated graphical hand.

5. A system for moving a simulated graphical hand in relation to the movement of a physical hand, where said simulated graphical hand moves in a simulated environment comprising a simulated graphical impediment to free motion, where said physical hand moves in an environment lacking analogous physical impediments, said system comprising:

a device comprising goniometers for measuring the angles of the joints of said physical hand, a tracking device for measuring the spatial placement of said physical hand relative to an inertial reference frame and means for mounting said device on said physical hand, said device providing digitized measured signals associated with the angles and the spatial placement; and

a data processor, comprising data related to the spatial placement of said simulated graphical impediment and constraints of said simulated graphical impediment and said simulated graphical hand, for producing modified signals from said digitized measured signals using said data and a simulated spring attached between first and second simulated hands, where the angles and placement of said first simulated hand uses said digitized measured signals and the angles and placement of said second simulated hand uses said modified signals, and said first and second simulated hands are superimposed in the absence of said second simulated hand encountering said simulated impediment;

such that when said second simulated hand encounters said simulated graphical impediment, said second simulated hand is flexed and displaced from said first simulated hand and when said simulated graphical impediment is removed, becomes superimposed with said first simulated hand at a rate in proportion to the movement of said first simulated hand, said second simulated hand being depicted graphically as said simulated graphical hand.

6. A method for moving a simulated-multi-articulated structure in a graphical environment, comprising an impediment to free motion, in relation to the movement of an analogous physical-multi-articulated structure in a physical environment lacking said impediment, said method comprising steps of:

generating digitized signals representing the configuration and spatial placement of said physical-multi-articulated structure;

transferring said digitized signals to a data processor;

recording, as data, in said data processor, the spatial placement of said impediment and constraints of said impediment and said simulated-multi-articulated structure;

generating modified signals from said-digitized signals and said data specifying the configuration and spatial placement of said simulated-multi-articulated structure; and

impeding the free motion of said simulated-multi-articulated structure when said simulated-multi-articulated structure encounters said impediment causing said simulated-multi-articulated structure to flex, said flex being manifested in displacement of multiple points of the simulated-multi-articulated structure relative to a non-flex position thereof.

7. A computer program product for use in conjunction with a computer system, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer, program mechanism, comprising:

a program module that moves a simulated-multi-articulated structure in, a graphical environment having an impediment to free motion in relation to the movement of an analogous physical-multi-articulated structure in a physical environment lacking said impediment, said program module including instructions that when executed by said computer system performs;

for;

generating first data representing the configuration and spatial placement of said physical-multi-articulated structure;

recording as second data the spatial placement of said impediment and constraints of said impediment and said simulated-multi-articulated structure;

generating third data from said first data and said second data specifying the configuration and spatial placement of said simulated-multi-articulated structure; and

impeding the free motion of said simulated-multi-articulated structure when said simulated-multi-articulated structure encounters said impediment causing said simulated-multi-articulated structure to flex, said flex being manifested in displacement of multiple points of the simulated-multi-articulated structure relative to a non-flex position thereof.

8. A system for moving a simulated structure in relation to the movement of an analogous physical structure, where said simulated structure moves in a simulated environment comprising a simulated impediment to free motion, where said physical structure moves in an environment lacking an analogous physical impediment, said system comprising:

a device measuring the configuration of said physical structure and the spatial placement of said physical structure relative to a reference frame and providing information associated with the configuration and spatial placement; and

a processor, comprising values related to the constraints of said simulated structure and said simulated impediment, receiving said information and modifying said information using said values to generate a set of modified signal'"'"'s specifying the configuration and spatial placement of said simulated structure, such that the free motion of the simulated structure is impeded when encountering said simulated impediment causing said simulated structure to flex, said flex being manifested in displacement of multiple points of the simulated-multi-articulated structure relative to a non-flex position thereof.

9. A device, comprising:

a multi-articulated structure; and

a sensor coupled to the multi-articulated structure, the sensor configured to determine a configuration of the multi-articulated structure and a spatial placement of the multi-articulated structure relative to an inertial reference frame,

said sensor further configured to transmit a signal associated with the configuration and the spatial placement of the multi-articulated structure, the signal being configured to generate a set of modified signals specifying data values associated with a configuration and a spatial placement of a simulated multi-articulated structure displayed in a graphical environment, the set of modified signals configured to deform the simulated multi-articulated structure when a simulated interaction occurs between the simulated multi-articulated structure and a simulated object such that multiple points of the simulated-multi-articulated structure are displaced relative to a non-deformation position of the simulated-multi-articulated structure.

10. The device of claim 9, wherein said sensor includes a goniometer configured to determine at least one angle of a joint of the multi-articulated structure.

11. The device of claim 9, wherein said sensor includes a tracking device configured to determine the spatial placement of the multi-articulated structure relative to the inertial reference frame.

12. The device of claim 9, the set of modified signals being a first set of modified signals, the simulated multi-articulated structure being a first simulated multi-articulated structure, wherein the signal associated with the configuration and the spatial placement of the multi-articulated structure is further configured to generate a second set of modified signals specifying data values associated with a configuration and a spatial placement of a second simulated multi-articulated structure.

13. The device of claim 12, the first simulated multi-articulated structure being superimposed on the second simulated multi-articulated structure, wherein when the simulated interaction occurs between the simulated multi-articulated structure and the simulated object, the first simulated multi-articulated structure is separated from the second simulated multi-articulated structure.