Natural robot control

• US 6,741,911 B2
• Filed: 09/20/2001
• Issued: 05/25/2004
• Est. Priority Date: 09/20/2000
• Status: Active Grant

• Current Assignee: John Castle Simmons
• Original Assignee: John Castle Simmons
• Inventors: Simmons, John Castle
• Primary Examiner(s): Cuchlinski, Jr., William A.
• Assistant Examiner(s): Marc, McDieunel, Marc, McDieunel
• Application Number:
  US09/960,294
• Publication Number:
  US 20030030397A1
• Time in Patent Office: 978 Days
• Field of Search: 700/245, 700/253, 701/23, 318/568.11, 318/568.12, 318/568.17, 901/23, 901/34
• US Class Current: 700/245
• CPC Class Codes: B25J 9/0006 Exoskeletons, i.e. resembling a human figure
• CPC Class Codes: B25J 9/1689 Teleoperation
• CPC Class Codes: G05B 2219/40136 Stereo audio and vision
• CPC Class Codes: G05B 2219/40144 Force sensation feedback from slave
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

An apparatus and process for controlling a remote robot responsive to the integrated sensory perceptions, natural body movements, inertia and gravitation of an operator/user who is himself responding to a three dimensional virtual world of visual and sensory conditions responsive to the environment at the remote site where the robot is operating. The user requires no training, special commands or devices to direct the robot because the user has the perception of actually being at the remote location, handling remote objects he sees and feels in his hands, walking, climbing, viewing and feeling the remote world.

Claims

1. An apparatus for automatically resizing robotics equipment to match the stature and density attributes of a desired personality comprising:

telescoping “

bones”

are either threaded to lengthen or shorten as one of the two elements rotates or expanded by other means; and

motor or actuator means expand or contract the “

bones”

; and

weights mounted on the outside of the bones, typically starting at the mid-point can be moved by rotation on a thread external to the outside tube of the bone or, alternatively, dense weights inside the inner tube are moved along the length of the bone to move the center of gravity of the bone and thus, as a group, they determine the center of gravity of the remote robotics; and

motor or actuator means to move the weights; and

processing means to capture from remote sensing devices or direct entry the attributes to be emulated and issue the commands to the above motors/actuators which are operatively connected.

2. A device for supporting a person above the ground in a fashion allowing the person to have wide bodily mobility without encountering the interference of the supportive equipment comprising:

a primary support bar connected at one end to the user'"'"'s external body support assembly, typically from the lower back, allowing free movement without the user bumping into the support bar and, at the other end, to the motion and attitude control means; and

motion and attitude control means bearing, via the connected primary support bar, the weight of the user comprising;

a roll control mechanism controlling the roll position of the user by determining the rotation of the primary support bar with a rotating stepper or other power means; and

a primary pitch control means operatively connected to the primary support bar rotating the primary support bar, and thus the user, to a desired pitch position; and

computing means to direct the motors, actuators or other powered means to move the pitch and roll components to the proper locations.

3. The device for supporting a person of claim 2 further comprising:

an additional powered swivel on the primary support bar close to the point of attachment to the user'"'"'s body support equipment directed by the computing means to provide, in addition to or in lieu of the aforementioned primary pitch control mechanism, to provide a pitch rotation with a much shorter radius of rotation and thus much less undesirable vertical or horizontal travel when simple rotation is all that is desired.

4. The device for supporting a person of claim 2 further comprising:

a vertical control apparatus, normally operatively connected to the primary pitch bar and raising/lowering that assembly, along with the user as directed by the computing means.

5. The device for supporting a person of claim 2 further comprising:

a yaw control device upon which the aforementioned assemblies rest, normally connected to the vertical control apparatus, which rotates to change the relative yaw of the user.

6. The device for supporting a person of claim 2 further comprising:

an X,Y control device upon which the aforementioned assemblies rest which moves the entire assembly along the floor or on a supported track or other support device above the floor to provide inertial feedback to the user of travel along the plane of the floor.

7. An external apparatus for controlling a human, animal or mechanical body, comprising:

a plurality of body control elements attached to or in effective contact with a body part for stabilizing or mobilizing the body part;

a plurality of connective media operatively connected to the body control elements for pulling, pushing, twisting, rotating, or any combination thereof, the body control elements;

an actuation means operatively connected to the connective media, the body control elements, or any combination thereof, for applying pressure to, moving, or any combination thereof, the body control elements; and

a controller means operatively connected to the actuation means for directing the actuation means to acquire for the joint a currently desired position, apply a currently desired pressure, or a combination thereof;

whereby

the body can be stabilized and mobilized without the necessary requirement of external joints or support bars.

8. The external apparatus of claim 7, further comprising:

a plurality of active connective media which replace the functions of some or all of the connective media, said active connective media being capable of additionally providing the actuation means by self-contraction, expansion, or any combination thereof, in response to a stimulus or empowerment.

9. The external apparatus of claim 7, further comprising:

a computing means associated with controller means for directing the application of pressure to, or effecting the movement of, or any combination thereof, a joint on a desired axis or set of axes.

10. The external apparatus of claim 9, wherein:

the computing means effects the effective synchronization of multiple joints in a coordinated action or set of actions;

whereby

the multiple joints are controlled in a synchronized fashion causing multi-jointed limbs or the entire body to maintain a synchronized overall position.

11. The external apparatus of claim 9, wherein:

the computing means directs said actuation means to direct the joint through a predefined or calculated path over the course of joint rotation;

whereby

the joint can be directed through a preferred path or, potentially, even around a moving or non-rigid axis which can prevent opposition to a joint'"'"'s natural rotation.

12. The external apparatus of claim 9, wherein:

the computing means is operatively connected to the actuation means, a joint position sensing means, a pressure sensing means, or any combination thereof.

13. The external apparatus of claim 12, wherein:

the controller means can monitor and be responsive to actual positions or pressures or a combination thereof, provided by at least one of the joint position sensing means and the pressure sensing means.

14. The external apparatus of claim 7, wherein:

a plurality of actuation means coupled to connective media are placed near the joint to actuate the joint in directions reflective of each desired axis of rotation.

15. The external apparatus of claim 7, further comprising:

one or more external joints, joint guides, hinges, or any combination thereof, located to stabilize joint rotation, provide a structural connection between separate body support equipment elements, or any combination thereof.

16. The external apparatus of claim 7, wherein:

the controller means includes a reporting means for reporting the current positions of the actuation means.

17. The external apparatus of claim 16, wherein:

the controller means operatively connects to and monitors the reporting means for adjusting the position of, the tension upon, or any combination thereof, the body control elements.

18. The external apparatus of claim 7, wherein:

the body control elements can be used for bodies of different sizes.

19. The external apparatus of claim 7, wherein:

the external apparatus can be automatically configured or the external apparatus can be configured through a computer user interface to effectively match a particular body.

20. The external apparatus of claim 7, wherein:

any combination of the body control elements can be automatically adjusted for comfort or precision.

21. A device for directing the actions of a plurality of joints or limbs, or combination thereof, of organic, mechanical or virtual bodies, comprising:

a first joint control means for the control of joint dynamics on a first body;

a second joint control means for the control of joint dynamics of a second body;

a reporting means for reporting the joint dynamics of the first body or the second body, or a combination thereof; and

a computing means operatively connected to said first joint control means and said second joint control means;

whereby

the second body reflects some or all of the actions of the first body, the first body may be enforced to be essentially in agreement with some or all of the actual joint dynamics of the second body, and the first body may experience some or all of the second body'"'"'s actions and environment.

22. The device of claim 21, wherein:

said first joint control means, or said second joint control means, or a combination thereof, include a physical joint manipulation mechanism.

23. The device of claim 21, wherein:

At least one of said joint control means control the joints of a virtual body in a virtual environment.

24. The device of claim 21, wherein:

said joint dynamics comprise the position, pressure, speed of movement, true velocity, direction of movement, or attitude, or any combination thereof, of or upon a joint.

25. The device of claim 21, wherein:

the second body attempts to emulate the joint dynamics of the first body and computing means directs the first body to reflect some or all of the resulting joint dynamics of the first body.

26. The device of claim 21, wherein:

said computing means directs the second body to emulate at least one of said joint dynamics of the first body.

27. The device of claim 21, wherein:

said computing means directs the first body to emulate at least one of said joint dynamics of the second body.

28. The device of claim 21, wherein:

said reporting means includes physical sensory devices for reporting said joint dynamics of organic or mechanical joints.

29. The device of claim 21, wherein:

said reporting means includes a virtual means for reporting virtual joint dynamics.

30. The device of claim 21, wherein:

said computing means is a single processor.

31. The device of claim 21, wherein:

said computing means includes a plurality of processors in the same location or in different locations, or a combination thereof, linked by a communication means.

32. The device of claim 31, wherein:

body dynamics, consisting of the joint dynamics of one or more joints, or limb dynamics, derived for a limb or limbs from the joint dynamics of nearby joints, or any combination thereof, as monitored through reporting means by computing means, are used by computing means to maintain a congruity between the body attitudes of the first and second bodies.

33. The device of claim 21, wherein

reporting means includes a sensor means, operatively connected to said computing means and mounted directly to, or in effective proximity to, body parts, joint control means, or a combination thereof for one of or both bodies, for measuring joint dynamics, body dynamics consisting of the joint dynamics of one or more joints, or limb dynamics derived for a limb or limbs from the joint dynamics of nearby joints, or any combination thereof.

34. The device of claim 21, wherein:

said joint dynamics, body dynamics and limb dynamics include at least one of attitude, position, acceleration, speed, velocity, or pressure.

35. The device of claim 21, wherein:

said computing means monitors said sensor means measurements, said computing means then uses said sensor means measurements to cause

(A) the second body to attempt to emulate the joint dynamics, the body dynamics, or the limb dynamics, or any combination thereof, of the first body, and

(B) the first body to attempt to emulate the joint dynamics, the body dynamics, or the limb dynamics, or any combination thereof, of the second body;

whereby

the joint dynamics, body dynamics, or limb dynamics, or any combination thereof, of the two bodies become or remain essentially synchronized.

36. The device of claim 21, further comprising:

an image capture means for capturing an image; and

an image display means for displaying said images captured by said image capture means;

whereby

because the heads of the two bodies maintain similar or identical attitudes causing the attitude of the image displayed on the image display means to essentially coincide with attitude of the head with the image display means and the head with the image capture means.

37. The device of claim 36, wherein:

a plurality of said image capture means capture images from positions on or near the head of one body and a plurality of said image display means are congruently arrayed on or near the head of the other body.

38. The device of claim 36, wherein:

said image capture means captures an image from the perspective of a single view.

39. The device of claim 36, wherein:

said image capture means includes video cameras or holographic devices or any combination thereof.

40. The device of claim 36, wherein:

said image capture means is capable of providing a transmittable image for transmission between the two bodies.

41. The device of claim 36, wherein:

said image capture means is stereoscopic.

42. The device of claim 36, wherein:

said image display means is operatively connected to said image capture means.

43. The device of claim 36, wherein:

said image display means provides separate images such that a stereoscopic image is formed.

44. The device of claim 36, wherein:

said image display means provides a single image such that a single landscape image is formed.

45. The device of claim 21, wherein:

said computing means applies pressure upon a plurality of first body joints responsive to current pressure on a plurality of second body joints or said computing means applies pressure upon a plurality of second body joints responsive to current pressure on a plurality of first body joints, or any combination thereof;

whereby

the first body experiences a resistance pressure related to burdens or resistances being experienced by the second body and/or the second body forces to be exerted can be modified by the actions of the first body.

46. The device of claim 21, wherein:

said computing means manages a plurality of joints simultaneously, causing groups of joints or the entire body to respond or act in a coordinated manner and have a predictable multi-limb position or a predictable full-body position.

47. The device of claim 21, wherein:

said computing means processes changes to the joint dynamics essentially in real time or with respect to some shared time-frame.

48. The device of claim 21, wherein:

said computer means proportionately scales or adjusts a plurality of data relating to the first body and the second body that are different in size;

whereby

said device is effective even when there are substantial disparities between the sizes of the first body and the second body which also allows one body to easily direct the actions of a very small or very large body.

49. The device of claim 21, further comprising:

a means for communicating sound between the two bodies; and

a sound sensor means, operatively connected to the sound communication means, is attached to the second body and said sound sensor means effectively moves with the second body such that a captured sound signal is relative to a position of the second body in the environment of the second body; and

a sound reproducing means attached to the first body, operatively connected to said means for communicating sound receives said captured sound signal from the sound sensor means of the second body, then reproduces those sounds such that the first body sound reproduction is effectively relative to the corresponding locations of the sound sensor means on the second body;

whereby

the sound reproduced at the first body location is directionally reflective of the sound that originated in the environment of the second body.

50. The device of claim 21, wherein:

said computing means uses the joint dynamics reported by said first reporting means to determine a first plurality of forces on the first body that are not desirable reflections of some or all of a second plurality of forces on the second body, said computing means then directs said first joint control means to exert a third plurality of forces to offset the undesirable portions of the forces on the first body;

whereby

the first body can be aided such that the first body can move more freely while feeling less or none of the forces above and beyond the forces of the second body.

51. A method for providing physical assistance with the undesirable burdens on a body, comprising the steps of:

(a) obtaining a plurality of dynamics measurements from the body;

(b) calculating an amount of the undesired burden upon a body part from the dynamics measurements or a plurality of known data, or any combination thereof;

(c) calculating an amount of force, or an amount of movement, or a combination thereof, to offset some or all of said amount of the undesirable burdens on a body; and

(d) directing a plurality of equipment worn by the body to apply a calculated amount of force, or attempt a calculated amount of movement, or a combination thereof.

52. A method according to claim 51, wherein:

the undesirable burdens include the weight of any burden borne by the body, the weight of the body itself, or the weight of the equipment worn, or any combination thereof.

53. A method according to claim 51, wherein:

said plurality of known data includes the weight of the equipment worn by the body or the weight of the limbs of the body, any other burden of weight borne by the body, or any combination thereof.

54. A method according to claim 51, wherein:

a plurality of sensor information includes a plurality of sensor readings, or a plurality of joint dynamics measurements, or any combination thereof.

55. A device allowing physical bodies to direct the actions of virtual bodies and physically experience a potentially complex virtual environment, comprising;

a first joint control means for controlling the joint dynamics of a first body;

a first reporting means for reporting the joint dynamics of the first body;

a means for providing a virtual reality environment in which a virtual body exists that essentially emulates the first body;

a second reporting means for reporting the joint dynamics of said virtual body;

a second joint control means for controlling the joint dynamics of said virtual body;

a computing means operatively connected to said first joint control means, said second joint control means, said first reporting means, and said second reporting means;

whereby

the virtual body reflects the actions of the first body, the first body may be enforced to be essentially in agreement with some or all of the actual accomplished joint dynamics of the virtual body, and the first body may experience sensations from the actions of the virtual body and virtual body environment which can be used to create a complex simulation without a physical set of objects, since the objects encountered can all be virtual.

56. The device of claim 55, wherein:

said joint dynamics comprise the position, pressure upon, speed of movement, velocity, angle, or direction of movement, or any combination thereof, of a joint or the body part controlled by the joint, or any combination thereof.

57. The device of claim 55, wherein:

said means for providing a virtual reality environment includes a processor means incorporating a set of software instructions which includes said second reporting means and said second joint control means.

58. The device of claim 55, wherein:

said computing means monitors said first and second reporting means and directs said virtual body to attempt to emulate at least one of said plurality of joint dynamics of the first body.

59. The device of claim 55, wherein:

said computing means monitors said first and second reporting means and directs said first body to emulate at least one of said joint dynamics of the second body.

60. The device of claim 55, further comprising:

a means for perceiving sound;

a means for communicating sound between the two bodies;

a virtual sound sensor means, operatively connected to the sound communication means, is attached to the virtual body and said virtual sound sensor means effectively moves with the virtual body such that a captured sound signal is relative to a position of the virtual body in the virtual environment;

a sound reproducing means attached to the first body, operatively connected to said means for communicating sound, receives said captured sound signal from the virtual sound sensor means of the virtual body, then reproduces the perception of those sounds such that the first body sound reproduction is effectively located relative to the corresponding locations of the virtual sound sensor means on the virtual body;

whereby

the sound reproduced at the first body location is directionally reflective of the sound that originated in the virtual environment of the virtual body.

61. The device of claim 55, further comprising:

a means for sensing temperature;

a means for communicating sensed temperature between the two bodies;

a virtual temperature sensor means, operatively connected to the temperature communication means, is attached to the virtual body, said virtual temperature sensor means senses virtual temperatures effectively felt by the virtual body in the virtual environment;

a temperature reproducing means attached to the first body, operatively connected to said means for communicating sensed temperature, receives the captured virtual temperature from the virtual temperature sensor means of the virtual body, then reproduces the physical perception of these temperatures on the first body such that the first body effectively feels the corresponding temperatures of the virtual body;

whereby

the temperature reproduced at the first body location is reflective of the position and the temperature that originated in the virtual environment of the virtual body.

62. An apparatus for the capture and reproduction of vibration signals for the purpose of communicating the sense of touch, comprising:

a sensing means on a first body made up of a plurality of vibration sensitive elements;

a reproductive means arrayed on a second body, said reproductive means reproduces a plurality of signals from said sensing means on the first body;

a communication means operatively connected to said sensing means and said reproductive means for carrying the signals or data from said sensing means to said reproductive means either directly or by recorded delay;

whereby

a sound, pressure, or impact encountered, or any combination thereof by the first body can be sensed at a congruent point on the second body providing a proper sense of location and feel of the contact even when the first body is far from the second body and connected by any form of communication link.

63. The apparatus in claim 62, wherein:

said plurality of vibration sensitive elements produce variations in a carried signal relative to pressure, impact, or vibration, or any combination thereof.

64. The apparatus in claim 62, wherein:

said reproductive means is configured on the second body in essentially the same layout as said sensing means on the first body.

65. An apparatus for the capture and reproduction of heat signals for the purpose of communicating the sense of temperature, comprising:

a sensing means comprising a plurality of temperature sensitive elements associated with a first body;

a reproductive means arrayed on or near a second body in a layout similar to the placement of said sensing means on the first body for effecting changes in temperature on the second body responsive to a plurality of signals or data from said sensing means;

a communications means operatively connected to said sensing means and said reproductive means for carrying the signals or data from said sensing means to said reproductive means either directly or by recorded delay;

whereby

the second body can sense the temperatures and temperature fluctuations experienced on the first body on the appropriate locations of the second body for a natural sense of the first body'"'"'s experience.

66. A device for easily adjusting the balance or weight, or any combination thereof, of robotics equipment comprising:

a center of gravity adjustment means comprising weights associated with a plurality of skeletal elements which can be fixed at different points along the length of the skeletal elements;

whereby

skeletal members of a robot can be modified to match a desired weight, or balance, or both, or be adjusted to have a desired set of balancing characteristics, or any combination thereof, to tune the precision and usefulness of robotics operations without the requirement of redesign or dismantlement.

67. The device of claim 66 further comprising:

a plurality of adjustable length skeletal elements;

an actuation means operatively connected to said adjustable length skeletal elements, or said center of gravity adjustment means, or any combination thereof, for adjusting them;

a computing means for directing the positional control of said adjustable length skeletal elements, or said center of gravity adjustment means, or any combination thereof;

a sensory means operatively connected to said computing means for reporting the positions of said adjustable length skeletal elements and said center of gravity adjustment means, allowing said computer means to accurately set them;

whereby

even very complex robotic assemblies can be rapidly computer-adjusted or adjusted by humans with the aid of a computer interface without redesign or disassembly to be balanced, sized or both to match an ideal design or configuration with that process potentially completely automated when the desired body attributes are known, retrievable or sensed on a body to be emulated.

68. A device for allowing a body to operate in a controlled, virtual or customized environment, comprising:

a plurality of body control equipment for controlling the motion and positions of a body, or providing desired forces upon the body, or a combination thereof;

an interactive virtual reality means operatively connected to said body control equipment for providing haptic response to the body control equipment reflective of objects and actions in the virtual world, or for causing a virtual body to emulate the actions of the body wearing said body control equipment, or any combination thereof;

a motion control means connected to and supporting the body control equipment for providing pitch control, roll control, yaw control, or horizontal motion control, or any combination thereof, to the body;

whereby

the body can experience an environment with a potentially limitless range of perceived travel or activity without ever moving beyond a limited physical area which is applicable to full-immersion virtual reality where the user feels and interacts with and climbs upon a virtual environment which requires no physical simulation room or props to be constructed, fall-proof and computer guided physical therapy, remote control of robotics where the person controlling the remote robotic feels the experience, joint dynamics, and body attitudes of the remote robot, or any application requiring spatial body control.