Haptic information presentation system and method

• US 9,171,437 B2
• Filed: 04/03/2013
• Issued: 10/27/2015
• Est. Priority Date: 11/20/2003
• Status: Active Grant

• Current Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Murata Manufacturing Co Limited
• Original Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Miraisens, Inc. (Murata Manufacturing Co Limited)
• Inventors: Nakamura, Norio, Fukui, Yukio, Sakai, Masataka
• Primary Examiner(s): Balaoing, Ariel
• Assistant Examiner(s): LITTLEJOHN JR, MANCIL H
• Application Number:
  US13/855,935
• Publication Number:
  US 20130265149A1
• Time in Patent Office: 937 Days
• Field of Search: 340/407.1, 340/407.2, 341 20- 27, 341/34, 345168-183, 715/701, 715/702
• US Class Current: 1/1
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: G06F 2203/013 Force feedback applied to a game
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G10K 15/04 Sound-producing devices G10K15/02 takes precedence
• CPC Class Codes: H04B 3/36 Repeater circuits H04B3/58 takes precedence

Abstract

A system and method are disclosed in which in a conventional non-grounding man-machine interface having no reaction base on the human body and for giving the existence of a virtual object and the impact force of a collision to a person, a haptic sensation of a torque, a force and the like can be continuously presented in the same direction, which cannot be presented by only the physical characteristic of a haptic sensation presentation device. In a haptic presentation device, the rotation velocity of at least one rotator in the haptic presentation device is controlled by a control device, and a vibration, a force or a torque as the physical characteristic is controlled, so that the user is made to conceive various haptic information of the vibration, force, torque or the like.

Claims

1. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm, of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates a virtual object which presents a haptic sensation on a surface of the display member;

a control device that controls a motion of the virtual object from detection information from the sensor a haptic information presentation device that presents haptic information comprising illusion as if the operator operates real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information, wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator, wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and muscle reflection;

wherein the sensory quantity comprises at least one of;

a sensory quantity where an integral value of the sensory quantity does not become zero even if a further integral value of the physical quantity becomes zero; and

a sensory quantity that does not exist physically.

5. The haptic information processing device according to claim 1, wherein the display member comprises at least one of a touch panel, a screen, an object to be moved, and a virtual reality space.

6. The haptic information processing device according to claim 1, wherein the haptic information processing device is provided in at least one of a virtual reality environment device, a portable information terminal, a game device, a navigation device, a man-machine interface device, a wearable device, a desk device, a glove type device, a clothing type device, a baton type device, a pen-shaped device, a touch panel, a device comprising a button, and a device comprising an operation panel.

7. The haptic information processing device according to claim 1, wherein the physical quantity comprises at least one of force, a vibration, a torque, an acceleration, a deceleration, an air pressure, a video, and sound, and

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or

wherein the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

8. The haptic information processing device according to claim 1, wherein the sensory quantity is changed according to at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state, and/or

wherein the sensory quantity comprises at least one of:

a shape feeling;

an elastic force feeling;

a tactile sensation;

a feeling in which force is transmitted on a presentation object;

a feeling in which an object rolls on a presentation object;

a force sensation;

a vibration sensation;

a torque sensation;

an operation feeling;

an existence feeling;

an elastic feeling;

a texture sensation;

a pushing feeling by the operator;

an expansion feeling;

a pressure feeling;

a pulling feeling by the operator;

a pulled feeling from outside;

a sensation such that the operator feels the weight that is heavier than actual;

a sensation such that the operator feels the weight that is different from actual;

a pushed feeling from outside;

a feeling in which force is transmitted on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which an object rolls on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which force passes through at least one of the finger, the palm, the hand, the leg, the head, and the body;

a texture sensation on a surface of an object;

a feeling such that the operator grasps an object and moves the object;

an intensity sensation and/or an intensity change sensation of at least one of force, a force sensation, a vibration, a vibration sensation, fatigue, and a fatigue sensation;

a feeling such that the operator presses an object by at least one of the finger, the palm, the hand, the leg, the head, and the body;

a three-dimensional shape feeling of a sphere or a cubic;

an elastic feeling of a sphere or a cubic;

a soft feeling of a sphere or a cubic;

a torque sensation only in one direction;

a hysteresis sensation;

a feeling such that the intensity of a presentation physical quantity is corrected;

an operation feeling at a performance speed which the operator can play music, and/or in a playing method which the operator can play the music; and

an existence sensation of an object or a shock sensation due to a collision in a virtual space by presenting force to the operator and/or giving resisting force or reaction force to limit the motion of the operator.

9. The haptic information processing device according to claim 1, wherein the haptic information presentation device comprises at least one of a motor, a piezoelectric element, a magnet, a speaker, a rotator, an eccentric rotator, and a rotation unit.

10. The haptic information processing device according to claim 1, wherein,

the virtual object comprises at least one of a virtual scroll wheel, a virtual button, a virtual joystick, and a virtual operation panel, a real scroll wheel, a real button, a real joystick, and a real operation panel.

11. The haptic information processing device according to claim 1, wherein the control device controls the haptic information presentation device by at least one motion of the finger of the operator, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface thereby to present at least one motion of:

a motion of the virtual object at least one of on a surface of the display member, inside of the display member, and around the surface of the display member;

a motion for grasping the virtual object by at least one movement of the finger, the palm, the hand and the body;

a motion for pushing the virtual object by at least one movement of the finger, the palm, the hand, the leg, the head, and the body;

a motion for presenting presence and/or collision of an object in a virtual space;

a motion due to an pressing operation and/or a touch operation;

an accelerated or decelerated motion;

a motion due to at least one sensitivity change of a vibration sensation, a haptic sensation, and a torque sensation in accordance with at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and

neural firing state;

a motion acting on a muscle spindle in the muscle;

a motion due to at least one change of a vibration sensation, a force sensation, and

a torque sensation by activating at least one of a vibration, force, and a torque in synchronization with the contraction of the muscle;

a motion due to at least one sensation change of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state;

a motion for suitably changing the space distribution of the haptic sensation in conformity with at least one movement of the finger, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface;

a motion due to at least one operation of a posture sensor, a touch panel, and a corresponding button;

a motion due to input information from the detection of the posture and/or the motion of the finger;

a motion for frictioning and/or rubbing the display member;

a motion due to at least one operation of a posture, a position, an acceleration, an deceleration, a switch, and a dial;

a motion for selecting and/or grasping and/or manipulating the virtual object;

a motion for presenting a posture change by detecting at least one of a grasping way, a direction and a position by the posture sensor;

a motion for suppressing the motion of the display member and/or swaying the display member in the horizontal direction by detecting a posture change by the posture sensor;

a motion for presenting force in a normal direction to a direction of the haptic information presentation device;

a motion that is limited by giving resisting force and/or reaction force;

a motion that is limited by giving resisting force and/or reaction force, wherein the resisting force and/or reaction force are spatially and/or temporally changed;

a motion of the virtual object which is presented by at least one of video information, visual information, auditory information, and haptic information;

a motion of the virtual object which is presented by at least one of a shape change, a surface change, movement, and rotation; and

a motion with gyro effect.

12. A sensor for the haptic information processing device according to claim 1.

13. The haptic information processing device according to claim 1, wherein data indicative of at least one of haptic device physical and sensory characteristics is stored in memory.

14. The haptic electronic device according to claim 1, wherein at least one of haptic device physical and sensory characteristics is modified.

2. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates in a space around a surface of the display member;

a control device that controls a motion of the virtual object which presents a haptic sensation through the space, from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator,

wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and muscle reflection,

wherein the sensory quantity comprises at least one of;

a sensory quantity where an integral value of the sensory quantity does not become zero even if a further integral value of the physical quantity becomes zero; and

a sensory quantity that does not exist physically.

15. The haptic information processing device according to claim 2, wherein the display member comprises at least one of a touch panel, a screen, an object to be moved, and a virtual reality space.

16. The haptic information processing device according to claim 2, wherein the haptic information processing device is provided in at least one of a virtual reality environment device, a portable information terminal, a game device, a navigation device, a man-machine interface device, a wearable device, a desk device, a glove type device, a clothing type device, a baton type device, a pen-shaped device, a touch panel, a device comprising a button, and a device comprising an operation panel.

17. The haptic information processing device according to claim 2, wherein the physical quantity comprises at least one of force, a vibration, a torque, an acceleration, a deceleration, an air pressure, a video, and sound, and/or

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or

wherein the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

18. The haptic information processing device according to claim 2, wherein the sensory quantity is changed according to at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state, and/or

wherein the sensory quantity comprises at least one of:

a shape feeling;

an elastic force feeling;

a tactile sensation;

a feeling in which force is transmitted on a presentation object;

a feeling in which an object rolls on a presentation object;

a force sensation;

a vibration sensation;

a torque sensation;

an operation feeling;

an existence feeling;

an elastic feeling;

a texture sensation;

a pushing feeling by the operator;

an expansion feeling;

a pressure feeling;

a pulling feeling by the operator;

a pulled feeling from outside;

a sensation such that the operator feels the weight that is heavier than actual;

a sensation such that the operator feels the weight that is different from actual;

a pushed feeling from outside;

a feeling in which force is transmitted on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which an object rolls on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which force passes through at least one of the finger, the palm, the hand, the leg, the head, and the body;

a texture sensation on a surface of an object;

a feeling such that the operator grasps an object and moves the object;

an intensity sensation and/or an intensity change sensation of at least one of force, a force sensation, a vibration, a vibration sensation, fatigue, and a fatigue sensation;

a feeling such that the operator presses an object by at least one of the finger, the palm, the hand, the leg, the head, and the body;

a three-dimensional shape feeling of a sphere or a cubic;

an elastic feeling of a sphere or a cubic;

a soft feeling of a sphere or a cubic;

a torque sensation only in one direction;

a hysteresis sensation;

a feeling such that the intensity of a presentation physical quantity is corrected;

an operation feeling at a performance speed which the operator can play music, and/or in a playing method which the operator can play the music; and

an existence sensation of an object or a shock sensation due to a collision in a virtual space by presenting force to the operator and/or giving resisting force or reaction force to limit the motion of the operator.

19. The haptic information processing device according to claim 2, wherein the haptic information presentation device comprises at least one of a motor, a piezoelectric element, a magnet, a speaker, a rotator, an eccentric rotator, and a rotation unit.

20. The haptic information processing device according to claim 2, wherein

the virtual object comprises at least one of a virtual scroll wheel, a virtual button, a virtual joystick, and a virtual operation panel, a real scroll wheel, a real button, a real joystick, and a real operation panel.

21. The haptic information processing device according to claim 2, wherein the control device controls the haptic information presentation device by at least one motion of the finger of the operator, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface thereby to present at least one motion of:

a motion of the virtual object at least one of on a surface of the display member, inside of the display member, and around the surface of the display member;

a motion for grasping the virtual object by at least one movement of the finger, the palm, the hand and the body;

a motion for pushing the virtual object by at least one movement of the finger, the palm, the hand, the leg, the head, and the body;

a motion for presenting presence and/or collision of an object in a virtual space;

a motion due to an pressing operation and/or a touch operation;

an accelerated or decelerated motion;

a motion due to at least one sensitivity change of a vibration sensation, a haptic sensation, and a torque sensation in accordance with at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and

neural firing state;

a motion acting on a muscle spindle in the muscle;

a motion due to at least one change of a vibration sensation, a force sensation, and

a torque sensation by activating at least one of a vibration, force, and a torque in synchronization with the contraction of the muscle;

a motion due to at least one sensation change of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state;

a motion for suitably changing the space distribution of the haptic sensation in conformity with at least one movement of the finger, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface;

a motion due to at least one operation of a posture sensor, a touch panel, and a corresponding button;

a motion due to input information from the detection of the posture and/or the motion of the finger;

a motion for frictioning and/or rubbing the display member;

a motion due to at least one operation of a posture, a position, an acceleration, an deceleration, a switch, and a dial;

a motion for selecting and/or grasping and/or manipulating the virtual object;

a motion for presenting a posture change by detecting at least one of a grasping way, a direction and a position by the posture sensor;

a motion for suppressing the motion of the display member and/or swaying the display member in the horizontal direction by detecting a posture change by the posture sensor;

a motion for presenting force in a normal direction to a direction of the haptic information presentation device;

a motion that is limited by giving resisting force and/or reaction force;

a motion that is limited by giving resisting force and/or reaction force, wherein the resisting force and/or reaction force are spatially and/or temporally changed;

a motion of the virtual object which is presented by at least one of video information, visual information, auditory information, and haptic information;

a motion of the virtual object which is presented by at least one of a shape change, a surface change, movement, and rotation; and

a motion with gyro effect.

22. A sensor for the haptic information processing device according to claim 2.

23. The haptic information processing device according to claim 2, wherein data indicative of at least one of haptic device physical and sensory characteristics is stored in memory.

24. The haptic electronic device according to claim 2, wherein at least one of haptic device physical and sensory characteristics is modified.

3. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates on a surface of the display member;

a control device that controls a motion of a virtual object which presents a haptic sensation around the surface of the display member from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to operator, or

(ii) a sensory quantity to be presented to the operator, wherein the haptic sensory characteristic is obtained by using at least one of the nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and, muscle reflection, and

wherein the sensory quantity comprises at least one of;

a sensory quantity where an integral value of the sensory quantity does not become zero even if a further integral value of the physical quantity becomes zero; and

a sensory quantity that does not exist physically.

25. The haptic information processing device according to claim 3, wherein the display member comprises at least one of a touch panel, a screen, an object to be moved, and a virtual reality space.

26. The haptic information processing device according to claim 3, wherein the haptic information processing device is provided in at least one of a virtual reality environment device, a portable information terminal, a game device, a navigation device, a man-machine interface device, a wearable device, a desk device, a glove type device, a clothing type device, a baton type device, a pen-shaped device, a touch panel, a device comprising a button, and a device comprising an operation panel.

27. The haptic information processing device according to claim 3, wherein the physical quantity comprises at least one of force, a vibration, a torque, an acceleration, a deceleration, an air pressure, a video, and sound, and/or

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or

wherein the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

28. The haptic information processing device according to claim 3, wherein the sensory quantity is changed according to at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state, and/or

wherein the sensory quantity comprises at least one of:

a shape feeling;

an elastic force feeling;

a tactile sensation;

a feeling in which force is transmitted on a presentation object;

a feeling in which an object rolls on a presentation object;

a force sensation;

a vibration sensation;

a torque sensation;

an operation feeling;

an existence feeling;

an elastic feeling;

a texture sensation;

a pushing feeling by the operator;

an expansion feeling;

a pressure feeling;

a pulling feeling by the operator;

a pulled feeling from outside;

a sensation such that the operator feels the weight that is heavier than actual;

a sensation such that the operator feels the weight that is different from actual;

a pushed feeling from outside;

a feeling in which force is transmitted on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which an object rolls on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which force passes through at least one of the finger, the palm, the hand, the leg, the head, and the body;

a texture sensation on a surface of an object;

a feeling such that the operator grasps an object and moves the object;

an intensity sensation and/or an intensity change sensation of at least one of force, a force sensation, a vibration, a vibration sensation, fatigue, and a fatigue sensation;

a feeling such that the operator presses an object by at least one of the finger, the palm, the hand, the leg, the head, and the body;

a three-dimensional shape feeling of a sphere or a cubic;

an elastic feeling of a sphere or a cubic;

a soft feeling of a sphere or a cubic;

a torque sensation only in one direction;

a hysteresis sensation;

a feeling such that the intensity of a presentation physical quantity is corrected;

an operation feeling at a performance speed which the operator can play music, and/or in a playing method which the operator can play the music; and

an existence sensation of an object or a shock sensation due to a collision in a virtual space by presenting force to the operator and/or giving resisting force or reaction force to limit the motion of the operator.

29. The haptic information processing device according to claim 3, wherein the haptic information presentation device comprises at least one of a motor, a piezoelectric element, a magnet, a speaker, a rotator, an eccentric rotator, and a rotation unit.

30. The haptic information processing device according to claim 3, wherein

the virtual object comprises at least one of a virtual scroll wheel, a virtual button, a virtual joystick, and a virtual operation panel, a real scroll wheel, a real button, a real joystick, and a real operation panel.

31. The haptic information processing device according to claim 3, wherein the control device controls the haptic information presentation device by at least one motion of the finger of the operator, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface thereby to present at least one motion of:

a motion of the virtual object at least one of on a surface of the display member, inside of the display member, and around the surface of the display member;

a motion for grasping the virtual object by at least one movement of the finger, the palm, the hand and the body;

a motion for pushing the virtual object by at least one movement of the finger, the palm, the hand, the leg, the head, and the body;

a motion for presenting presence and/or collision of an object in a virtual space;

a motion due to an pressing operation and/or a touch operation;

an accelerated or decelerated motion;

a motion due to at least one sensitivity change of a vibration sensation, a haptic sensation, and a torque sensation in accordance with at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and

neural firing state;

a motion acting on a muscle spindle in the muscle;

a motion due to at least one change of a vibration sensation, a force sensation, and

a torque sensation by activating at least one of a vibration, force, and a torque in synchronization with the contraction of the muscle;

a motion due to at least one sensation change of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state;

a motion for suitably changing the space distribution of the haptic sensation in conformity with at least one movement of the finger, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface;

a motion due to at least one operation of a posture sensor, a touch panel, and a corresponding button;

a motion due to input information from the detection of the posture and/or the motion of the finger;

a motion for frictioning and/or rubbing the display member;

a motion due to at least one operation of a posture, a position, an acceleration, an deceleration, a switch, and a dial;

a motion for selecting and/or grasping and/or manipulating the virtual object;

a motion for presenting a posture change by detecting at least one of a grasping way, a direction and a position by the posture sensor;

a motion for suppressing the motion of the display member and/or swaying the display member in the horizontal direction by detecting a posture change by the posture sensor;

a motion for presenting force in a normal direction to a direction of the haptic information presentation device;

a motion that is limited by giving resisting force and/or reaction force;

a motion that is limited by giving resisting force and/or reaction force, wherein the resisting force and/or reaction force are spatially and/or temporally changed;

a motion of the virtual object which is presented by at least one of video information, visual information, auditory information, and haptic information;

a motion of the virtual object which is presented by at least one of a shape change, a surface change, movement, and rotation; and

a motion with gyro effect.

32. A sensor for the haptic information processing device according to claim 3.

33. The haptic information processing device according to claim 3, wherein data indicative of at least one of haptic device physical and sensory characteristics is stored in memory.

34. The haptic electronic device according to claim 3, wherein at least one of haptic device physical and sensory characteristics is modified.

4. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates a haptic sensation around the display member;

a control device that controls a motion of a virtual object from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator,

wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and, muscle reflection, and

wherein the sensory quantity comprises at least one of;

a sensory quantity where an integral value of the sensory quantity does not become zero even if a further integral value of the physical quantity becomes zero; and

a sensory quantity that does not exist physically.

35. The haptic information processing device according to claim 4, wherein the display member comprises at least one of a touch panel, a screen, an object to be moved, and a virtual reality space.

36. The haptic information processing device according to claim 4, wherein the haptic information processing device is provided in at least one of a virtual reality environment device, a portable information terminal, a game device, a navigation device, a man-machine interface device, a wearable device, a desk device, a glove type device, a clothing type device, a baton type device, a pen-shaped device, a touch panel, a device comprising a button, and a device comprising an operation panel.

37. The haptic information processing device according to claim 4, wherein the physical quantity comprises at least one of force, a vibration, a torque, an acceleration, a deceleration, an air pressure, a video, and sound, and/or

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or

wherein the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

38. The haptic information processing device according to claim 4, wherein the sensory quantity that does not exist physically is said sensory quantity where an integral value of the sensory quantity does not become zero even if the further integral value of the physical quantity becomes zero,

wherein the haptic sensory characteristic comprises at least one of:

a nonlinear characteristic;

a hysteresis characteristic;

a masking characteristic relating to a haptic sense;

a threshold characteristic; and

muscle reflection.

39. The haptic information processing device according to claim 4, wherein the sensory quantity is changed according to at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state, and/or

wherein the sensory quantity comprises at least one of:

a shape feeling;

an elastic force feeling;

a tactile sensation;

a feeling in which force is transmitted on a presentation object;

a feeling in which an object rolls on a presentation object;

a force sensation;

a vibration sensation;

a torque sensation;

an operation feeling;

an existence feeling;

an elastic feeling;

a texture sensation;

a pushing feeling by the operator;

an expansion feeling;

a pressure feeling;

a pulling feeling by the operator;

a pulled feeling from outside;

a sensation such that the operator feels the weight that is heavier than actual;

a sensation such that the operator feels the weight that is different from actual;

a pushed feeling from outside;

a feeling in which force is transmitted on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which an object rolls on at least one of the finger, the palm, the hand, the leg, the head, and the body;

a feeling in which force passes through at least one of the finger, the palm, the hand, the leg, the head, and the body;

a texture sensation on a surface of an object;

a feeling such that the operator grasps an object and moves the object;

an intensity sensation and/or an intensity change sensation of at least one of force, a force sensation, a vibration, a vibration sensation, fatigue, and a fatigue sensation;

a feeling such that the operator presses an object by at least one of the finger, the palm, the hand, the leg, the head, and the body;

a three-dimensional shape feeling of a sphere or a cubic;

an elastic feeling of a sphere or a cubic;

a soft feeling of a sphere or a cubic;

a torque sensation only in one direction;

a hysteresis sensation;

a feeling such that the intensity of a presentation physical quantity is corrected;

an operation feeling at a performance speed which the operator can play music, and/or in a playing method which the operator can play the music; and

an existence sensation of an object or a shock sensation due to a collision in a virtual space by presenting force to the operator and/or giving resisting force or reaction force to limit the motion of the operator.

40. The haptic information processing device according to claim 4, wherein the haptic information presentation device comprises at least one of a motor, a piezoelectric element, a magnet, a speaker, a rotator, an eccentric rotator, and a rotation unit.

41. The haptic information processing device according to claim 4, wherein

the virtual object comprises at least one of a virtual scroll wheel, a virtual button, a virtual joystick, and a virtual operation panel, a real scroll wheel, a real button, a real joystick, and a real operation panel.

42. The haptic information processing device according to claim 4, wherein the control device controls the haptic information presentation device by at least one motion of the finger of the operator, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface thereby to present at least one motion of:

a motion of the virtual object at least one of on a surface of the display member, inside of the display member, and around the surface of the display member;

a motion for grasping the virtual object by at least one movement of the finger, the palm, the hand and the body;

a motion for pushing the virtual object by at least one movement of the finger, the palm, the hand, the leg, the head, and the body;

a motion for presenting presence and/or collision of an object in a virtual space;

a motion due to an pressing operation and/or a touch operation;

an accelerated or decelerated motion;

a motion due to at least one sensitivity change of a vibration sensation, a haptic sensation, and a torque sensation in accordance with at least one of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and

neural firing state;

a motion acting on a muscle spindle in the muscle;

a motion due to at least one change of a vibration sensation, a force sensation, and

a torque sensation by activating at least one of a vibration, force, and a torque in synchronization with the contraction of the muscle;

a motion due to at least one sensation change of muscle tensile state, physical state, physiological state, psychological state, breath state, posture state, and neural firing state;

a motion for suitably changing the space distribution of the haptic sensation in conformity with at least one movement of the finger, the palm, the hand, the leg, the head, the body, the operation device, and the man-machine interface;

a motion due to at least one operation of a posture sensor, a touch panel, and a corresponding button;

a motion due to input information from the detection of the posture and/or the motion of the finger;

a motion for frictioning and/or rubbing the display member;

a motion due to at least one operation of a posture, a position, an acceleration, an deceleration, a switch, and a dial;

a motion for selecting and/or grasping and/or manipulating the virtual object;

a motion for presenting a posture change by detecting at least one of a grasping way, a direction and a position by the posture sensor;

a motion for suppressing the motion of the display member and/or swaying the display member in the horizontal direction by detecting a posture change by the posture sensor;

a motion for presenting force in a normal direction to a direction of the haptic information presentation device;

a motion that is limited by giving resisting force and/or reaction force;

a motion that is limited by giving resisting force and/or reaction force, wherein the resisting force and/or reaction force are spatially and/or temporally changed;

a motion of the virtual object which is presented by at least one of video information, visual information, auditory information, and haptic information;

a motion of the virtual object which is presented by at least one of a shape change, a surface change, movement, and rotation; and

a motion with gyro effect.

43. A sensor for the haptic information processing device according to claim 4.

44. The haptic information processing device according to claim 4, wherein data indicative of at least one of haptic device physical and sensory characteristics is stored in memory.

45. The haptic electronic device according to claim 4, wherein at least one of haptic device physical and sensory characteristics is modified.

46. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates a virtual object which presents a haptic sensation on a surface of the display member;

a control device that controls a motion of the virtual object from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator,

wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and muscle reflection,

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

47. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates in a space around a surface of the display member;

a control device that controls a motion of the virtual object which presents a haptic sensation through the space, from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator,

wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and muscle reflection, and

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

48. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates on a surface of the display member;

a control device that controls a motion of a virtual object which presents a haptic sensation around the surface of the display member from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator,

wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and muscle reflection, and

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.

49. A haptic information processing device comprising:

a display member;

a sensor that detects at least one operation of a finger of an operator, a palm of the operator, a hand of the operator, a leg of the operator, a head of the operator, a body of the operator, an operation device, and a man-machine interface, which operates a haptic sensation around the display member;

a control device that controls a motion of a virtual object from detection information from the sensor;

a haptic information presentation device that presents haptic information comprising illusion as if the operator operates a real object, from the detection information from the sensor by applying a haptic sensory characteristic of the operator to the virtual object; and

a transmitter and/or a receiver that perform transmission and/or reception of the haptic information,

wherein the haptic sensory characteristic comprises;

(i) one or more of a physical quantity being given to the operator and a physical quantity being brought by the operation of the operator, and a sensory quantity to be presented to the operator, or

(ii) a sensory quantity to be presented to the operator,

wherein the haptic sensory characteristic is obtained by using at least one of a nonlinear characteristic, a hysteresis characteristic, a masking characteristic relating to a haptic sense, a threshold characteristic, and muscle reflection, and

wherein the physical quantity comprises a plurality of physical quantities corresponding to a plurality of operation points, and the plurality of operation points on a curve representing the relationship of the haptic sensory characteristic are utilized, and/or the physical quantity is obtained by setting each of operation points on a curve representing the relationship of the haptic sensory characteristic and each operation duration time on each of operation points.