FORCE SENSE PRESENTATION DEVICE, MIXED REALITY SYSTEM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING APPARATUS

• US 20080059131A1
• Filed: 08/28/2007
• Published: 03/06/2008
• Est. Priority Date: 08/29/2006
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tokita, Toshinobu, Nishimura, Naoki, Nogami, Atsushi
• Granted Patent: US 7,920,124 B2
• Time in Patent Office: Days
• US Class Current: 703/5
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A force sense presentation device for presenting a sense of force in virtual space to a user, comprises: a fixed unit which is gripped by the user; a force sense presentation unit which presents a sense of force; an actuator which supplies a driving force and operates the force sense presentation unit relative to the fixed unit; a joint unit which is provided between the fixed unit and the force sense presentation unit, and guides the relative operation of the force sense presentation unit; and a force sense rendering unit which controls the relative operation of the force sense presentation unit by the actuator, wherein the force sense presentation device simulates a device which is gripped and used by the user, and the force sense rendering unit controls the relative operation of the force sense presentation unit based on a position and orientation of the device in the virtual space.

Claims

1. A force sense presentation device for presenting a sense of force in virtual space to a user, comprising:

a fixed unit which is gripped by the user;

a force sense presentation unit which presents a sense of force;

an actuator which supplies a driving force and operates said force sense presentation unit relative to said fixed unit;

a joint unit which is provided between said fixed unit and said force sense presentation unit, and guides the relative operation of said force sense presentation unit; and

a force sense rendering unit which controls the relative operation of said force sense presentation unit by said actuator,

wherein the force sense presentation device simulates a device which is gripped and used by the user, and

said force sense rendering unit controls the relative operation of said force sense presentation unit based on a position and orientation of the device in the virtual space.

2. The force sense presentation device according to claim 1, wherein said actuator includes at least one of a hydraulic/pneumatic actuator, a solenoid, a DC motor, an AC motor, a stepping motor, a linear motor, an electrostatic motor, an ultrasonic motor, a polymer actuator, a shape memory alloy, a brake, a clutch, an ER fluid, and an MR fluid.

3. The force sense presentation device according to claim 1, wherein said joint unit includes at least one of a guide, a bearing, a hinge, and a link mechanism.

4. The force sense presentation device according to claim 1, wherein said actuator relatively operates said force sense presentation unit in at least one of an oscillation direction, a perpendicular direction, and a parallel direction with respect to a longitudinal direction of the force sense presentation device, and a direction about an axis of the force sense presentation device.

5. The force sense presentation device according to claim 1, further comprising at least one of a ring, a three-dimensional pattern conforming to fingers, and a mark serving as a reference of user'"'"'s grip of the force sense presentation device.

6. The force sense presentation device according to claim 1, wherein said force sense rendering unit comprises:

an interference determination unit which determines interference between the virtual device and a virtual object in the virtual space;

a force sense calculation unit which calculates a sense of force to be presented based on the interference determined by said interference determination unit; and

a control unit which calculates driving information of said actuator based on the calculated sense of force and controls the relative operation of said force sense presentation unit by said actuator based on the driving information.

7. The force sense presentation device according to claim 1, further comprising a measuring unit which measures a relative position and orientation in the virtual space between the device and the virtual object in the virtual space,

wherein said force sense rendering unit further controls the relative operation of said force sense presentation unit based on the position and orientation measured by said measuring unit.

8. The force sense presentation device according to claim 7, further comprising an image output unit which outputs, to a display, an image representing positions and orientations of the virtual object and the device in the virtual space based on the position and orientation measured by said measuring unit.

9. The force sense presentation device according to claim 7, wherein said measuring unit includes at least one of an image sensing unit, an image processing unit, a magnetic sensor, an acceleration sensor, a gyro, a GPS, and an optical sensor.

10. The force sense presentation device according to claim 6, wherein

the virtual object is a virtual thread member, and

said force sense rendering unit controls to virtually present a sense of force in an operation of threadably engaging the thread member by using a machine tool.

11. The force sense presentation device according to claim 6, wherein

the virtual object is a virtual internal thread, and

said force sense rendering unit controls to virtually present a sense of force in an operation of threadably engaging a thread member by using a wrench.

12. The force sense presentation device according to claim 6, wherein

the virtual object is a virtual thread member,

said fixed unit is gripped by one hand of the user,

said force sense presentation unit is gripped by the other hand of the user, and

said force sense rendering unit controls to virtually present a sense of force in an operation of threadably engaging the thread member by using a machine tool.

13. The force sense presentation device according to claim 10, wherein said force sense rendering unit relatively operates said force sense presentation unit in a direction reverse to a direction in which the user moves the force sense presentation device.

14. The force sense presentation device according to claim 13, wherein said force sense rendering unit controls the relative operation based on a speed at which the user moves the force sense presentation device.

15. A mixed reality system including a force sense presentation device for presenting a sense of force in virtual space to a user, an information processing apparatus, and a display, wherein

said force sense presentation device comprises:

a fixed unit which is gripped by the user;

a force sense presentation unit which presents a sense of force;

an actuator which supplies a driving force and operates said force sense presentation unit relative to said fixed unit; and

a joint unit which is provided between said fixed unit and said force sense presentation unit and guides the relative operation of said force sense presentation unit,

the system comprising;

a force sense rendering unit which controls the relative operation of said force sense presentation unit by said actuator;

a measuring unit which measures a relative position and orientation in the virtual space between the device and a virtual object in the virtual space; and

an image output unit which outputs, to said display, an image representing positions and orientations of the virtual object and the device in the virtual space based on the position and orientation measured by said measuring unit,

wherein said force sense presentation device simulates a device which is gripped and used by the user, and

said force sense rendering unit controls the relative operation of said force sense presentation unit based on the position and orientation measured by said measuring unit.

16. The system according to claim 15, further comprising a read unit which reads out map information from a storage unit,

wherein said force sense rendering unit calculates a guide route to guide the user based on the map information and the position and orientation measured by said measuring unit and controls the relative operation based on the guide route.

17. An information processing method executed by an information processing apparatus for controlling a force sense presentation device which has an operation unit to operate a virtual machine tool object simulating a machine tool and presents a sense of force corresponding to a given control value, the method comprising:

the first determination step of determining whether the virtual machine tool object is interfering with a virtual thread object simulating a thread member;

the second determination step of determining whether the virtual thread object is interfering with a virtual threaded hole object simulating a threaded hole;

the mode control step of shifting a control mode of the force sense presentation device from a normal mode to a non-normal mode when the virtual machine tool object is interfering with the virtual thread object, and the virtual thread object is interfering with the virtual threaded hole object;

the calculation step of executing a calculation process of obtaining, based on a revolution count of the virtual thread object, a torque for the virtual machine tool object generated in accordance with interference between the virtual thread object and the virtual threaded hole object when the control mode is shifted to the non-normal mode; and

the transmission step of obtaining a control value to make the force sense presentation device present a sense of force based on the torque obtained in the calculation step and transmitting the control value to the force sense presentation device.

19. The method according to claim 17, wherein the virtual thread object is a virtual object simulating an external thread, and the virtual threaded hole object is a virtual object simulating an internal thread.

20. The method according to claim 17, wherein in the calculation step, the calculation process is executed by using a table in which a torque corresponding to each revolution count of the virtual thread object is registered.

21. The method according to claim 17, wherein in the calculation step, the torque corresponding to the revolution count of the virtual thread object is obtained by using a relational expression of the revolution count and the torque.

22. The method according to claim 20, wherein the torque corresponding to the revolution count of the virtual thread object changes along with an increase in the revolution count.

25. A program stored in a storage medium which causes a computer to execute an information processing method of claim 17.

26. A computer-readable storage medium which stores a program of claim 25.

18. An information processing method executed by an information processing apparatus for controlling a force sense presentation device which presents a sense of force in attaching a virtual thread object simulating a thread member to a virtual threaded hole object simulating a threaded hole, the method comprising:

the determination step of determining an interference state between the virtual thread object and the virtual threaded hole object;

the calculation step of obtaining a torque corresponding to a revolution count of the virtual thread object when it is determined in the determination step that the virtual thread object is interfering with the virtual threaded hole object; and

the control step of controlling to make the force sense presentation device present a sense of force based on the torque obtained in the calculation step.

23. An information processing apparatus for controlling a force sense presentation device which has an operation unit to operate a virtual machine tool object simulating a machine tool and presents a sense of force corresponding to a given control value, the apparatus comprising:

a first determination unit which determines whether the virtual machine tool object is interfering with a virtual thread object simulating a thread member;

a second determination unit which determines whether the virtual thread object is interfering with a virtual threaded hole object simulating a threaded hole;

a mode control unit which shifts a control mode of the force sense presentation device from a normal mode to a non-normal mode when the virtual machine tool object is interfering with the virtual thread object, and the virtual thread object is interfering with the virtual threaded hole object;

a calculation unit which executes a calculation process of obtaining, based on a revolution count of the virtual thread object, a torque for the virtual machine tool object generated in accordance with interference between the virtual thread object and the virtual threaded hole object when the control mode is shifted to the non-normal mode; and

a transmission unit which obtains a control value to make the force sense presentation device present a sense of force based on the torque obtained by said calculation unit and transmits the control value to the force sense presentation device.

24. An information processing apparatus for controlling a force sense presentation device which presents a sense of force in attaching a virtual thread object simulating a thread member to a virtual threaded hole object simulating a threaded hole, the apparatus comprising:

a determination unit which determines an interference state between the virtual thread object and the virtual threaded hole object;

a calculation unit which obtains a torque corresponding to a revolution count of the virtual thread object when it is determined by said determination unit that the virtual thread object is interfering with the virtual threaded hole object; and

a control unit which controls to make the force sense presentation device present a sense of force based on the torque obtained by said calculation unit.