Information input/output device, information input/output method and computer program

• US 8,648,797 B2
• Filed: 05/06/2009
• Issued: 02/11/2014
• Est. Priority Date: 05/08/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Nagasaka, Kenichiro
• Primary Examiner(s): MANDEVILLE, JASON M
• Application Number:
  US12/436,222
• Publication Number:
  US 20090282331A1
• Time in Patent Office: 1,742 Days
• Field of Search: 345156-158, 710/1, 710/5, 710/15, 710 18- 19
• US Class Current: 345/156
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

An information input/output device includes: a position input unit configured to designate a position by a user in a prescribed operation area S_d in real space; a designated position measurement unit configured to measure a designated position “r_d” in the operation area S_d, which is designated by the user in the position input unit; an operation area offset amount determination unit configured to determine an offset amount “Δr_v0” of an origin position “r_v0” of an operation area S_v in virtual space based on the measured position “r_d” in the operation area S_d; and a designated position determination unit configured to determine a designated position “r_v” in virtual space which corresponds to the position in real space designated by the user through the position input unit based on the measured position “r_d” in the operation area S_d, the origin position “r_v0” of the operation area S_v in virtual space and the determined offset amount “Δr_v0”.

Claims

1. An information input/output device comprising:

a position input unit, including a gimbaled finger holder portion holding a fingertip of a user by inserting the fingertip, configured to control a pointer in virtual space by designating a position by the user in a prescribed operation area S_d in real space, and configured to feed back force in real space with respect to the fingertip through driven joints connected to the finger holder portion, the operation area S_d being divided into;

a movement area at the peripheral edges of the operation area S_d, and

an interaction area inside the movement area;

a designated position measurement unit configured to measure a designated position “

r_d”

in the operation area S_d, which is designated by the user in the position input unit;

an operation area offset amount determination unit configured to determine whether the designated position “

r_d”

is in the movement area or the interaction area and to determine an offset amount “

Δ

r_v0”

of an origin position “

r_v0”

of a coordinate system of an operation area S_v in virtual space corresponding to the operation area S_d in real space based on the measured position “

r_d”

in the operation area S_d, wherein when the designated position “

r_d”

is in the movement area, the origin position “

r_v0”

is moved in the operation area S_v in virtual space based on the offset amount “

Δ

r_v0”

, and when the designated position “

r_d”

is in the interaction area, the origin position “

r_v0”

is not moved based on the offset amount “

Δ

r_v0” and

a feed back force is limited to interactions with virtual objects corresponding to the operation area S_v in virtual space within the interaction area of the operation area S_d in real space based on a location of the pointer;

a designated position determination unit configured to determine a designated position “

r_v”

in virtual space which corresponds to the position in real space designated by the user through the position input unit based on the measured position “

r_d”

in the operation area S_d, the origin position “

r_v0”

of the operation area S_v in virtual space and the determined offset amount “

Δ

r_v0”

; and

a camera position determination unit configured to move a coordinate system of a camera model for drawing a state in the virtual space according to the movement of the origin position “

r_v0”

based on the offset amount “

Δ

r_v0”

.

2. The information input/output device according to claim 1, wherein the position input unit abuts on a fingertip or other parts of a body of the user, inputting “

r_d”

based on motion of the abutting portion as the designated position.

3. The information input/output device according to claim 2, further comprising:

a force/tactile feedback means for applying a force/tactile sense according to the position “

r_v”

in virtual space determined by the designated position determination unit to part of the body of the user abutting on the position input unit.

4. The information input/output device according to claim 3, wherein the force/tactile feedback means includes

an object data accumulation unit configured to accumulate attribute information including geometric attribute and dynamic attribute possessed by respective objects existing in virtual space,

a dynamics computation unit configured to perform dynamics computation with respect to respective objects accumulated in the object data accumulation unit as well as changing positions and postures of respective objects based on the computation results,

a collision detection unit configured to detect a collision event between respective objects accumulated in the object data accumulation unit,

a collision force calculation unit configured to calculate force acted at a collision point detected in the collision detection unit,

an action force calculation means for calculating an action force to be applied to part of the body of the user which abuts on the position input unit based on the force calculated in the collision force calculation unit, and

a force/tactile addition means for applying a force/tactile sense based on the action force calculated by the action force calculation means to part of the body of the user which abuts on the position input unit.

5. The information input/output device according to claim 1, further comprising:

a display;

and

a virtual environment drawing unit configured to render the state in virtual space from the viewpoint of the coordinate system defined as the camera model and displaying the state on the display.

6. The information input/output device according to claim 2, further comprising:

plural position input units respectively corresponding to plural fingers “

j”

of the user (“

j”

is a positive integer); and

wherein the operation area offset amount determination unit determines the offset amount “

Δ

r_v0”

of the origin position “

r_v0”

of the operation area S_v in virtual space based on an average position or a weighted average position of designated positions “

r_jd”

in the operation area S_d designated by respective position input units.

7. An information input/output device comprising:

a position input unit including a gimbaled finger holder portion holding a fingertip of a user by inserting the fingertip, configured to control a pointer in virtual space by designating a position by the user in a prescribed operation area S_d in real space, formed so as to perform rotational motion, taking a given front posture as the center, in which the user designates a posture angle as a position in a given operation area S_d in real space, and configured to feed back force in real space with respect to the fingertip through driven joints connected to the finger holder portion, the operation area S_d being divided into;

an interaction area in a range of a posture angle ±

Θ

with respect to the front posture, and

a movement area which is outside the interaction area;

a designated position measurement unit configured to measure a posture angle R_d designated in the position input unit;

an operation area offset amount determination unit configured to determine whether the measured designated posture angle R_d is in the interaction area or the movement area, and to determine an offset amount “

Δ

γ

_v0”

of a front posture angle “

R_v0”

of a coordinate system of an operation area S_v in virtual space corresponding to the operation area S_d in real space based on the measured posture angle R_d, wherein when the designated posture angle “

R_d”

is in the movement area, the posture angle “

R_v0”

is moved in the operation area S_v in virtual space based on the offset amount “

Δ

γ

_v0”

, and when the designated posture angle R_d is in the interaction area, the posture angle “

R_v0”

is not moved based on the offset amount “

Δ

γ

_v0” and

a feed back force is limited to interactions with virtual objects corresponding to the operation area S_v in virtual space within the interaction area of the operation area S_d in real space based on a location of the pointer;

a designated position determination unit configured to determine a designated posture R_v in virtual space corresponding to the position in real space designated by the user through the position input unit based on the measured designated posture angle R_d in the operation area S_d, the front posture angle “

R_v0”

of the operation area S_v in virtual space and the determined offset amount “

Δ

γ

_v0”

; and

a camera position determination unit configured to move a coordinate system of a camera model for drawing a state in the virtual space according to the movement of the front posture angle “

R_v0”

based on the offset amount “

Δ

γ

_v0”

.

8. The information input/output device according to claim 7, further comprising:

a display;

and

a virtual environment drawing unit configured to render the state in virtual space from the viewpoint of the coordinate system defined as the camera model and displaying the state on the display.

9. An information input/output method comprising the steps of:

acquiring a designated position “

r_d”

in a prescribed operation area S_d designated by a user through a position input unit which controls a pointer in virtual space by designating a position by the user in the operation area S_d in real space, the position input unit including a gimbaled finger holder portion holding a fingertip of a user by inserting the fingertip, and the position input unit configured to feed back force in real space with respect to the fingertip through driven joints connected to the finger holder portion, the operation area S_d being divided into;

a movement area at the peripheral edges of the operation area S_d, and

an interaction area inside the movement area;

determining whether the designated position “

r_d”

is in the movement area or the interaction area;

determining an offset amount “

Δ

r_v0”

of an origin position “

r_v0”

of a coordinate system of an operation area S_v in virtual space corresponding to the operation area S_d in real space based on the acquired position “

r_d”

in the operation area S_d, wherein when the designated position “

r_d”

is in the movement area, the origin position “

r_v0”

is moved based on the offset amount “

Δ

r_v0” and

when the designated position “

r_d”

is in the interaction area, the origin position “

r_v0”

is not moved based on the offset amount “

Δ

r_v0” and

a feed back force is limited to interactions with virtual objects corresponding to the operation area S_v in virtual space within the interaction area of the operation area S_d in real space based on a location of the pointer;

determining a designated position “

r_v”

in virtual space which corresponds to the position in real space designated by the user through the position input unit based on the acquired position “

r_d”

in the operation area S_d, the origin position “

r_v0”

of the operation area S_v in virtual space and the determined offset amount “

Δ

r_v0”

; and

moving a coordinate system of a camera model for drawing a state in the virtual space according to the movement of the origin position “

r_v0”

based on the offset amount “

Δ

r_v0”

.

10. A non-transitory computer-readable medium storing a computer program that, when executed by a computer, causes the computer to execute:

a designated position acquiring process acquiring a designated position “

r_d”

in a prescribed operation area S_d designated by the user through a position input unit configured to control a pointer in virtual space by designating a position by the user in the operation area S_d in real space, the position input unit including a gimbaled finger holder portion holding a fingertip of a user by inserting the fingertip, and the position input unit configured to feed back force in real space with respect to the fingertip through driven joints connected to the finger holder portion, the operation area S_d being divided into;

a movement area at the peripheral edges of the operation area S_d, and

an interaction area inside the movement area;

an operation area offset amount determination process determining whether the designated position “

r_d”

is in the movement area or the interaction area, and determining an offset amount “

Δ

r_v0”

of an origin position “

r_v0”

of a coordinate system of an operation area S_v in virtual space corresponding to the operation area S_d in real space based on the acquired position “

r_d”

in the operation area S_d, wherein when the designated position “

r_d”

is in the movement area, the origin position “

r_v0”

is moved in the operation area S_v in virtual space based on the offset amount “

Δ

r_v0”

, and when the designated position “

r_d”

is in the interaction area, the origin position “

r_v0”

is not moved based on the offset amount “

Δ

r_v0” and

a feed back force is limited to interactions with virtual objects corresponding to the operation area S_v in virtual space within the interaction area of the operation area S_d in real space based on a location of the pointer;

a designated position determination process determining a designated position “

r_v”

in virtual space which corresponds to the position in real space designated by the user through the position input unit based on the acquired position “

r_d”

in the operation area S_d, the origin position “

r_v0”

of the operation area S_v in virtual space and the determined offset amount “

Δ

r_v0”

; and

a camera position determination process moving a coordinate system of a camera model for drawing a state in the virtual space according to the movement of the origin position “

r_v0”

based on the offset amount “

Δ

r_v0”

.