User interface apparatus, user interface method, game apparatus, and program storage medium

• US 7,056,216 B2
• Filed: 02/22/2000
• Issued: 06/06/2006
• Est. Priority Date: 06/11/1999
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ohshima, Toshikazu
• Primary Examiner(s): Thai, Xuan M.
• Assistant Examiner(s): MOSSER, ROBERT E
• Application Number:
  US09/510,334
• Publication Number:
  US 20020072416A1
• Time in Patent Office: 2,296 Days
• Field of Search: 463 36- 39, 434/252
• US Class Current: 463/36
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

There is disclosed a user interface apparatus to which the user (player) can easily sensuoulsy become accustomed, and which can accurately recognize instructions (commands) that the user (player) intended. The user interface apparatus has a location/posture sensor (100) attached to the head of the user, and a location/posture sensor (200) attached to his or her hand, analyzes hand action on the basis of the relative position of the location/posture of the hand with respect to those of the head, and estimates the current action state from the analysis result. The apparatus then determines a user instruction (player command) on the basis of the action state.

Claims

1. A user interface apparatus comprising:

a first sensor adapted to be attached to a head of a user, wherein said first sensor detects a position and an orientation of the head;

a second sensor adapted to be attached to a second portion of the user, which is different from the head;

an estimating unit arranged to estimate a relative position of the second portion with respect to the position and orientation of the head in accordance with results of detection by said first and second sensors, wherein said estimating unit determines the relative position by transforming coordinates between a head coordinate system that is based on the position and orientation of the head detected by said first sensor and another coordinate system that is based on information detected by said second sensor;

a generation unit arranged to generate command information on the basis of a transition of the estimated relative position; and

an image generating unit arranged to generate an image on the basis of said command information.

2. The apparatus according to claim 1, wherein the second portion is a hand.

3. The apparatus according to claim 1, wherein said second sensor detects a location and orientation of the second portion.

4. The apparatus according to claim 1, wherein the action information includes information which pertains to an orientation of the second portion with respect to the orientation of the first portion.

5. The apparatus according to claim 1, wherein the action information includes information which pertains to a moving direction of a location of the second portion with respect to the orientation of the first portion.

6. The apparatus according to claim 1, further comprising:

means for storing a value of the relative position of the second portion with respect to the first portion, and a plurality of state values which are defined in advance as a result of transition of the value; and

means for storing a plurality of different user instruction values corresponding to the plurality of state values.

7. The apparatus according to claim 1, wherein said determination unit decomposes the determined user instruction into a plurality of instruction operands, and outputs the operands.

8. The apparatus according to claim 1, further comprising a third sensor for detecting a bent angle of a finger.

9. The apparatus according to claim 1, further comprising a head-mounted display for displaying the image generated by said image generating unit.

10. A user interface method for outputting a user instruction to a predetermined apparatus or program, comprising:

a step adapted to detecting a location of a head of a user and a location of a second portion of the user, which is different from the head, by using a first sensor adapted to be attached to the head and a second sensor adapted to be attached to the second portion, wherein the first sensor adapted to detect a position and an orientation of the head;

a step of estimating a relative position of the second portion with respect to the position and orientation of the head in accordance with results of detection by the first and second sensors in said detecting step, wherein said estimating step determines the relative position by transforming coordinates between a head coordinate system that is based on the position and orientation of the head adapted to be detected by the first sensor and another coordinate system that is based on information detected by the second sensor;

a step of generating command information on the basis of a transition of the estimated relative position; and

a step of determining an instruction by the user corresponding to the generated command information and outputting the determined user instruction to the apparatus or program.

11. A computer readable storage medium, which stores a program for controlling an apparatus to output a user instruction to a predetermined apparatus or program, the stored program comprising:

a program step adapted to detecting a location of a head of a user and a location of a second portion of the user, which is different from the head, by using a first sensor adapted to be attached to the head and a second sensor adapted to be attached to the second portion, wherein the first sensor adapted to detect a position and an orientation of the head;

a program step of estimating a relative position of the second portion with respect to the position and orientation of the head in accordance with results of detection by the first and second sensors in said program step of detecting, wherein said estimating step determines the relative position by transforming coordinates between a head coordinate system that is based on the position and orientation of the head adapted to be detected by the first sensor and another coordinate system that is based on information detected by the second sensor;

a program step of generating command information on the basis of a transition of the estimated relative position; and

a program step of determining an instruction by the user corresponding to the generated command information and outputting the determined user instruction to the predetermined apparatus or program.