INSTRUCTION INPUT DEVICE, INSTRUCTION INPUT METHOD, PROGRAM, RECORDING MEDIUM, AND INTEGRATED CIRCUIT

• US 20130002551A1
• Filed: 06/16/2011
• Published: 01/03/2013
• Est. Priority Date: 06/17/2010
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Corporation of America (Panasonic Holdings Corporation)
• Inventors: Maeda, Shigenori, Imoto, Hiroyasu
• Granted Patent: US 8,933,886 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

An instruction input device includes: a first direction detection unit detecting the first direction in which the user is looking; a second direction detection unit detecting a second direction in which the user is performing a pointing operation; a gaze position calculation unit calculating a gaze position of the user on the screen; a reference coordinate group calculation unit calculating a reference line in space corresponding to the gaze position and connecting the user and the screen; an offset amount calculation unit calculating a distance of the second direction between the reference line and the input coordinate indicating the user'"'"'s hand as an offset amount with respect to the gaze position; and a pointer display position calculation unit calculating a position in which a distance of the first screen predetermined direction between the position and the gaze position is the offset amount on the screen.

Claims

1-20. -20. (canceled)

21. An instruction input device which calculates a display position of a pointer to be displayed on a screen, based on a pointing operation of movement of a hand of a user, the instruction input device comprising:

a user position detection unit configured to detect a position of the user;

an input coordinate detection unit configured to detect, as an input coordinate, a coordinate indicating a position of the hand of the user;

a first direction detection unit configured to detect, based on a body posture of the user, a first direction which is determined with respect to a screen vertical direction and is a direction in which the user is looking;

a second direction detection unit configured to detect, based on the body posture of the user, a second direction which is determined with respect to a screen horizontal direction and is a direction in which the user is performing the pointing operation;

a gaze position calculation unit configured to calculate a gaze position of the user on the screen, based on the position of the user and the first direction;

a reference line calculation unit configured to calculate a reference line in space between the user and the screen, based on the position of the user and the first direction, the reference line being a line corresponding to the gaze position;

an offset amount calculation unit configured to calculate a distance in the second direction between the reference line and the input coordinate as an offset amount with respect to the gaze position in a first screen predetermined direction along the screen; and

a pointer display position calculation unit configured to calculate, as the display position of the pointer to be displayed on the screen, a position in which a distance of the first screen predetermined direction between the position and the gaze position is the offset amount on the screen.

22. The instruction input device according to claim 21,

wherein the position of the user is a face position or a head position of the user.

23. The instruction input device according to claim 21, further comprising a three-dimensional coordinate input device which outputs a self position when held by the user,

wherein the input coordinate detection unit is configured to detect, as the input coordinate, a coordinate of the self position output by the three-dimensional coordinate input device.

24. The instruction input device according to claim 21, further comprising

an imaging unit configured to capture an image of the user,

wherein the input coordinate detection unit is configured to detect the input coordinate indicating the position of the hand of the user, based on an image captured by the imaging unit.

25. The instruction input device according to claim 24,

wherein the input coordinate detection unit is further configured to recognize right and left hands of the user based on an image captured by the imaging unit, and detect the input coordinate for each of the recognized right and left hands.

26. The instruction input device according to claim 21,

wherein the first direction detection unit is configured to detect the first direction based on the body posture of the user which includes at least one of a face orientation and a gaze.

27. The instruction input device according to claim 21,

wherein the second direction detection unit is configured to detect the second direction based on the body posture of the user which includes at least a body orientation.

28. The instruction input device according to claim 27,

wherein the second direction detection unit is configured to detect the second direction based on the body posture of the user and a direction of the screen with respect to the position of the user.

29. The instruction input device according to claim 21,

wherein the reference line calculation unit is configured to calculate the reference line based on the position of the user, the first direction, and the second direction, in the case where an angle between the first direction and the second direction is smaller than a predetermined angle.

30. The instruction input device according to claim 21, further comprising

a user information database which stores information indicating at least a correspondence relationship between a head position and a chest position of the user,

wherein the reference line calculation unit is configured to calculate the reference line based on the position of the user, the first direction, and information indicating the correspondence relationship obtained from the user information database.

31. The instruction input device according to claim 21, further comprising:

a user information database which stores information indicating at least a chest position of each user; and

a user recognition unit configured to recognize the user,

wherein the reference line calculation unit is configured to calculate the reference line based on the position of the user, the first direction, and information which indicates at least the chest position of the user and is obtained from the user information database based on user recognition information indicating the user recognized by the user recognition unit.

32. The instruction input device according to claim 30,

wherein the reference line calculation unit is configured to calculate the reference line based on the position of the user, the first direction, information indicating the correspondence relationship obtained from the user information database, and the second direction, in the case where an angle between the first direction and the second direction is smaller than a predetermined angle.

33. The instruction input device according to claim 21, further comprising

a user information database which stores information indicating at least a correspondence relationship between a head position and an arm length of the user,

wherein the offset amount calculation unit is configured to calculate the offset amount based on the second direction and information indicating the correspondence relationship obtained from the user information database.

34. The instruction input device according to claim 21, further comprising:

a user information database which stores information indicating at least an arm length for each user; and

a user recognition unit configured to recognize the user,

wherein the offset amount calculation unit is configured to calculate the offset amount based on the second direction and information which indicates at least the arm length of the user, and

the information is obtained from the user information database based on user recognition information indicating the user recognized by the user recognition unit.

35. The instruction input device according to claim 21, further comprising

a display unit configured to display the pointer on the display position calculated by the pointer display position calculation unit.

36. The instruction input device according to claim 21,

wherein the offset amount calculation unit is configured to calculate a distance in a third direction between the reference line and the input coordinate as an other offset amount with respect to the gaze position in a second screen predetermined direction along the screen,

the pointer display position calculation unit is configured to calculate, as a display position of the pointer displayed on the screen, a position in which a distance in the second screen predetermined direction between the position and the gaze position is the other offset amount on the screen, and

the third direction is a direction different from the first direction and the second direction, and the second screen predetermined direction is a direction different from the first screen predetermined direction.

37. An instruction input method for calculating a display position of a pointer to be displayed on a screen, based on a pointing operation of movement of a hand of a user, the instruction input method comprising:

detecting a position of the user;

detecting, as an input coordinate, a coordinate indicating a position of the hand of the user;

detecting, based on a body posture of the user, a first direction which is determined with respect to a screen vertical direction and is a direction in which the user is looking;

detecting, based on the body posture of the user, a second direction which is determined with respect to a screen horizontal direction and is a direction in which the user is performing the pointing operation;

calculating a gaze position of the user on the screen, based on the position of the user and the first direction;

calculating a reference line in space between the user and the screen, based on the position of the user and the first position, the reference line being a line corresponding to the gaze position;

calculating a distance in the second direction between the reference line and the input coordinate as an offset amount with respect to the gaze position in a first screen predetermined direction along the screen; and

calculating, as a display position of the pointer to be displayed on the screen, a position in which a distance in the first screen predetermined direction between the position and the gaze position is the offset amount on the screen.

38. A program recorded on a non-transitory recording medium for calculating a display position of a pointer to be displayed on a screen, based on a pointing operation of movement of a hand of a user, the program causing a computer to execute:

detecting a position of the user;

detecting, as an input coordinate, a coordinate indicating a position of the hand of the user;

detecting, based on a body posture of the user, a first direction which is determined with respect to a screen vertical direction and is a direction in which the user is looking;

detecting, based on the body posture of the user, a second direction which is determined with respect to a screen horizontal direction and is a direction in which the user is performing the pointing operation;

calculating a gaze position of the user on the screen, based on the position of the user and the first direction;

calculating a reference line in space between the user and the screen, based on the position of the user and the first position, the reference line being a line corresponding to the gaze position;

calculating a distance in the second direction between the reference line and the input coordinate as an offset amount with respect to the gaze position in a first screen predetermined direction along the screen; and

calculating, as a display position of the pointer to be displayed on the screen, a position in which a distance in the first screen predetermined direction between the position and the gaze position is the offset amount on the screen.

39. A recording medium storing the program according to claim 38.

40. An integrated circuit which calculates a display position of a pointer to be displayed on a screen, based on a pointing operation of movement of a hand of a user, the integrated circuit comprising:

a user position detection unit configured to detect a position of the user;

an input coordinate detection unit configured to detect, as an input coordinate, a coordinate indicating a position of the hand of the user;

a first direction detection unit configured to detect, based on a body posture of the user, a first direction which is determined with respect to a screen vertical direction and is a direction in which the user is looking;

a second direction detection unit configured to detect, based on the body posture of the user, a second direction which is determined with respect to a screen horizontal direction and is a direction in which the user is performing the pointing operation;

a gaze position calculation unit configured to calculate a gaze position of the user on the screen, based on the position of the user and the first direction;

a reference line calculation unit configured to calculate a reference line in space between the user and the screen, based on the position of the user and the first position, the reference line being a line corresponding to the gaze position;

an offset amount calculation unit configured to calculate a distance in the second direction between the reference line and the input coordinate as an offset amount with respect to the gaze position in a first screen predetermined direction along the screen; and

a pointer display position calculation unit configured to calculate, as a display position of the pointer to be displayed on the screen, a position in which a distance of the first screen predetermined direction between the position and the gaze position is the offset amount on the screen.