3D Pointing Method, 3D Display Control Method, 3D Pointing Device, 3D Display Control Device, 3D Pointing Program, and 3D Display Control Program

• US 20080225007A1
• Filed: 10/12/2005
• Published: 09/18/2008
• Est. Priority Date: 10/12/2004
• Status: Active Grant

• Current Assignee: Nippon Telegraph and Telephone Corporation
• Original Assignee: Nippon Telegraph and Telephone Corporation
• Inventors: Nakadaira, Atsushi, Suzuki, Naobumi, Ochi, Daisuke
• Granted Patent: US 7,880,726 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object

Abstract

A three-dimensional pointing method is disclosed. In the three-dimensional pointing method of the present invention, a desired point in a three-dimensional space represented on a display apparatus is pointed at based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane, and, based on pen pressure that is pressure applied to the pen tip of the input pen, time for continuing to point or operation of an operation means provided in the input pen. In addition, in the three-dimensional pointing method of the present invention, a depth direction coordinate of a three-dimensional pointer to be displayed in the three-dimensional space is changed according to the pen pressure of the input pen, the time for continuing to point or the operation of the operation means provided in the input pen, and the three-dimensional pointer is displayed.

Claims

1. :

A three-dimensional pointing method for pointing at a desired point in a three-dimensional space represented on a display apparatus based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane and based on pen pressure that is pressure applied to the pen tip of the input pen, comprising the step of;

changing a depth direction coordinate of a three-dimensional pointer to be displayed in the three-dimensional space according to the pen pressure of the input pen, and displaying the three-dimensional pointer.

5. :

The three-dimensional pointing method as claimed in claim 1, wherein it is determined that an object is pointed at when the object exists within a predetermined distance from three-dimensional coordinates of a point at which the three-dimensional pointer points.

6. :

The three-dimensional pointing method as claimed in claim 1, wherein, when an object displayed in the three-dimensional space is pointed at with the three-dimensional pointer, if operation for selecting or holding the object is performed,

the three-dimensional position of the object is changed according to change of the three-dimensional position of the three-dimensional pointer after the operation for selecting or holding the object is performed, and the object is displayed.

7. :

The three-dimensional pointing method as claimed in claim 1, wherein, when an object displayed in the three-dimensional space is pointed at with the three-dimensional pointer, if operation for starting to operate, edit or process the object is performed,

the object that is pointed at is displayed two-dimensionally on a plane, of the display apparatus, that is closest to an operator, and

the two-dimensionally displayed object accepts the two-dimensional operation, editing, or processing by the input pen.

8. :

The three-dimensional pointing method as claimed in claim 1, wherein the two-dimensional coordinates of the position at which the pen tip of the input pen points on the detection plane is regarded as two-dimensional coordinates of the point at which the three-dimensional pointer points, and the depth direction coordinate of the three-dimensional pointer is changed while keeping the two-dimensional coordinates of the point at which the three-dimensional pointer points to be constant.

15. :

The three-dimensional pointing apparatus as claimed in claim 8, wherein the input information obtaining means further obtains information of a rotation angle of the input pen that is a rotation angle around the axis of the input pen when pointing with the pen tip of the input pen is performed on the detection plane, and

the pointer position/rotation angle calculation means performs the calculation by changing a rotation angle around the axis of the three-dimensional pointer according to the rotation angle of the input pen.

2. :

A three-dimensional pointing method for pointing at a desired point in a three-dimensional space represented on the display apparatus based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane, pen pressure that is pressure applied to the pen tip of the input pen, an inclination angle that is an angle between an axis of the input pen and the detection plane, and an direction angle that is an angle between a projection of the axis of the input pen onto the detection plane and a predetermined line on the detection plane, comprising the steps of;

obtaining an extension of the axis of the input pen in the three-dimensional space based on the inclination angle and the direction angle of the input pen;

displaying a three-dimensional pointer on the extension in the three-dimensional space; and

changing a coordinate of a three-dimensional pointer in the direction of the extension in the three-dimensional space according to the pen pressure of the input pen, and displaying the three-dimensional pointer.

3. :

A three-dimensional pointing method for pointing at a desired point in a three-dimensional space represented on the display apparatus based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane, and based on time for continuing to point or operation of an operation means provided in the input pen, comprising the step of;

changing a depth direction coordinate of a three-dimensional pointer to be displayed in the three-dimensional space according to the time for continuing to point with the pen tip of the input pen or the operation of the operation means of the input pen, and displaying the three-dimensional pointer.

4. :

A three-dimensional pointing method for pointing at a desired point in a three-dimensional space represented on the display apparatus based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane, time for continuing to point or operation of an operation means provided in the input pen, an inclination angle that is an angle between an axis of the input pen and the detection plane, and an direction angle that is an angle between a projection of the axis of the input pen onto the detection plane and a predetermined line on the detection plane, comprising the steps of;

obtaining an extension of the axis of the input pen in the three-dimensional space based on the inclination angle and the direction angle of the input pen;

displaying a three-dimensional pointer on the extension in the three-dimensional space; and

changing a coordinate of a three-dimensional pointer in the direction of the extension in the three-dimensional space according to the time for continuing to point with the pen tip of the input pen or according to the operation of the operation means of the input pen, and displaying the three-dimensional pointer.

9. :

A three-dimensional pointing apparatus for generating a pointer based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane and based on pen pressure that is pressure applied to the pen tip of the input pen, and displaying the generated pointer at a desired point in three-dimensional space represented on a display apparatus to perform pointing, comprising;

input information obtaining means for obtaining information of the two-dimensional coordinates and the pen pressure of the input pen;

pointer position/rotation angle calculation means for calculating a position and an rotation angle of the pointer to be displayed in the three-dimensional space represented on the display apparatus based on the information obtained by the input information obtaining means;

pointer generation means for generate the pointer based on the calculation result of the pointer position/rotation angle calculation means;

pointing determination means for determining whether there is an object that is pointed at by the pointer generated by the pointer generation means in the three-dimensional space represented on the display apparatus;

object generation means for generating the object to be displayed in the three-dimensional space represented on the display apparatus; and

display control means for displaying the pointer generated by the pointer generation means and the object generated by the object generation means in the three-dimensional space represented on the display apparatus,

wherein the pointer position/rotation angle calculation means changes a depth direction coordinate of the three-dimensional pointer to be displayed in the three-dimensional space according to the pen pressure of the input pen in the calculation.

12. :

The three-dimensional pointing apparatus as claimed in claim 9, wherein the object generation means comprising means for changing the three-dimensional position of the object according to change of the three-dimensional position of the three-dimensional pointer to generate the object, and

when an object displayed in the three-dimensional space is pointed at with the three-dimensional pointer, if operation for selecting or holding the object is performed,

the three-dimensional position of the object is changed according to change of the three-dimensional position of the three-dimensional pointer after the operation for selecting or holding the object is performed, and the object is displayed.

13. :

The three-dimensional pointing apparatus as claimed in claim 9, wherein, in addition to the information of the two-dimensional coordinates and the pen pressure, the input information obtaining means obtains an inclination angle that is an angle between an axis of the pen and the detection plane, and an direction angle that is an angle between a projection of the axis of the input pen onto the detection plane and a predetermined line on the detection plane, and

the pointer position/rotation angle calculation means obtains an extension of the axis of the input pen in the three-dimensional space based on the inclination angle and the direction angle of the input pen, sets a position of a three-dimensional pointer to be on the extension in the three-dimensional space, and performs the calculation by changing a coordinate of the three-dimensional pointer in the direction of the extension in the three-dimensional space according to the pen pressure of the input pen.

16. :

The three-dimensional pointing apparatus as claimed in claim 9, the object generation means comprising means for, when an object displayed in the three-dimensional space is pointed at, if operation for starting to operate, edit or process the object is performed, generating an object that is a projection of the pointed object at onto a plane, of the display apparatus, that is closest to an operator.

17. :

The three-dimensional pointing apparatus as claimed in claim 16, wherein, after the object two-dimensionally displayed on the closest plane accepts the two-dimensional operation, editing, or processing with the input pen, when operation for ending the operation, editing, or processing for the object is performed, the object generation means generates an object obtained by restoring the two-dimensionally displayed object to a three-dimensionally display state just before the operation for starting the operation, editing or processing is performed.

18. :

The three-dimensional pointing apparatus as claimed in claim 9, wherein the input pen has a structure in which the length of the pen tip is shortened according to the pen pressure, and

the three-dimensional pointer has a shape similar to the pen tip of the input pen, or a shape similar to a part of the pen tip.

20. :

The three-dimensional pointing apparatus as claimed in claim 9, wherein the pointer position/rotation angle calculation means sets the two-dimensional coordinates of the point at which the three-dimensional pointer points to be the two-dimensional coordinates of the position at which the pen tip of the input pen points on the detection plane, and changes the depth direction coordinate of the three-dimensional pointer while keeping the two-dimensional coordinates of the point at which the three-dimensional pointer points to be constant.

21. :

A three-dimensional pointing program for causing a computer to execute processes in each means of the three-dimensional pointing apparatus as claimed in claim 9.

10. :

A three-dimensional pointing apparatus for generating a pointer based on two-dimensional coordinates of a position that is pointed at by a pen tip of an input pen on a predetermined detection plane and based on time for continuing to point or operation of an operation means provided in the input pen, and displaying the pointer at a desired point in a three-dimensional space represented on a display apparatus to perform pointing, comprising;

input information obtaining means for obtaining information of the two-dimensional coordinates of the input pen, and information of contact/noncontact status of the pen tip of the input pen or information of operation of an operation means of the input pen;

input information process means for calculating the time for continuing to point with the pen tip of the input pen or an amount by which the operation means is operated based on the information obtained by the input information obtaining means;

pointer position/rotation angle calculation means for calculating a position of the pointer to be displayed in the three-dimensional space represented on the display apparatus based on the information obtained by the input information obtaining means;

pointer generation means for generating the pointer based on the calculation result of the pointer position/rotation angle calculation means;

pointing determination means for determining whether there is an object that is pointed at by the pointer generated by the pointer generation means in the three-dimensional space represented on the display apparatus;

object generation means for generating the object to be displayed in the three-dimensional space represented on the display apparatus; and

display control means for displaying the pointer generated by the pointer generation means and the object generated by the object generation means in the three-dimensional space represented on the display apparatus,

wherein the pointer position/rotation angle calculation means performs the calculation by changing a depth direction coordinate of the three-dimensional pointer to be displayed in the three-dimensional space according to the time for continuing to point or according to the operation of the operation means provided in the input pen.

11. :

The three-dimensional pointing apparatus as claimed in claim 10, wherein the pointing determination means determines that an object is pointed at when the object exists within a predetermined distance from three-dimensional coordinates of a point at which the three-dimensional pointer points.

14. :

The three-dimensional pointing apparatus as claimed in claim 10, wherein, the input information obtaining means obtains information of an inclination angle that is an angle between an axis of the pen and the detection plane, and information of a direction angle that is an angle between a projection of the axis of the input pen onto the detection plane and a predetermined line on the detection plane, and

the pointer position/rotation angle calculation means obtains an extension of the axis of the input pen in the three-dimensional space based on the inclination angle and the direction angle of the input pen, sets a position of a three-dimensional pointer to be on the extension in the three-dimensional space, and performs the calculation by changing a coordinate of the three-dimensional pointer in the direction of the extension in the three-dimensional space according to the time for continuing to point or the operation of the operation means provided in the input pen.

19. :

The three-dimensional pointing apparatus as claimed in claim 10, wherein the input pen has a structure in which the length of the pen tip is shortened according to the time for continuing to point with the pen tip or according to the operation of the operation means, and

the three-dimensional pointer has a shape similar to the pen tip of the input pen, or a shape similar to a part of the pen tip.

22. :

A three-dimensional pointing method for moving a pointer, in a three-dimensional space, that is displayed in the three-dimensional space of a display apparatus that can represent the three-dimensional space, and pointing at a desired point in the three-dimensional space, comprising;

a step 1 of moving or rotating the pointer on a two-dimensional plane that is perpendicular to a depth direction of the three-dimensional space of the display apparatus, and moving the pointer in the depth direction;

a step 2 of moving a part for performing pointing in the pointer in the depth direction while keeping, to be constant, a depth direction position of a predetermined point of the pointer other than the part for performing pointing in the pointer, and while keeping a shape and a size of the pointer to be constant, wherein the part for performing pointing in the pointer includes a point for performing pointing and the neighborhood; and

a step 3 for causing the display apparatus to display the pointer moved in the step 1 and the step 2.

23. :

The three-dimensional pointing method as claimed in claim 22, wherein, in step 2, the pointer is rotated around a predetermined center point or center axis, wherein points on the surface or the inside of the pointer are excluded for the predetermined center point or the center axis.

26. :

The three-dimensional pointing method as claimed in claim 23, wherein the center point or the center axis around which the pointer is rotated moves according to a rotation angle when the pointer rotates.

29. :

The three-dimensional pointing method as claimed in claim 22, wherein, when a part of the pointer lies off the three-dimensional space that can be represented by the display apparatus when the pointer moves in the depth direction, the lain-off part is projected onto a two-dimensional plane, of two-dimensional planes that can be represented by the display apparatus, that is close to the lain-off part, or the lain-off part is bent, so as to display the part.

30. :

The three-dimensional pointing method as claimed in claim 22, wherein, in step 3, the display apparatus is caused to display a reference pointer, with the pointer, in which the depth direction position is constant.

24. :

A three-dimensional pointing method for moving a pointer, in a three-dimensional space, that is displayed in the three-dimensional space of a display apparatus that can represent the three-dimensional space, and pointing at a desired point in the three-dimensional space, comprising;

a step 1 of moving or rotating the pointer on a two-dimensional plane that is perpendicular to a depth direction of the three-dimensional space of the display apparatus, and moving the pointer in the depth direction;

a step 4 of moving a part for performing pointing in the pointer in the depth direction while keeping a depth direction position of a predetermined point of the pointer other than the part for performing pointing in the pointer to be constant, and while deforming a shape and a size of the pointer; and

a step 3 for causing the display apparatus to display the pointer moved in the step 1 and the step 4.

25. :

The three-dimensional pointing method as claimed in claim 24, wherein, in step 4, the pointer is rotated around a predetermined center point or center axis, wherein points on the surface or the inside of the pointer are excluded for the predetermined center point or the center axis.

27. :

The three-dimensional pointing method as claimed in claim 24, the pointer comprising;

a first part in which the depth direction position, a position on the two-dimensional plane, the shape and the size are constant;

a second part in which at least the depth direction position changes; and

a third part for connecting the first part with the second part,

wherein, in the step 4, the second part of the pointer is moved in the depth direction.

28. :

The three-dimensional pointing method as claimed in claim 27, wherein, in the step 4, the second part is moved in the depth direction while changing the position of the second part in the two-dimensional plane, or a shape of the second part, or a size of the second part of the three-dimensional pointer.

31. :

A three-dimensional pointing apparatus for displaying a pointer in a three-dimensional space represented on a display apparatus that can represent the three-dimensional space, moving the pointer three-dimensionally based on input information from an input apparatus, and pointing at an arbitrary point in the three-dimensional space, comprising;

input information obtaining means for obtaining input information from the input apparatus;

pointer position/deformation amount calculation means for calculating a display position and a deformation amount of the pointer based on the input information obtained by the input information obtaining means;

pointer generation means for generating a pointer to be displayed at the display position calculated by the pointer position/deformation amount calculation means;

pointing determination means for determining whether there is an object at a point at which the pointer points based on the display position calculated by the pointer position/deformation amount calculation means;

object generation means for changing the object to a state indicating that the object is pointed at when it is determined that there is the object that is pointed at in the pointing determination means; and

display control means for causing the display apparatus to display the pointer generated by the pointer generation means and the object generated by the object generation means.

32. :

The three-dimensional pointing apparatus as claimed in claim 31, the pointer generation means comprising means for moving a part for performing pointing in the pointer in the depth direction while keeping a depth direction position of a predetermined position of the pointer other than the part for performing pointing in the pointer, to be constant, and while keeping a shape and a size of the pointer to be constant, wherein the part for performing pointing includes a point for performing pointing and the neighborhood.

33. :

The three-dimensional pointing apparatus as claimed in claim 31, the pointer generation means comprising means for moving a part for performing pointing in the pointer in the depth direction while keeping a depth direction position of a predetermined position of the pointer other than the part for performing pointing in the pointer to be constant, and while deforming a shape and a size of the pointer.

34. :

The three-dimensional pointing apparatus as claimed in claim 32, wherein the means for moving the part for performing pointing in the pointer in the depth direction rotates the pointer around a predetermined center point or center axis, wherein points on the surface or the inside of the pointer are excluded for the predetermined center point or the center axis.

35. :

The three-dimensional pointing apparatus as claimed in claim 31, the pointer generation means comprising;

means for dividing the pointer into a first part in which the depth direction position, a position on the two-dimensional plane, the shape and the size are constant, and a second part in which at least the depth direction position changes, and for moving only the second part in the depth direction; and

means for connecting the first part with the second part after moving the second part in the depth direction.

36. :

The three-dimensional pointing apparatus as claimed in claim 31, the pointer generation means comprising;

means for determining whether a part of the pointer lies off the three-dimensional space that can be represented by the display apparatus when the pointer moves in the depth direction; and

means for, when there is the lain-off part, projecting the lain-off part onto a two-dimensional plane, of two-dimensional planes that can be represented by the display apparatus, that is close to the lain-off part, or bending the lain-off part.

37. :

A three-dimensional pointing program for causing a computer to execute processes in each means of the three-dimensional pointing apparatus as claimed in claim 31.

38. :

A three-dimensional display control method for controlling display states of a pointer and one or more objects, when displaying the pointer and one or more objects in a three-dimensional space represented on a display apparatus that can represented the three-dimensional space, moving the pointer three-dimensionally based on input information from an input apparatus, and pointing at an arbitrary point in the three-dimensional space, comprising;

a step 1 of calculating a display position of the pointer based on the input information;

a step 2 of displaying the pointer at the display position calculated in the step 1; and

a step 3 of determining whether there is an object in the front side of the depth position of the pointer based on the display position of the pointer calculated in the step 1, and transparentizing the object in the front side of the depth position of the pointer and displaying the object.

39. :

The three-dimensional display control method as claimed in claim 38, wherein, in step 3, only an object, of objects located in the front side of the depth position of the pointer, that overlaps with the pointer is transparentized and displayed.

40. :

The three-dimensional display control method as claimed in claim 38, the step 3 comprising;

transparentizing and displaying an object of the objects located in the front side of the depth position of the pointer, from which objects that are specified or selected based on predetermined input information received from the input apparatus are excluded.

41. :

The three-dimensional display control method as claimed in claim 39, the step 3 comprising;

changing transparency of an object according to depth direction distance between the object located in the front side of the depth position of the pointer and the pointer so as to increase the transparency as the depth direction distance between the object and the pointer becomes larger.

42. :

The three-dimensional display control method as claimed in claim 39, the step 3 comprising;

transparentizing only a region within a predetermined shape having a center point, on the object, that overlaps with a point at which the pointer point, and displaying the object

43. :

The three-dimensional display control method as claimed in claim 42, wherein the predetermined shape to be transparentized changes according to the depth direction distance between the pointer and the object, such that the larger the depth direction distance is, the larger the predetermined shape.

44. :

The three-dimensional display control method as claimed in claim 38, the step 3 comprising;

a step of, when the pointer stands still for a predetermined time, restoring the transparentized object to the opaque state that is a state before being transparentized, and displaying the object.

45. :

A three-dimensional display control apparatus for controlling display states of a pointer and one or more objects, when displaying the pointer and one or more objects in a three-dimensional space represented on a display apparatus that can represented the three-dimensional space, moving the pointer three-dimensionally based on input information from an input apparatus, and pointing an arbitrary point in the three-dimensional space, comprising;

input information obtaining means for obtaining input information from the input apparatus;

pointer position calculation means for calculating a display position of the pointer based on the input information obtained by the input information obtaining means;

pointer generation means for generating a pointer to be displayed at the display position calculated by the pointer position calculation means;

object change determination means for determining whether there is an object in the front side of the depth position of the pointer based on the display position of the pointer calculated by the pointer position calculation means, and determining whether to transparentize the object in the front side of the depth position of the pointer;

object generation/transparentizing means for generating the object to be displayed on the display apparatus and transparentizing the object that is determined to be transparentized by the object change determination means; and

display control means for causing the display apparatus to display the pointer generated by the pointer generation means or the object transparentized by the object generation/transparentizing means.

46. :

The three-dimensional display control apparatus as claimed in claim 45, the object change determination means comprising;

means for determining whether there is an object, of the objects located in the front side of the depth position of the pointer, that is specified or selected based on predetermined input information from the input apparatus, and

wherein the object change determination means causes the object generation/transparentizing means to transparentize objects from which the specified object is excluded.

47. :

The three-dimensional display control apparatus as claimed in claim 45, the object generation/transparentizing means comprising;

means for calculating depth direction distance between the pointer and the object to be transparentized, and

wherein the object generation/transparentizing means changes the transparency of the object to be transparentized according to the depth direction distance.

48. :

The three-dimensional display control apparatus as claimed in claim 45, the object generation/transparentizing means comprising;

means for calculating a point that overlaps with a point, on the object to be transparentized, at which the pointer points, and

wherein the object generation/transparentizing means transparentizes only a region within a predetermined shape having the calculated point as a center.

49. :

The three-dimensional display control apparatus as claimed in claim 45, the object change determination means comprising;

means for determining whether the pointer stands still for a predetermined time, and

wherein, when the pointer stands still for the predetermined time, the object change determination means causes the object generation/transparentizing means to restore the transparentized object to the opaque state before being transparentized.

50. :

A three-dimensional pointing program for causing a computer to execute processes in each means of the three-dimensional display control apparatus as claimed in claim 45.