Graphical user interface and position or attitude detector

• US 6,727,885 B1
• Filed: 09/06/2000
• Issued: 04/27/2004
• Est. Priority Date: 09/07/1999
• Status: Expired due to Term

• Current Assignee: Nikon Corporation, Nikon Technologies Inc. (Nikon Corporation)
• Original Assignee: Nikon Corporation, Nikon Technologies Inc. (Nikon Corporation)
• Inventors: Kasuya, Atsumi, Ishino, Yukinobu
• Primary Examiner(s): Lao, Lun-Yi
• Application Number:
  US09/656,464
• Time in Patent Office: 1,329 Days
• Field of Search: 345/856-862, 345/156-184, 348/734, 348/180, 348/181, 348/191, 348/744-746, 348/747, 348/749, 702/150-154
• US Class Current: 345/156
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

A graphical user interface is proposed for generating an interface signal with a target on a display aimed. An image sensor has an image plane on which an image of the display including characteristic points is formed, a point of the image formed at a predetermined position of the image plane corresponding to the target. A processor calculates the position of the target on the display relative to the characteristic points on the basis of the output of the image sensor including the information of the positions of the characteristic points. And the interface signal is generated with the position of the target calculated. More specifically, the processor includes an image processor for identifying the positions of the characteristic points on the image plane, a first processor for calculating a first data on the basis of the positions of the characteristic points on the image plane, and a second processor for calculating a second data on the basis of the first data and the positions of the characteristic points on the image plane. This is not only useful in the graphical user interface, but also in detecting the position of a target point or the attitude of a three-dimensional object in general.

Claims

1. A graphical user interface for generating an interface signal with an aimed-at target on a display comprising:

an image sensor having an image plane on which an image including characteristic points is formed, a predetermined position of the image plane corresponding to a target point;

a processor for calculating the position of the target on the display using parameters of an attitude of the display relative to the image plane on the basis of four characteristic points; and

a signal generator for generating an interface signal with the position of the target calculated.

2. The graphical user interface according to claim 1, further comprising an aiming device for helping the image sensor to form the image of the display with the target in coincidence with the predetermined position.

3. The graphical user interface according to claim 1, further comprising an aiming device including a light beam emitter for emitting a light beam along the optical axis of the image sensor toward the display, whereby the image of the target is in coincidence with the predetermined position on the image plane when the light beam hits the target on the display.

4. The graphical user interface according to claim 1, further comprising an aiming device including a viewfinder for aiming the target.

5. The graphical user interface according to claim 4, wherein the viewfinder includes an optical finder for optically aiming the target.

6. The graphical user interface according to claim 5, wherein the viewfinder includes a monitor of the output of the image sensor.

7. The graphical user interface according to claim 1, further comprising a controller for ordering the image sensor to take the image of the display.

8. The graphical user interface according to claim 7, wherein the controller is designed to order the image sensor to take one image in response to a manual operation.

9. The graphical user interface according to claim 7, wherein the controller is designed to order the image sensor to take a series of images in succession.

10. The graphical user interface according to claim 1, wherein the processor includes an image processor for identifying the positions of the characteristic points on the image plane, a first processor for calculating a first data on the basis of the positions of the characteristic points on the image plane, and a second processor for calculating a second data on the basis of the first data and the positions of the characteristic points on the image plane, the position of the target on the display being detected on the basis of the second data.

11. A position detector for detecting a position of a target point on a given plane having a plurality of characteristic points comprising:

an image sensor having an image plane on which an image of the given plane is formed with the characteristic points included in the image, a point of the image which is formed at a predetermined position of the image plane corresponding to the target point to be detected on the given plane;

an image processor for identifying the positions of the characteristic points on the image plane;

a first processor for calculating a first data on the basis of the positions of the characteristic points on the image plane; and

a second processor for calculating a second data on the basis of the first data and the positions of the characteristic points on the image plane, the position of the target point on a given plane being detected on the basis of the second data.

12. The position detector according to claim 11, wherein the first data includes information of attitude of the given plane relative to the image plane.

13. The position detector according to claim 11, wherein the first processor calculates the first data in accordance with the three-dimensional perspective projection.

14. The position detector according to claim 13, wherein the first processor includes a calculator for calculating a pair of vanishing points on the basis of the positions of the characteristic points on the image plane.

15. The position detector according to claim 14, wherein the first processor rotates the positions of the characteristic points around an axis perpendicular to the image plane so that one of the vanishing points comes on one of the coordinate axes on the image plane.

16. The position detector according to claim 15, wherein the second processor includes a first line calculator for calculating a first line between the predetermined position and one of the vanishing point, a second line calculator for calculating a second line between a pair of characteristic points and a intersection calculator for calculating an intersection between the first and second lines.

17. The position detector according to claim 11, wherein the first processor includes a calculator for calculating a pair of vanishing points on the basis of the positions of the characteristic points on the image plane, and wherein the second processor includes a first line calculator for calculating a first line between the predetermined position and one of the vanishing point, a second line calculator for calculating a second line between the position of a pair of characteristic points and a intersection calculator for calculating an intersection between the first and second lines.

18. The position detector according to claim 11, further comprising an aiming device for helping the image sensor to form the image of the given plane with the target point in coincidence with the predetermined position.

19. The position detector according to claim 18, wherein the aiming device includes a light beam emitter for emitting a light beam along the optical axis of the image sensor toward the given plane, whereby the image of the target point is in coincidence with the predetermined position on the image plane when the light beam hits the target point on the given plane.

20. The position detector according to claim 18, wherein the aiming device includes a viewfinder for aiming the target point.

21. An attitude detector for detecting an attitude of a given plane having a plurality of characteristic points comprising:

an image sensor having an image plane on which an image is formed with four characteristic points included in the image, a predetermined position being the origin of the coordinates of the image plane;

a helping device for aiming the given plane;

an image processor for identifying the positions of four characteristic points on the image plane;

a first processor for calculating a pair of vanishing points on the basis of the positions of four characteristic points on the image plane; and

a second processor for calculating intersection points on the basis of the vanishing points and the positions of the four characteristic points on the image plane, the attitude of the given plane being detected on the basis of the intersection points.

22. The attitude detector according to claim 21, wherein the first processor rotates the positions of the characteristic points around an axis perpendicular to the image plane so that one of the vanishing points comes on one of the coordinate axes on the image plane.

23. The attitude detector according to claim 21, wherein the second processor includes a first line calculator for calculating a first line between the predetermined point and one of the vanishing points, a second line calculator for calculating a second line between a pair of characteristic points and a intersection calculator for calculating an intersection between the first and second lines.

24. The attitude detector according to claim 21, wherein the first processor includes a calculator for calculating a pair of vanishing points on the basis of the positions of the characteristic points on the image plane, and wherein the second processor includes a first line calculator for calculating a first line between the predetermined point and one of the vanishing points, a second line calculator for calculating a second line between a pair of characteristic points and a intersection calculator for calculating an intersection between the first and second lines.

25. The attitude detector according to claim 21, wherein the helping device includes a light beam emitter for emitting a light beam along the optical axis of the image sensor toward the given plane.

26. The attitude detector according to claim 21, wherein the helping device includes a viewfinder with a cross being optically equivalent to the predetermined position.

27. A graphical user interface for generating an interface signal with an aimed-at target on a display comprising:

an image sensor having an image plane on which an image including characteristic points is formed, a predetermined position of the image plane corresponding to the target;

an aiming device for helping the image sensor to form the image of the display with the target in coincidence with the predetermined position;

a transmitter for transmitting the output of the image sensor to an outside processor for a calculation of the position of the target on the display relative to four characteristic points; and

a signal generator for generating an interface signal with the position of the target calculated.

28. The graphical user interface according to claim 27, wherein the aiming device includes a light beam emitter for emitting a light beam along the optical axis of the image sensor toward the display, whereby the image of the target is in coincidence with the predetermined position on the image plane when the light beam hits the target on the display.

29. The graphical user interface according to claim 27, wherein the aiming device includes a viewfinder for aiming the target.

30. The graphical user interface according to claim 27, the graphical user interface is for use in a computer.

31. The graphical user interface according to claim 27, the graphical user interface is for use in a shooting game.