MOUSE INTERFACE APPARATUS USING CAMERA, SYSTEM AND METHOD USING THE SAME, AND COMPUTER RECORDABLE MEDIUM FOR IMPLEMENTING THE SAME

• US 20080062124A1
• Filed: 12/30/2006
• Published: 03/13/2008
• Est. Priority Date: 09/13/2006
• Status: Abandoned Application

• Current Assignee: AT&T Corporation (AT&T, Inc.), Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Lee, Jae Kwang, Park, Sang Wook, Kim, Hye Mi, Kim, Hongseok, Yu, Jung Jae, Lee, In Ho, Park, Chang Joon, Kim, Jae Hean
• Application Number:
  US11/618,809
• Publication Number:
  US 20080062124A1
• Time in Patent Office: Days
• US Class Current: 345/157
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

Provided is a mouse interface apparatus using a camera, and system and method using the mouse interface apparatus. The mouse interface apparatus detects image coordinates (x_i, y_i) of a plurality of points existing on edges of a camera-photographing computer window image from the camera-photographing computer window image, obtains a homography using the detected image coordinates (x_i, y_i) and plane coordinates (Xi, Yi) of the plurality of preset points on a plane of a real computer window, the homography converting the detected image coordinates (x_i, y_i) into the plane coordinates (Xi, Yi) on the real computer window, converts an arbitrary coordinate on the camera-photographing image into the plane coordinate on the computer window according to a movement of the camera using the obtained homography, and positions a cursor on the plane coordinate to move the position of the cursor according to the movement of the camera.

Claims

1. A mouse interface apparatus using a camera, comprising:

an image coordinate detecting module detecting image coordinates (x_i, y_i) of a plurality of points existing on edges of a computer window from the computer window image obtained through a camera;

a homography processing module obtaining a homography using the detected image coordinates (x_i, y_i) and plane coordinates (Xi, Yi) on a plane of a real computer window; and

a cursor coordinate outputting module outputting a plane coordinate of an arbitrary point on the computer window image using the nomography to determine the output coordinate as a position of a cursor.

2. The mouse interface apparatus of claim 1, further comprising a mouse driver moving the position of the cursor depending on a variation in the plane coordinate determined by the cursor coordinate outputting module.

3. The mouse interface apparatus of claim 1, wherein the plurality of points are comprised of at least four points existing on the edges of the computer window.

4. The mouse interface apparatus of claim 1, wherein the homography is comprised of a 3*3 matrix converting the image coordinates (x_i, y_i) into the plane coordinates (Xi, Yi).

5. The mouse interface apparatus of claim 4, wherein the homography is H obtained by a below equation

$\begin{array}{c}A\\ [\begin{array}{ccccccc}{x}_1& y_1& 1& 0& 0& 0& -x_1\end{array}\end{array}$

X 1 - y 1 

X 1 - X 1 0 0 0 x 1 y 1 1 - x 1 

X 1 - y 1 

X 1 - Y 1 ⋮

x N y N 1 0 0 0 - x N 

X N - y N 

X N - X N 0 0 0 x N y N 1 - x N 

X N - y N 

X N - Y N ] 

h [ h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ] = 0

6. The mouse interface apparatus of claim 1, wherein the homography processing module continues to obtain a homography according to a variation in the image coordinates (x_i, y_i).

7. A mouse interface system comprising:

a camera continuously photographing a computer window to obtain a camera-photographing computer window image including the computer window;

a processor unit obtaining a homography using image coordinates (x_i, y_i) of a plurality of points existing on edges of the camera-photographing computer window image from the obtained image and plane coordinates (Xi, Yi) on a plane of a real computer window corresponding to the plurality of preset points and obtaining and outputting a coordinate on the real computer window corresponding to an arbitrary coordinate on the camera-photographing image using the obtained nomography; and

a monitor positioning a cursor on the coordinate outputted by the processor unit to displaying the cursor thereon.

8. The mouse interface system of claim 7, wherein the homography converts the image coordinates into the plane coordinates on the computer window.

9. The mouse interface system of claim 8, wherein the processor unit continues to obtain the homography if the image coordinates (x_i, y_i) are changed by a movement of the camera.

10. The mouse interface system of claim 9, wherein the processor unit continues to obtain the coordinate on the real computer window corresponding to the arbitrary coordinate on the camera-photographing image according to the continuously obtained homography.

11. The mouse interface system of claim 10, wherein the processor unit further comprises a mouse driver for moving the position of the cursor displayed on the monitor according to a change in the output coordinate.

12. The mouse interface system of any of claims 7 to 11, wherein the plurality of points on the edges of the computer window are at least four.

13. A mouse interface system comprising:

a camera continuously which photographs a computer window to obtain an image including the computer window, obtains a homography using image coordinates (x_i, y_i) of a plurality of points existing on edges of the camera-photographing computer window image from the obtained image and plane coordinates (Xi, Yi) on a plane of a real computer window corresponding to the plurality of preset points, obtains and outputs a coordinate on the real computer window corresponding to an arbitrary coordinate on the camera-photographing image using the obtained nomography.

14. The mouse interface system of claim 13, further comprising a mouse driver for controlling the camera such that the cursor is positioned at a position corresponding to the coordinate outputted from the camera on the computer window monitor.

15. The mouse interface system of claim 14, wherein the camera and the mouse driver wireless transmit and receive a signal on the coordinate outputted from the camera.

16. The mouse interface system of claim 14, wherein the camera and the mouse driver are connected by a USB port.

17. A mouse interface system comprising:

a camera continuously which photographs a computer window to obtain an image including the computer window; and

a computer including a processor unit which obtains a homography using image coordinates (x_i, y_i) of a plurality of points existing on edges of the camera-photographing computer window image from the obtained image and plane coordinates (Xi, Yi) on a plane of a real computer window corresponding to the plurality of preset points, obtains and outputs a coordinate on the real computer window corresponding to an arbitrary coordinate on the camera-photographing image using the obtained homography.

18. The mouse interface system of claim 17, wherein the camera comprises an image processing unit detecting the image coordinates (x_i, y_i) which are positions of the plurality of points existing on the edges of the camera-photographing computer window image from the obtained image on the camera-photographing image.

19. The mouse interface system of claim 17, wherein the computer comprises a mouse driver for controlling the computer such that the cursor is positioned at a position corresponding to the coordinate outputted from the camera on the computer window monitor.

20. The mouse interface system of any of claims 17 to 19, wherein the plurality of points are at least four points on the edges of the computer window.

21. The mouse interface system of claim 20, wherein the camera and the computer wireless transmit and receive a signal outputted from the camera.

22. The mouse interface system of claim 20, wherein the camera and the computer are connected by a USB port.

23. The mouse interface system of claim 21, wherein the homography is continuously varied according to a movement of the camera.

24. A mouse interface method using a camera comprising the steps of:

(a) photographing an image of a computer window through a camera;

(b) detecting image coordinates of a plurality of points existing on edges of the computer window from the photographed image;

(c) obtaining a homography using the detected image coordinates and plane coordinates of a plurality of points on a real computer window;

(d) converting an arbitrary point on the image photographed by the camera into a coordinate of the real computer window using the homography to obtain a position of a cursor on a monitor;

(e) positioning the cursor at a coordinate of the computer.

25. The mouse interface method of claim 24, wherein the plurality of points are at least four.

26. The mouse interface method of claim 25, wherein the homography is comprised of a 3*3 matrix converting the image coordinates (x_i, y_i) into the plane coordinates (Xi, Yi).

27. The mouse interface method of claim 26, wherein the homography is H obtained by a below equation

$\begin{array}{c}A\\ [\begin{array}{ccccccc}{x}_1& y_1& 1& 0& 0& 0& -x_1\end{array}\end{array}$

X 1 - y 1 

X 1 - X 1 0 0 0 x 1 y 1 1 - x 1 

X 1 - y 1 

X 1 - Y 1 ⋮

x N y N 1 0 0 0 - x N 

X N - y N 

X N - X N 0 0 0 x N y N 1 - x N 

X N - y N 

X N - Y N ] 

h [ h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ] = 0

28. The mouse interface method of any of claims 24 to 27, wherein the steps (a)-(e) are continuously repeated according to a variation of the image coordinate according to the movement of the camera.

29. A computer readable recording medium having a program installed to implement functions comprising:

an image coordinate detecting function of detecting image coordinates (x_i, y_i) of a plurality of points existing on edges of a camera-photographing computer window image from the camera-photographing computer window image, the image coordinates (x_i, y_i) corresponding to positions of the plurality of points on the camera-photographing computer window image;

a homography processing function of obtaining a homography using the detected image coordinates (x_i, y_i) and plane coordinates (Xi, Yi) of the plurality of preset points on a plane of a real computer window; and

a cursor coordinate outputting function of obtaining a coordinate of a point on the real computer window corresponding to an arbitrary point on the camera-photographing computer window image to determine a position of a cursor.

30. The computer readable recording medium of claim 29, wherein the program further comprises a function of moving the position of the cursor according to a change in the coordinate determined by the cursor coordinate outputting function.

31. The computer readable recording medium of claim 30, wherein the homography processing function is to implement a function of continuously obtaining the homography according to a change in the image coordinates (x_i, y_i).

32. The computer readable recording medium of claim 31, wherein the homography processing function is to implement a 3*3 matrix converting the image coordinates (x_i, y_i) into the plan9e coordinates (Xi, Yi).

33. The computer readable recording medium of claim 32, wherein the plurality of points are at least four points existing on the edges of the computer window.