Determining positioning and/or relative movement of graphical-user interface element based on display images

• US 7,420,540 B2
• Filed: 12/01/2003
• Issued: 09/02/2008
• Est. Priority Date: 12/01/2003
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Olbrich, Craig A.
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Said; Mansour M
• Application Number:
  US10/726,078
• Publication Number:
  US 20050116931A1
• Time in Patent Office: 1,737 Days
• Field of Search: 345156-160, 345162-166, 345169-170, 345173-175, 345179-180, 345/184, 345/1.2, 345/2.1, 345/2.2, 345/2.3, 178/18.01
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface

Abstract

A system for a computing device of one embodiment of the invention is disclosed that includes an image-capturing mechanism and a controller. The image-capturing mechanism is capable of capturing images. Each image includes at least one corner of a display communicatively coupled to the computing device. The controller is to determine at least one of positioning and relative movement for a graphical-user interface element displayed on the display, based on the images captured by the image-capturing mechanism.

Claims

1. A system for a computing device comprising:

an image-capturing mechanism capable of capturing images, each image including at least one corner of a display communicatively coupled to the computing device, the images including a first image including the at least one corner of the display and a second image including the at least one corner of the display; and

,

a controller to determine at least one of positioning and relative movement for a graphical-user interface element displayed on the display, based on the images captured by the image-capturing mechanism, by determining;

a first location of the at least one corner of the display in the first image, a second location of the at least one corner of the display in the second image, and an amount and a direction of movement of the at least one corner of the display from the first location in the first image to the second location in the second image.

2. The system of claim 1, wherein at least one of the images includes the display completely.

3. The system of claim 1, wherein at least one of the images includes a center of the display.

4. The system of claim 1, wherein the controller is to determine positioning for the graphical-user interface element based on a position of the at least one corner of the display within an image captured by the image-capturing mechanism relative to a field of view of the image.

5. The system of claim 1, wherein the display is a brightest object within each image.

6. The system of claim 1, wherein the computing device is at least one of:

a desktop computer, a laptop computer, a handheld computing device, a personal digital assistant (PDA) device, an audio-visual device, and a visual-only device.

7. The system of claim 1, wherein the image-capturing mechanism is at least one of a camera device and a photosensitive device.

8. The system of claim 1, wherein the display comprises at least one of:

a cathode-ray tube (CRT) device;

a liquid-crystal display (LCD) device;

a flat-panel display (FPD) device;

a plasma display device; and

, an object on which images are projected by a projection display device.

9. The system of claim 8, wherein the object on which images are projected by the projection display device is a screen.

10. The system of claim 1, wherein the display is part of the computing device.

11. The system of claim 1, wherein the graphical-user interface element comprises a graphical-user interface pointer.

12. The system of claim 1, wherein the controller is part of the computing device.

13. The system of claim 12, wherein the controller comprises software running on the computing device.

14. The system of claim 12, wherein the controller comprises hardware.

22. The pointing device of claim 1, further comprising one or more buttons disposed within the housing.

23. The pointing device of claim 22, wherein the one or more buttons comprises an activation button that is actuated to cause the image-capturing mechanism to capture the images.

24. The pointing device of claim 22, wherein the one or more buttons comprises one or more action buttons that are actuated to cause actions relative to graphical-user interface elements displayed on the display.

25. The pointing device of claim 23, wherein the image-capturing mechanism is to capture the images while the activation button remains actuated until the activation button is released.

26. The pointing device of claim 23, wherein the image-capturing mechanism is to capture the images upon the activation button being actuated and released, and is to stop capturing the images upon the activation button being actuated and released again.

15. A pointing device for a computing device comprising:

an image-capturing mechanism capable of capturing images, each image including at least one corner of a display communicatively coupled to the computing device, the images including a first image including the at least one corner of the display and a second image including the at least one corner of the display; and

,

a communications mechanism to communicatively couple the pointing device to the computing device,

wherein at least one of positioning and relative movement for a graphical-user interface element displayed on the display is determined based on the images captured by the image-capturing mechanism, by determining;

a first location of the at least one corner of the display in the first image, a second location of the at least one corner of the display in the second image, and an amount and a direction of movement of the at least one corner of the display from the first location in the first image to the second location in the second image.

16. The pointing device of claim 15, wherein at least one of the images includes the display completely, and at least one of the images includes a center of the display.

17. The pointing device of claim 15, wherein positioning for the graphical-user interface element is determined based on a position of the at least one corner of the display within an image captured by the image-capturing mechanism relative to a field of view of the image.

18. The pointing device of claim 15, wherein the display is a brightest object within each image.

19. The pointing device of claim 15, wherein the communications mechanism is to communicate the images captured by the image-capturing mechanism to the computing device, wherein the computing device is to determine at least one of positioning and relative movement for the graphical-user interface element displayed on the display based on the images captured by the image-capturing mechanism.

20. The pointing device of claim 15, further comprising a controller to determine at least one of positioning and relative movement for the graphical-user interface element displayed on the display based on the images captured by the image-capturing mechanism,

wherein the communications mechanism is to communicate information regarding at least one of positioning and relative movement for the graphical-user interface element as determined by the controller to the computing device.

21. The pointing device of claim 15, further comprising a housing within which the image-capturing mechanism and the communications mechanism are at least partially disposed.

27. A pointing device for a computing device comprising:

means for capturing images of at least one corner of a display communicatively coupled to the computing device, the images including a first image including the at least one corner of the display and a second image including the at least one corner of the display; and

,

means for determining relative movement for a graphical-user interface element displayed on the display based on a change in position of the at least one corner of the display between the first image and the second image captured by the image-capturing mechanism, by determining;

a first location of the at least one corner of the display in the first image, a second location of the at least one corner of the display in the second image, and an amount and a direction of movement of the at least one corner of the display from the first location in the first image to the second location in the second image.

28. A pointing device for a computing device comprising:

means for capturing images of at least one corner of a display communicatively coupled to the computing device, the images including a first image including the at least one corner of the display and a second image including the at least one corner of the display; and

,

means for determining positioning for a graphical-user interface element displayed on the display based on a position of the at least one corner of the display within an image captured by the image-capturing mechanism relative to a field of view of the image, by determining;

a first location of the at least one corner of the display in the first image, a second location of the at least one corner of the display in the second image, and an amount and a direction of movement of the at least one corner of the display from the first location in the first image to the second location in the second image.

29. A method comprising:

capturing a first image of at least one corner of a display communicatively coupled to a computing device;

capturing a second image of the at least one corner of the display;

determining an amount and a direction of movement of the at least one corner of the display between the first image and the second image; and

,

causing a graphical-user interface element displayed on the display by the computing device to move based on the amount and the direction of the movement of the at least one corner of the display,

wherein determining the amount and the direction of movement of the at least one corner of the display between the first image and the second image comprises;

determining a first location of the at least one corner of the display in the first image;

determining a second location of the at least one corner of the display in the second image; and

,

determining the amount and the direction of movement of the at least one corner of the display from the first location in the first image to the second location in the second image.

30. The method of claim 29, further comprising a user moving a pointing device having an image-capturing mechanism and communicatively coupled to the computing device between the image-capturing mechanism capturing the first image and the second image in a direction and by an amount at which the user desires to have the graphical-user interface element moved.

31. The method of claim 29, wherein causing the graphical-user interface element displayed on the display by the computing device to move based on the amount and the direction of the movement of the at least one corner of the display comprises causing the graphical-user interface element to move in a direction opposite to the direction of the movement of the at least one corner of the display by an amount relative to a size of the display proportional to the amount of the movement of the at least one corner of the display relative to a size of the image.

32. A method comprising:

capturing an image of at least a center of a display communicatively coupled to a computing device;

determining a location of at least the center of the display within the image;

determining an offset amount and an offset direction between the location of at least the center of the display within the image and a center of the image; and

,

causing a graphical-user interface element to be displayed on the display by the computing device at a position based on the offset amount and the offset direction between the location of at least the center of the display within the image and the center of the image.

33. The method of claim 32, further comprising a user aiming a pointing device having an image-capturing mechanism and communicatively coupled to the computing device at a position at which the user desires to have the graphical-user interface element displayed.

34. The method of claim 32, wherein causing the graphical user-interface element to be displayed on the display by the computing device at a position based on the offset amount and the offset direction between the location of at least the center of the display within the image and the center of the image comprises causing the graphical user-interface to be displayed on the display at a position offset from a center of the display relative to a size of the display proportional to the offset amount relative to a size of the image and in a direction opposite of the offset direction.

35. A method comprising:

providing an image-capturing mechanism of a pointing device capable of capturing images, each image including at least one corner of a display communicatively coupled to a computing device, the images including a first image including the at least one corner of the display and a second image including the at least one corner of the display; and

,

providing a controller of the pointing device capable of determining at least one of positioning and relative movement for a graphical-user interface element displayed on the display, based on the images captured by the image-capturing mechanism, by determining;

a first location of the at least one corner of the display in the first image, a second location of the at least one corner of the display in the second image, and an amount and a direction of movement of the at least one corner of the display from the first location in the first image to the second location in the second image.

36. The method of claim 35, further comprising providing a communications mechanism of the pointing device capable of communicating at least one of positioning and relative movement for the graphical-user interface element from the controller to the computing device.

37. The method of claim 35, further comprising providing an activation button of the pointing device that is capable of being actuated to cause the image-capturing mechanism to capture the images.

38. The method of claim 35, further comprising providing one or more action buttons that are capable of being actuated to cause actions relative to graphical-user interface elements displayed on the display.

39. The method of claim 35, further comprising providing a housing within which the image-capturing mechanism and the controller are at least partially disposed.