Easily deployable interactive direct-pointing system and presentation control system and calibration method therefor

• US 8,866,742 B2
• Filed: 02/07/2014
• Issued: 10/21/2014
• Est. Priority Date: 05/28/2004
• Status: Active Grant

• Current Assignee: UltimatePointer, LLC
• Original Assignee: UltimatePointer, LLC
• Inventors: Banning, Erik Jan
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US14/175,960
• Publication Number:
  US 20140152565A1
• Time in Patent Office: 256 Days
• Field of Search: 345156-169, 345173-178, 345204-215, 178 1801- 1809
• US Class Current: 345/158
• CPC Class Codes: G03B 21/00 Projectors or projection-type viewers; Accessories therefor devices for changing pictures G03B23/00
• CPC Class Codes: G03B 21/006 using LCD's
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03549 Trackballs G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A method for controlling movement of a computer display cursor based on a point-of-aim of a pointing device within an interaction region includes projecting an image of a computer display to create the interaction region. At least one calibration point having a predetermined relation to said interaction region is established. A pointing line is directed to substantially pass through the calibration point while measuring a position of and an orientation of the pointing device. The pointing line has a predetermined relationship to said pointing device. Movement of the cursor is controlled within the interaction region using measurements of the position of and the orientation of the pointing device.

Claims

1. A system for use in controlling two positional coordinates of a feature on a computer-generated image, the system comprising:

a first enclosure including;

a processing unit;

computer program storage coupled to said processing unit;

an image generation element coupled to said processing unit and for coupling to an image display device and for providing the computer-generated image;

a first power source for providing power to devices located in said first enclosure; and

a first enclosure communication element coupled to said processing unit;

a second enclosure separate from said first enclosure, said second enclosure including;

a level sensing device providing a level sensor output signal;

at least one of a connection to an external second power source and an internal third power source for providing power to devices located in said second enclosure; and

a second enclosure communication element coupled to said level sensing device and for coupling to said first enclosure communication element, said second enclosure communication element for providing first data dependent on said level sensor output signal; and

a third enclosure designed to be handheld and operated by a user in mid-air, said third enclosure separate from said first enclosure and said second enclosure, said third enclosure including;

a light source configured for use in collecting data dependent on position of said third enclosure;

a user input device configured to allow a user to provide directional inputs and for providing a user output signal;

at least one of a connection to an external fourth power source and an internal fifth power source for providing power to devices located in said third enclosure; and

a third enclosure communication element coupled to said user input device and for coupling to said first enclosure communication element, said third enclosure communication element for providing second data dependent on said user output signal.

2. The system according to claim 1, wherein said third enclosure communication element includes a unique identification value.

3. The system of claim 1, wherein said third enclosure further includes:

a sensing device coupled to said third enclosure communication element for determining the orientation of said third enclosure, said sensing device providing an orientation output signal,

wherein said third enclosure communication element is further for providing second data dependent on said orientation output signal.

4. The system according to claim 1, wherein said level sensing device includes an accelerometer.

5. The system according to claim 1, wherein the position is a three dimensional position.

6. The system according to claim 1, wherein the position is a two dimensional position.

7. The system according to claim 1, wherein the position is a one dimensional position.

8. A non-transitory computer-readable medium storing computer-executable instructions for directing a processing unit in a computer to perform a method for controlling positional coordinates of a feature on a rectangular computer screen interaction region spanned by a first and a second computer axis, the computer communicating with an enclosure designed to be handheld by a user and operated by the user in mid-air, the enclosure defining a pointing line with a predetermined relation to the enclosure, the pointing line defining a point of aim as the intersection of the pointing line with a rectangular interaction structure, the interaction structure being spanned by a horizontal first structure axis and a vertical second structure axis, the enclosure for providing first data that allows for the complete definition of the orientation of said pointing line in three dimensions, the enclosure comprising an accelerometer for providing a first output signal and a gyro for providing a second output signal, the first data being dependent on the first and second output signals, the method comprising:

receiving the first data from the enclosure; and

using the first data to control the positional coordinates of the feature,

wherein said step of using the first data to control the feature does not depend on the angles between the computer axes and the structure axes and operates so that;

first data that is generated while the position of the enclosure is kept substantially fixed while the points of aim substantially lie on a line that is parallel to the first structure axis results in positional coordinates for the feature that substantially lie on a line that is parallel to the first computer axis of the computer screen interaction region, and

first data that is generated while the position of the enclosure is kept substantially fixed while the points of aim substantially lie on a line that is parallel to the second structure axis results in positional coordinates for the feature that substantially lie on a line that is parallel to the second computer axis of the computer screen interaction region,

whereby the positional coordinates of the feature can be controlled by the user by movement of the enclosure.

9. The non-transitory computer-readable medium according to claim 8, wherein the enclosure further comprises an image capturing device for providing a third output signal and

wherein the first data is furthermore dependent on the third output signal.

10. The non-transitory computer-readable medium according to claim 9, wherein the enclosure further comprises a magnetometer for providing a fourth output signal and

wherein the first data is furthermore dependent on the fourth output signal.

11. The non-transitory computer-readable medium according to claim 8, wherein the orientation of the pointing line in three dimensions is completely described by a first and a second angle, and

wherein the first output signal is used for determining both the first and the second angle.

12. The non-transitory computer-readable medium according to claim 11, wherein the enclosure further comprises a magnetometer for providing a third output signal and

wherein the first data is furthermore dependent on the third output signal.

13. The non-transitory computer-readable medium according to claim 8, wherein if a first calibration point with a predetermined relation to the interaction structure is identified to the user and the pointing line is directed at the first calibration point, the resulting first data is first calibration data, and

wherein said step of using the first data to control the feature further operates so that the first calibration data is used to control the feature.

14. The non-transitory computer-readable medium according to claim 13, wherein if a second calibration point with a predetermined relation to the interaction structure is identified to the user, the second calibration point being different from the first calibration point, and the pointing line is directed at the second calibration point, the resulting first data is second calibration data, and

wherein said step of using the first data to control the feature further operates so that the second calibration data is used to control the feature.

15. A pointing device for controlling a feature on an image generated by a computer, comprising:

a handheld enclosure having a predetermined relation to a pointing line;

a sensing device for generating data, with first data of said sensing device dependent on a first orientation of said enclosure at a first position while the pointing line is directed at a first calibration point, said first calibration point having a predetermined relation to the image, and with second data of said sensing device dependent on a second orientation of said enclosure at substantially said first position while the pointing line is directed at a non-calibration point on the image, said sensing device including an image capturing device; and

a communication element coupled to said sensing device for providing third data to the computer for controlling the feature on the image, said third data being based on said first data or said second data,

wherein the feature on the image can be controlled based on the relation between the first orientation and the second orientation of said enclosure.

16. The pointing device according to claim 15, wherein fourth data of said sensing device is dependent on a third orientation of said enclosure at substantially said first position while the pointing line is directed at a second calibration point, said second calibration point having a predetermined relation to the computer image,

wherein said third data is further based on said first data, said second data or said fourth data, and

wherein the feature on the image can be further controlled based on the relation between the third orientation and the second orientation and the first orientation of said enclosure.

17. The pointing device according to claim 16, wherein said sensing device further includes at least one of an accelerometer, a gravity sensor, a magnetometer, a gyroscope and an inclinometer.

18. The pointing device according to claim 15, wherein said sensing device further includes an accelerometer.

19. The pointing device according to claim 15, wherein said sensing device further includes a gravity sensor.

20. The pointing device according to claim 15, wherein said sensing device further includes a magnetometer.

21. The pointing device according to claim 15, wherein said sensing device further includes a gyroscope.

22. The pointing device according to claim 15, wherein said sensing device further includes an inclinometer.

23. The pointing device according to claim 15, wherein said sensing device further includes a gyroscope and an accelerometer.

24. A device for providing first data for use in controlling positional coordinates of a feature on a computer-generated image, the device comprising:

an enclosure designed to be handheld by a user and operated by the user in mid-air;

an accelerometer located in said enclosure for providing a first output signal;

a gyroscope located in said enclosure for providing a second output signal;

a user input device located in said enclosure for providing a user output signal, said user input device allowing a user to provide directional inputs;

a light source located in said enclosure and configured for use in collecting data dependent on distance;

at least one of a connection to an external first power source and an internal second power source for providing power to devices located in said enclosure; and

a communication element located in said enclosure and coupled to said accelerometer, said gyroscope, and said user input device, said communication element including a unique identification value, for providing first data dependent on said first output signal and said second output signal and for providing second data dependent on said user output signal.

25. The device according to claim 24, further comprising:

a magnetometer located in said enclosure and coupled to said communication element for providing a third output signal,

wherein first data provided by said communication element is further dependent on said third output signal.

26. The device according to claim 24, wherein said enclosure is further designed to be directed at a point,

wherein said first and second output signals include first calibration data if said device is directed at a first calibration point, said first calibration point having a predetermined relation to the computer-generated image, and

wherein said first data which includes said first calibration data can be used to control the feature on the computer-generated image.

27. The device according to claim 26,

wherein said first and second output signals include second calibration data if said device is directed at a second calibration point, said second calibration point having a predetermined relation to the computer-generated image and different from said first calibration point, and

wherein said first data which includes said second calibration data can be used to control the feature on the computer-generated image.

28. The device according to claim 24, wherein said first and second output signals include first calibration data if said device is placed in a first pre-defined position, and

wherein said first data which includes said first calibration data can be used to control the feature on the computer-generated image.

29. The device according to claim 28, wherein said first and second output signals include second calibration data if said device is placed in a second pre-defined position, the second pre-defined position being different than the first pre-defined position, and

wherein said first data which includes said second calibration data can be used to control the feature on the computer-generated image.

30. The device according to claim 24, the device further comprising:

a microphone located in said enclosure and coupled to said communication element for providing an audio output signal, said communication element providing third data dependent on said audio output signal.

31. The device according to claim 24, wherein said user input device includes a trackball.

32. A device for providing first data for use in controlling a feature on a computer-generated image, the device comprising:

an enclosure designed to be handheld by a user and operated by the user in mid-air;

an accelerometer located in said enclosure for providing a first output signal;

a gyroscope located in said enclosure for providing a second output signal;

a magnetometer located in said enclosure for providing a third output signal;

a user input device located in said enclosure for providing a user output signal, said user input device allowing a user to provide directional inputs;

at least one of a connection to an external first power source and an internal second power source for providing power to devices located in said enclosure; and

a communication element located in said enclosure and coupled to said accelerometer, said gyroscope, said magnetometer, and said user input device, for providing first data dependent on said first output signal and said second output signal and said third output signal and for providing second data dependent on said user output signal.

33. The device according to claim 32, wherein said controlling a feature comprises controlling positional coordinates of the feature on said computer generated image.

34. The device according to claim 32, the device further comprising:

a microphone located in said enclosure and coupled to said communication element for providing an audio output signal, and

an image capturing device in said enclosure and coupled to said communication element for providing a video output signal, and

wherein said communication element is for providing third data dependent on said audio output signal and said video output signal.

35. The device according to claim 32, wherein said user input device includes a trackball.