Easily deployable interactive direct-pointing system and presentation control system and calibration method therefor

• US 9,411,437 B2
• Filed: 05/14/2015
• Issued: 08/09/2016
• Est. Priority Date: 05/28/2004
• Status: Active Grant

• Current Assignee: UltimatePointer, LLC
• Original Assignee: UltimatePointer, LLC
• Inventors: Banning, Erik Jan
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US14/712,788
• Publication Number:
  US 20160054814A1
• Time in Patent Office: 453 Days
• Field of Search: 345156-169, 345173-178, 345204-215, 178 1801- 1809
• US Class Current: 1/1
• CPC Class Codes: G03B 21/00 Projectors or projection-type viewers; Accessories therefor devices for changing pictures G03B23/00
• CPC Class Codes: G03B 21/006 using LCD's
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/0338 with detection of limited linear or angular displacement of an operating part of the device from a neutral position, e.g. isotonic or isometric joysticks
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03549 Trackballs G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A method for controlling movement of a computer display cursor based on a point-of-aim of a pointing device within an interaction region includes projecting an image of a computer display to create the interaction region. At least one calibration point having a predetermined relation to said interaction region is established. A pointing line is directed to substantially pass through the calibration point while measuring a position of and an orientation of the pointing device. The pointing line has a predetermined relationship to said pointing device. Movement of the cursor is controlled within the interaction region using measurements of the position of and the orientation of the pointing device.

Claims

1. An apparatus for controlling contents of a computer generated image, comprising:

a first device that is designed to be handheld and wielded by a user in mid-air and comprises;

a first orientation sensor for providing a first orientation signal that is dependent on at least one orientation coordinate of the first orientation sensor;

at least one of a first power source and a connection to a second power source for supplying power to said first device; and

a first communication-and-control device coupled to said first orientation sensor and for providing first data that is dependent on said first orientation signal;

a second device that is designed to be head-mounted and comprises;

a second orientation sensor for providing a second orientation signal that is dependent on at least one orientation coordinate of the second orientation sensor;

at least on of a third power source and a connection to a fourth power source for supplying power to said second device; and

a second communication-and-control device coupled to said second orientation sensor and for providing second data that is dependent on said second orientation signal;

a display device for coupling to the computer and for displaying the computer generated image; and

a third communication-and-control device for coupling to the computer and for receiving the first and second data and for providing third data to the computer,

wherein said third data is used by the computer to control said contents.

2. The apparatus according to claim 1 wherein

the first orientation sensor comprises a first accelerometer and a first gyro and a first magnetometer,

the first orientation signal depends on at least two orientation coordinates of the first orientation sensor,

the second orientation sensor comprises a second accelerometer and a second gyro and a second magnetometer, and

the second orientation signal depends on at least two orientation coordinates of the second orientation sensor.

3. The apparatus according to claim 2 wherein the display device has a fixed position relative to the second device.

4. The apparatus according to claim 1 wherein said one orientation coordinate of the first orientation sensor is relative to a coordinate system that has at least one coordinate axis with fixed orientation relative to the second device.

5. The apparatus according to claim 1 wherein the second device is a base station that has a fixed position and a fixed orientation relative to the display device.

6. The apparatus according to claim 5 wherein the first device further comprises:

a first position sensor coupled to the first communication-and-control device and for providing a first position signal that is dependent on at least one position coordinate of the first position sensor relative to the base station, and

wherein the first data is dependent on the first position signal.

7. The apparatus according to claim 6 wherein the first orientation signal depends on at least two orientation coordinates of the first orientation sensor relative to the base station, and

wherein the first position signal depends on at least two position coordinates of the first position sensor relative to the base station.

8. A non-transitory computer-readable medium or media storing computer-executable instructions for causing a computer to perform a method for controlling a feature on a rectangular computer screen interaction region in conjunction with a first sensing device that has a fixed relation to a pointing line, the computer screen interaction region lying on a rectangular screen and having four edges parallel to edges of said screen,

the computer comprising:

a first communication-and-control device; and

a processing unit coupled to the first communication-and-control device and to said medium or media,

the first sensing device comprising;

an accelerometer for generating a first output signal;

a gyro for generating a second output signal; and

a second communication-and-control device coupled to said accelerometer and said gyro and configured to allow coupling to the first communication-and-control device and for providing first data to said first communication-and-control device,

wherein said first data is dependent on at least one of said first output signal and said second output signal and is sensitive to orientation of said pointing line when the position of the first sensing device is constant, said orientation being relative to a coordinate system defined by three perpendicular coordinate axes of which a first and a second coordinate axis are horizontal and a third coordinate axis is vertical,

the method comprising the steps of;

developing second data that is dependent on the first data;

using said second data to control the feature on the computer screen interaction region,

wherein the step of controlling the feature is independent of orientation of the computer screen interaction region relative to said coordinate system and for fixed positions of said first sensing device and positions of said feature inside the computer screen interaction region operates so that the feature is kept at substantially the same distance from a first edge of the computer screen interaction region if said second data is indicative of the pointing line having substantially remained in a plane parallel to said third coordinate axis.

9. The non-transitory computer-readable medium or media according to claim 8 wherein said first data depends on said first output signal and said second output signal and is indicative of an angle between said pointing line and a line that is vertical.

10. The non-transitory computer-readable medium or media according to claim 9 wherein said pointing line defines a point-of-aim as the intersection of the pointing line with an interaction structure that is a non-flat surface that is perpendicular to the first and second coordinate axes, and

wherein for fixed positions of the first sensing device and positions of said feature inside said computer screen interaction region the step of controlling the feature operates so that the feature is kept at the same distance from the first edge of the computer screen interaction region if said second data indicates that the motion of the point-of-aim has no substantial horizontal component and so that the feature is kept at the same distance from a second edge of the computer screen interaction region if said second data indicates that the motion of the point-of-aim has no substantial vertical component, said second edge being perpendicular to said first edge.

11. The non-transitory computer-readable medium or media according to claim 8,

wherein said pointing line defines a point-of-aim as the intersection of the pointing line with an interaction structure that is perpendicular to the first and second coordinate axes,

wherein said first data depends on said first output signal and said second output signal, and

wherein for fixed positions of the first sensing device and positions of said feature inside said computer screen interaction region the step of controlling the feature operates so that the feature is kept at the same distance from the first edge of the computer screen interaction region if said second data indicates that the point-of-aim has moved only vertically.

12. The non-transitory computer-readable medium or media according to claim 11 where said interaction structure is a rectangle with an edge parallel to said third coordinate axis.

13. The non-transitory computer-readable medium or media according to claim 8 wherein the first sensing device and the second communication-and-control device are housed in an enclosure that is designed to be handheld and used in mid-air.

14. The non-transitory computer-readable medium or media according to claim 13,

wherein said enclosure comprises a manipulation device coupled to said second communication-and-control device and designed to allow a user to manipulate two independent positional coordinates of the feature on the computer screen interaction region, said manipulation device for generating a third output signal that is independent of orientation of said enclosure, and

wherein said first data depends on said third output signal.

15. The non-transitory computer-readable medium or media according to claim 14, wherein said manipulation device is a thumbstick.

16. The non-transitory computer-readable medium or media according to claim 13, wherein said first sensing device furthermore comprises an image capturing device for providing a third output signal that is sensitive to orientation of the first sensing device, and wherein said first data depends on said third output signal.

17. The non-transitory computer-readable medium or media according to claim 13,

wherein said first communication-and-control device is designed to be coupled to a third communication-and-control device that is for providing third data to the first communication-and-control device, said third data being dependent on position of the enclosure, and

wherein the method further comprises the steps of:

developing fourth data that is dependent on the third data; and

using said fourth data to control the feature on the computer screen interaction region.

18. A non-transitory computer-readable medium or media storing computer-executable instructions for causing a computer to perform a method for controlling contents of a computer-generated image in conjunction with a first sensing device that is coupled to said computer and in conjunction with an enclosure that is designed to be handheld by a user in mid-air and that contains a second sensing device comprising an accelerometer and a gyro, said second sensing device for generating orientation data that is dependent on orientation of said enclosure, the method comprising the steps of:

querying the user to align said enclosure in a way that allows said first sensing device to generate first data that is dependent on the distance between the enclosure and the computer-generated image,

causing said first sensing device to generate the first data, and

using the first data to control the contents of the computer-generated image.

19. The non-transitory computer-readable medium or media according to claim 18,

wherein said enclosure is associated with a pointing line that intersects the enclosure and has a fixed orientation relative to the second sensing device, and

wherein said querying the user is querying the user to align the enclosure in such a way that said pointing line is substantially perpendicular to the computer-generated image.

20. The non-transitory computer-readable medium or media according to claim 18, wherein said first sensing device has components contained in a base station that is positioned in and aligned with a plane that contains the computer-generated image.