Apparatus for controlling contents of a computer-generated image using 3D measurements

• US 10,372,237 B2
• Filed: 02/03/2016
• Issued: 08/06/2019
• Est. Priority Date: 07/13/2005
• Status: Active Grant

• Current Assignee: UltimatePointer, LLC
• Original Assignee: UltimatePointer, LLC
• Inventors: Banning, Erik Jan
• Primary Examiner(s): Pham, Viet D
• Application Number:
  US15/014,879
• Publication Number:
  US 20160299575A1
• Time in Patent Office: 1,280 Days
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A method for controlling a computer display cursor in an interaction region includes projecting an image of the computer display to create the interaction region. A distance is established between a first point and a second point. The first point has a predetermined relation to the projection device, and the second point has a predetermined relation to the interaction region. At least one of an orientation and a position of a pointing line is measured. The pointing line has a predetermined relation to a pointing device. The established distance and the at least one of measured position and orientation are used to control the cursor position on the interaction region.

Claims

1. A non-transitory computer-readable medium or media storing computer-executable instructions for causing a computer to perform a method for controlling contents of a computer-generated image, the method comprising the steps of:

communicating with and receiving first data from a first communication device coupled to a first sensing device, the first data being dependent on a first output signal of the first sensing device, the first output signal being sensitive to three independent orientational coordinates and three independent positional coordinates of a first device, the first device for generating a pattern of light representing said computer-generated image, the first device comprising at least one element of the first sensing device and an image capturing device coupled to the first communication device and for generating a third output signal;

communicating with and receiving second data from a second communication device coupled to a second sensing device, the second data being dependent on a second output signal of the second sensing device, the second output signal being sensitive to three independent orientational coordinates and three independent positional coordinates of a second device that is different from the first device, the second device configured to be wielded in mid-air by a user, the second device comprising at least one element of the second sensing device; and

using said first and second data to control the contents of the computer-generated image,

wherein at least one of the first device and the second device is configured to be worn by the user as a head-set, and

wherein the method further comprises the steps of;

receiving third data from the first communication device, said third data being dependent on the third output signal; and

using said third data to control the contents of the computer-generated image.

2. The non-transitory computer-readable medium or media according to claim 1, wherein the first device is a projection device for projecting said computer-generated image on a projection region, and

wherein the second device is configured to be worn as a ring by the user.

3. The non-transitory computer-readable medium or media according to claim 1, wherein the first device is a display device for generating a visible representation of said computer-generated image, and

wherein the second device is configured to be handheld by the user.

4. The non-transitory computer-readable medium or media according to claim 3, wherein the first sensing device comprises a first gyro contained in the first device, and

wherein the second sensing device comprises a second gyro contained in the second device.

5. The non-transitory computer-readable medium or media according to claim 1, wherein the first device is configured to be worn as a head-set by the user, and

wherein the second device is configured to be handheld by the user.

6. An apparatus for controlling contents of an image generated by a computer, the apparatus comprising:

a first device for generating a pattern of light representing the image generated by the computer, said first device comprising at least one component of a first sensing device for generating first data for controlling the contents of the image generated by the computer, said at least one component of said first sensing device being sensitive to three independent orientational coordinates and three independent positional coordinates of said first device, the three independent positional coordinates being relative to a base station that comprises at least one component of said first sensing device, the base station separate from the first device.

7. The apparatus according to claim 6, further comprising

a second device that is different from the first device and configured to be wielded in mid-air by a user, said second device comprising at least one component of a second sensing device for generating second data for controlling the contents of the image generated by the computer, said at least one component of said second sensing device being sensitive to at least two independent positional coordinates of said second device relative to the base station, wherein the base station further comprises at least one component of said second sensing device and wherein the base station is separate from the second device.

8. The apparatus according to claim 7, wherein said second data is sensitive to three independent positional coordinates of said second device relative to the base station and three independent orientational coordinates of said second device, and

wherein at least one of said first device and said second device is configured to be worn by the user as a head-set.

9. The apparatus according to claim 6, wherein the apparatus further comprises:

a second device that is different from the first device and configured to be handheld and wielded in mid-air by a user, the second device comprising a user input device for generating second data for controlling the contents of the image generated by the computer, said user input device configured to allow the user to manipulate two positional coordinates of a feature on the image, wherein the second data is independent of the orientation and position of the second device.

10. The apparatus according to claim 9, wherein said second device comprises at least one component of a second sensing device for generating third data for controlling the contents of the image generated by the computer, said at least one component of said second sensing device being sensitive to three independent positional coordinates and three independent orientational coordinates of said second device, the three positional coordinates being relative to the base station.

11. The apparatus according to claim 10, wherein the first device is a display for displaying the image generated by the computer, and

wherein the user input device is at least one of a thumbstick and a trackpad.

12. The apparatus according to claim 9, wherein the first device is a projection device for projecting the image generated by the computer onto a projection region;

wherein the user input device is a trackball; and

wherein the apparatus further comprises a non-transitory computer-readable medium or media storing computer-executable instructions for causing the computer to perform a method for controlling the contents of the image, the method comprising the step of using the first data to cause the projection of the image to be rectangular.

13. The apparatus according to claim 6, wherein the first sensing device comprises a gyro, an accelerometer and a magnetometer, said gyro, said accelerometer and said magnetometer contained in said first device.

14. A non-transitory computer-readable medium or media storing computer-executable instructions for causing a computer to perform a method for controlling contents of a computer-generated image, the method comprising the steps of:

receiving first data that is dependent on a first output from a first sensing device, the first output being sensitive to three independent positional and three independent orientational coordinates of a first device, the first device for generating a pattern of light representing said computer-generated image, the first device comprising at least one element of the first sensing device;

receiving second data that is dependent on a second output from a user input device configured to allow a user to manipulate two positional coordinates of a feature on the image, the user input device contained in a second device that is different from the first device and configured to be handheld and wielded by the user in mid-air and on a third output from a second sensing device of which at least one element is contained in the second device, the third output being sensitive to three independent positional and three independent orientational coordinates of the second device; and

controlling the contents of the computer-generated image based at least in part on said first data and second data,

wherein the three positional coordinates of the first device and the three positional coordinates of the second device are relative to a base station, the base station comprising at least one element of the first sensing device and at least one element of the second sensing device, the base station separate from the first device and the second device.

15. The non-transitory computer-readable medium or media according to claim 14, wherein the first sensing device comprises an accelerometer, a gyro and a magnetometer, the accelerometer, the gyro and the magnetometer contained in the first device;

wherein the first device is a display for displaying said image; and

wherein the three independent positional coordinates of the first device are relative to a base station that comprises at least one element of the first sensing device.

16. The non-transitory computer-readable medium or media according to claim 14, wherein the three independent positional coordinates of the first device are relative to a base station that comprises at least one element of the first sensing device, the base station further comprising a third sensing device for generating a third output that is sensitive to an angle of the base station relative to gravity, wherein the base station is separate from the first device and the second device;

and

wherein the method further comprises the steps of;

receiving third data that is dependent on the third output signal, and

controlling the contents of the computer-generated image based at least in part on said third data.

17. A non-transitory computer-readable medium or media storing computer-executable instructions for causing a computer to perform a method for controlling contents of a computer-generated image, the method comprising the steps of:

communicating with and receiving first data from a first communication device coupled to a first sensing device, the first data being dependent on a first output signal of the first sensing device, the first output signal being sensitive to three independent orientational coordinates and three independent positional coordinates of a first device, the first device for generating a pattern of light representing said computer-generated image, the first device comprising at least one element of the first sensing device;

communicating with and receiving second data from a second communication device coupled to a second sensing device, the second data being dependent on a second output signal of the second sensing device, the second output signal being sensitive to three independent orientational coordinates and three independent positional coordinates of a second device that is different from the first device, the second device configured to be wielded in mid-air by a user, the second device comprising at least one element of the second sensing device; and

using said first and second data to control the contents of the computer-generated image,

wherein at least one of the first device and the second device is configured to be worn by the user as a head-set,

wherein the three independent positional coordinates of the first device are relative to a base station,

wherein the three independent positional coordinates of the second device are relative to the base station,

wherein the base station comprises at least one of a first power source and a connection to a second power source for providing power to the base station,

wherein the base station is separate from the first device and the second device,

wherein the first sensing device further comprises at least one element contained in the base station, and

wherein the second sensing device further comprises at least one element contained in the base station.

18. The non-transitory computer-readable medium or media according to claim 17, wherein the first device is a display device for generating a visible representation of said computer-generated image, and

wherein the second device is configured to be handheld by the user.

19. The non-transitory computer-readable medium or media according to claim 17, wherein the first sensing device comprises a first gyro contained in the first device, and

wherein the second sensing device comprises a second gyro contained in the second device.