Computer Control with Heads-Up Display

• US 20100007580A1
• Filed: 07/14/2008
• Published: 01/14/2010
• Est. Priority Date: 07/14/2008
• Status: Active Grant

• Current Assignee: Science Applications International Corporation
• Original Assignee: Science Applications International Corporation
• Inventors: Scales, John Richard
• Granted Patent: US 9,217,866 B2
• Time in Patent Office: Days
• US Class Current: 345/8
• CPC Class Codes: F41G 3/165 using a TV-monitor
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0189 Sight systems

Abstract

A method, computer-readable medium and apparatus are described for providing a user interface control for use with a manipulable object, such as a weapon. As the object is moved, images from a video camera are projected into a heads-up display. A user of the object may activate a user interface control by pressing a button on the object, causing an image of a user interface control to be projected into the heads-up display. Movement of the object may cause a cursor to be moved across the heads-up display, allowing the user to select user interface features without removing hands from the object. Pressing a button may cause a mouse click command to be generated, allowing the user to type or perform other functions.

Claims

1. A method comprising:

generating a video image corresponding to at least a portion of a field of view of a heads-up display;

superimposing the video image into the field of view of the heads-up display;

detecting activation of a first control mounted on a manipulable object and, in response thereto, generating an image of a computer control function in the heads-up display;

generating a cursor on the image of the computer control function; and

causing the cursor to move in response to movement of the object.

2. The method of claim 1, further comprising:

detecting activation of a second control mounted on the object and, in response thereto, generating a mouse click command.

3. The method of claim 2, further comprising:

detecting a second activation of the first control and, in response thereto, removing the image of the computer control function from the heads-up display.

4. The method of claim 1, wherein the object comprises a weapon.

5. The method of claim 1, wherein the image of the computer control function is generated for projection in only one eye of the heads-up display.

6. The method of claim 1, wherein the image of the computer control function is superimposed into the field of view of both eyes of the heads-up display.

7. The method of claim 1, wherein detecting activation of the first control comprises detecting that a first button was pressed.

8. The method of claim 2, wherein detecting activation of the second control comprises detecting that a second of two buttons was pressed.

9. The method of claim 1, wherein the image of the computer control function is generated in only one eye of the heads-up display, and wherein the video image is displayed in only the other eye of the heads-up display.

10. One or more computer-readable media comprising executable instructions that, when executed by a processor, perform:

generating a video image corresponding to at least a portion of a field of view of a heads-up display;

causing the video image to be superimposed into the field of view of the heads-up display;

detecting activation of a first control mounted on a manipulable object and, in response thereto, causing to be generated an image of a computer control function in the heads-up display;

generating a cursor on the image of the computer control function; and

causing the cursor to move in response to movement of the object.

11. The one or more computer-readable media of claim 10, wherein the executable instructions further perform:

detecting activation of a second control mounted on the object and, in response thereto, generating a mouse click command.

12. The one or more computer-readable media of claim 11, wherein the executable instructions further perform:

detecting a second activation of the first control and, in response thereto, removing the image of the computer control function from the heads-up display.

13. The one or more computer-readable media of claim 10, wherein the image of the computer control function is generated for projection in only one eye of the heads-up display.

14. The one or more computer-readable media of claim 10, wherein the image of the computer control function is superimposed into the field of view of both eyes of the heads-up display.

15. The one or more computer-readable media of claim 10, wherein the instructions detect that a first button was pressed in order to detect activation of the first control.

16. The computer-readable media of claim 11, wherein detecting activation of the second control comprises detecting that a second of two buttons was pressed.

17. The computer-readable media of claim 10, wherein the image of the computer control function is generated in only one eye of the heads-up display, and wherein the video image is displayed in only the other eye of the heads-up display.

18. An apparatus comprising:

a computer configured to accept input from a video source and to generate a first video image for output to a heads-up display and to generate a second image of a computer control function;

one or more switches coupled to the computer to indicate user input from a manipulable object, and

a memory storing instructions that, when executed by the computer, perform;

causing to be generated the first video image for output to at least a portion of a field of view of the heads-up display;

detecting activation of a first switch and, in response thereto, causing to be generated an image of a computer control function in the heads-up display;

generating a cursor on the image of the computer control function; and

causing the cursor to move as the manipulable object is moved.

19. The apparatus of claim 18, wherein the instructions further perform:

detecting activation of a second switch and, in response thereto, generating a mouse click command.

20. The apparatus of claim 19, wherein the executable instructions further perform:

detecting a second activation of the first switch and, in response thereto, removing the image of the computer control function from the heads-up display.

21. The apparatus of claim 18, wherein the image of the computer control function is generated for projection in only one eye of the heads-up display.

22. The apparatus of claim 18, wherein the image of the computer control function is superimposed into the field of view of both eyes of the heads-up display.

23. The apparatus of claim 18, wherein the image of the computer control function is generated in only one eye of the heads-up display, and wherein the video image is displayed in only the other eye of the heads-up display.

24. A weapon system comprising:

a weapon having mounted thereon at least one switch, a video camera, and a measurement device configured to output signals indicative of movement of the weapon;

a heads-up display providing a field of view and configured to project a first image into a first eye of a wearer; and

a computer programmed with executable instructions that, when executed by the computer, perform;

outputting the first image to the heads-up display;

detecting activation of the at least one switch and, in response thereto, replacing the first image with an image of a computer control function including a cursor;

detecting movement of the weapon from the measurement device; and

causing the cursor to move on the image of the computer control function in response to movement of the weapon.

25. The weapon system of claim 24, wherein the executable instructions further perform:

detecting activation of a second switch mounted on the weapon and, in response thereto, generating a mouse click command.

26. The weapon system of claim 25, wherein the executable instructions further perform:

detecting a second activation of the first switch and, in response thereto, removing the image of the computer control function from the heads-up display.

27. A kit comprising:

a heads-up display;

a switch plate configured to be mounted to a weapon and including at least one switch;

a computer configured to receive input from the switch plate and from one or more position sensors, and to output a video image into a field of view of the heads-up display; and

a memory storing instructions that, when executed by the computer, perform;

causing a video image to be superimposed into the field of view of the heads-up display;

detecting activation of the at least one switch and, in response thereto, generating an image of a computer control function in the heads-up display;

generating a cursor on the image of the computer control function; and

causing the cursor to move in response to movement sensed from the one or more position sensors.